Summary of the paper
| Title | Online Japanese Unknown Morpheme Detection using Orthographic Variation |
|---|---|
| Authors | Yugo Murawaki and Sadao Kurohashi |
| Abstract | To solve the unknown morpheme problem in Japanese morphological analysis, we previously proposed a novel framework of online unknown morpheme acquisition and its implementation. This framework poses a previously unexplored problem, online unknown morpheme detection. Online unknown morpheme detection is a task of finding morphemes in each sentence that are not listed in a given lexicon. Unlike in English, it is a non-trivial task because Japanese does not delimit words by white space. We first present a baseline method that simply uses the output of the morphological analyzer. We then show that it fails to detect some unknown morphemes because they are over-segmented into shorter registered morphemes. To cope with this problem, we present a simple solution, the use of orthographic variation of Japanese. Under the assumption that orthographic variants behave similarly, each over-segmentation candidate is checked against its counterparts. Experiments show that the proposed method improves the recall of detection and contributes to improving unknown morpheme acquisition. |
| Topics | Acquisition, Lexicon, lexical database, Tools, systems, applications |
Full paper |
Online Japanese Unknown Morpheme Detection using Orthographic Variation |
Slides |
- |
| Bibtex | @InProceedings{MURAWAKI10.171, author = {Yugo Murawaki and Sadao Kurohashi}, title = {Online Japanese Unknown Morpheme Detection using Orthographic Variation}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |