LREC 2000 - Papers

LREC 2000 2^nd International Conference on Language Resources & Evaluation

Conference Papers

Papers by paper title: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Papers by ID number: 1-50, 51-100, 101-150, 151-200, 201-250, 251-300, 301-350, 351-377.

List of all papers and abstracts.

Previous Paper Next Paper

Title Extraction of Unknown Words Using the Probability of Accepting the Kanji Character Sequence as One Word

Authors Shinnou Hiroyuki (Ibaraki University Dept. of Systems Engineering, 4-12-1 Nakanarusawa, Hitachi, Ibaeaki, 216-8511, Japan, shinnou@nlp.dse.ibaraki.ac.jp)
Ikeya Masanori (Ibaraki University Dept. of Systems Engineering, 4-12-1 Nakanarusawa, Hitachi, Ibaeaki, 216-8511, Japan, ikeya@nlp.dse.ibaraki.ac.jp)

Keywords

Session Session WO9 - Applications in the Written Area

Abstract In this paper, we propose a method to extract unknown words, which are composed of two or three kahji characters, from Japanase text. Generally the known word composed of kanji characters are segmented into other words by the morphological analysis. Moreover, the appearance probability of each segmented word is small. By these features, we can define the measure of accepting two or three kanji character sequence as an unknown word. On the other hand, we can find some segmentation patterns of unknown words. By applying our measure to kanji character sequences which have these patterns, we can extract unknown words. In the experiment, the F-measuer for extraction of known words composed of two and three kanji characters was about 0.7 and 0.4 respectively. Our method does not need to use the frequency of the word in the training corpus to judge whether its word is the unknown word or not. Therefore, our method has the advantage that low frequent unknown words are extracted.

Title	Extraction of Unknown Words Using the Probability of Accepting the Kanji Character Sequence as One Word
Authors	Shinnou Hiroyuki (Ibaraki University Dept. of Systems Engineering, 4-12-1 Nakanarusawa, Hitachi, Ibaeaki, 216-8511, Japan, shinnou@nlp.dse.ibaraki.ac.jp) Ikeya Masanori (Ibaraki University Dept. of Systems Engineering, 4-12-1 Nakanarusawa, Hitachi, Ibaeaki, 216-8511, Japan, ikeya@nlp.dse.ibaraki.ac.jp)
Keywords
Session	Session WO9 - Applications in the Written Area
Abstract	In this paper, we propose a method to extract unknown words, which are composed of two or three kahji characters, from Japanase text. Generally the known word composed of kanji characters are segmented into other words by the morphological analysis. Moreover, the appearance probability of each segmented word is small. By these features, we can define the measure of accepting two or three kanji character sequence as an unknown word. On the other hand, we can find some segmentation patterns of unknown words. By applying our measure to kanji character sequences which have these patterns, we can extract unknown words. In the experiment, the F-measuer for extraction of known words composed of two and three kanji characters was about 0.7 and 0.4 respectively. Our method does not need to use the frequency of the word in the training corpus to judge whether its word is the unknown word or not. Therefore, our method has the advantage that low frequent unknown words are extracted.