Summary of the paper
| Title | A Python Toolkit for Universal Transliteration |
|---|---|
| Authors | Ting Qian, Kristy Hollingshead, Su-youn Yoon, Kyoung-young Kim and Richard Sproat |
| Abstract | We describe ScriptTranscriber, an open source toolkit for extracting transliterations in comparable corpora from languages written in different scripts. The system includes various methods for extracting potential terms of interest from raw text, for providing guesses on the pronunciations of terms, and for comparing two strings as possible transliterations using both phonetic and temporal measures. The system works with any script in the Unicode Basic Multilingual Plane and is easily extended to include new modules. Given comparable corpora, such as newswire text, in a pair of languages that use different scripts, ScriptTranscriber provides an easy way to mine transliterations from the comparable texts. This is particularly useful for underresourced languages, where training data for transliteration may be lacking, and where it is thus hard to train good transliterators. ScriptTranscriber provides an open source package that allows for ready incorporation of more sophisticated modules ― e.g. a trained transliteration model for a particular language pair. ScriptTranscriber is available as part of the nltk contrib source tree at http://code.google.com/p/nltk/. |
| Topics | Tools, systems, applications, Multilinguality, Named Entity recognition |
Full paper |
A Python Toolkit for Universal Transliteration |
Slides |
A Python Toolkit for Universal Transliteration |
| Bibtex | @InProceedings{QIAN10.30, author = {Ting Qian and Kristy Hollingshead and Su-youn Yoon and Kyoung-young Kim and Richard Sproat}, title = {A Python Toolkit for Universal Transliteration}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |