Summary of the paper
| Title | Using semi-experts to derive judgments on word sense alignment: a pilot study |
|---|---|
| Authors | Soojeong Eom, Markus Dickinson and Graham Katz |
| Abstract | The overall goal of this project is to evaluate the performance of word sense alignment (WSA) systems, focusing on obtaining examples appropriate to language learners. Building a gold standard dataset based on human expert judgments is costly in time and labor, and thus we gauge the utility of using semi-experts in performing the annotation. In an online survey, we present a sense of a target word from one dictionary with senses from the other dictionary, asking for judgments of relatedness. We note the difficulty of agreement, yet the utility in using such results to evaluate WSA work. We find that one's treatment of related senses heavily impacts the results for WSA. |
| Topics | Word Sense Disambiguation, Evaluation methodologies, Lexicon, lexical database |
Full paper  |
Using semi-experts to derive judgments on word sense alignment: a pilot study |
| Bibtex | @InProceedings{EOM12.652, author = {Soojeong Eom and Markus Dickinson and Graham Katz}, title = {Using semi-experts to derive judgments on word sense alignment: a pilot study}, booktitle = {Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12)}, year = {2012}, month = {may}, date = {23-25}, address = {Istanbul, Turkey}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Mehmet Uğur Doğan and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-7-7}, language = {english} } |