Summary of the paper
| Title | Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation? |
|---|---|
| Authors | Susan Windisch Brown, Travis Rood and Martha Palmer |
| Abstract | This study attempts to pinpoint the factors that restrict reliable word sense annotation, focusing on the influence of the number of senses annotators use and the semantic granularity of those senses. Both of these factors may be possible causes of low interannotator agreement (ITA) when tagging with fine-grained word senses, and, consequently, low WSD system performance (Ng et al., 1999; Snyder & Palmer, 2004; Chklovski & Mihalcea, 2002). If number of senses is the culprit, modifying the task to show fewer senses at a time could improve annotator reliability. However, if overly nuanced distinctions are the problem, then more general, coarse-grained distinctions may be necessary for annotator success and may be all that is needed to supply systems with the types of distinctions that people make. We describe three experiments that explore the role of these factors in annotation performance. Our results indicate that of these two factors, only the granularity of the senses restricts interannotator agreement, with broader senses resulting in higher annotation reliability. |
| Topics | Word Sense Disambiguation, Corpus (creation, annotation, etc.), Semantics |
Full paper |
Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation? |
Slides |
Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation? |
| Bibtex | @InProceedings{BROWN10.927, author = {Susan Windisch Brown and Travis Rood and Martha Palmer}, title = {Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation?}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |