Summary of the paper
| Title | Towards Semi-Automated Annotation for Prepositional Phrase Attachment |
|---|---|
| Authors | Sara Rosenthal, William Lipovsky, Kathleen McKeown, Kapil Thadani and Jacob Andreas |
| Abstract | This paper investigates whether high-quality annotations for tasks involving semantic disambiguation can be obtained without a major investment in time or expense. We examine the use of untrained human volunteers from Amazons Mechanical Turk in disambiguating prepositional phrase (PP) attachment over sentences drawn from the Wall Street Journal corpus. Our goal is to compare the performance of these crowdsourced judgments to the annotations supplied by trained linguists for the Penn Treebank project in order to indicate the viability of this approach for annotation projects that involve contextual disambiguation. The results of our experiments on a sample of the Wall Street Journal corpus show that invoking majority agreement between multiple human workers can yield PP attachments with fairly high precision. This confirms that a crowdsourcing approach to syntactic annotation holds promise for the generation of training corpora in new domains and genres where high-quality annotations are not available and difficult to obtain. |
| Topics | Corpus (creation, annotation, etc.), Grammar and Syntax, Evaluation methodologies |
Full paper |
Towards Semi-Automated Annotation for Prepositional Phrase Attachment |
Slides |
Towards Semi-Automated Annotation for Prepositional Phrase Attachment |
| Bibtex | @InProceedings{ROSENTHAL10.828, author = {Sara Rosenthal and William Lipovsky and Kathleen McKeown and Kapil Thadani and Jacob Andreas}, title = {Towards Semi-Automated Annotation for Prepositional Phrase Attachment}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |