Summary of the paper
| Title | Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space |
|---|---|
| Authors | Joana Hois |
| Abstract | In this paper, we present a case study for measuring inter-annotator agreement on a linguistic ontology for spatial language, namely the spatial extension of the Generalized Upper Model. This linguistic ontology specifies semantic categories, and it is used in dialogue systems for natural language of space in the context of human-computer interaction and spatial assistance systems. Its core representation for spatial language distinguishes how sentences can be structured and categorized into units that contribute certain meanings to the expression. This representation is here evaluated in terms of inter-annotator agreement: four uninformed annotators were instructed by a manual how to annotate sentences with the linguistic ontology. They have been assigned to annotate 200 sentences with varying length and complexity. Their resulting agreements are calculated together with our own 'expert annotation' of the same sentences. We show that linguistic ontologies can be evaluated with respect to inter-annotator agreement, and we present encouraging results of calculating agreements for the spatial extension of the Generalized Upper Model. |
| Topics | Evaluation methodologies, Ontologies, Language modelling |
Full paper |
Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space |
Slides |
- |
| Bibtex | @InProceedings{HOIS10.231, author = {Joana Hois}, title = {Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |