Summary of the paper
| Title | Annotating Students’ Understanding of Science Concepts |
|---|---|
| Authors | Rodney D. Nielsen, Wayne Ward, James Martin and Martha Palmer |
| Abstract | This paper summarizes the annotation of fine-grained entailment relationships in the context of student answers to science assessment questions. We annotated a corpus of 15,357 answer pairs with 145,911 fine-grained entailment relationships. We provide the rationale for such fine-grained analysis and discuss its perceived benefits to an Intelligent Tutoring System. The corpus also has potential applications in other areas, such as question answering and multi-document summarization. Annotators achieved 86.2% inter-annotator agreement (Kappa=0.728, corresponding to substantial agreement) annotating the fine-grained facets of reference answers with regard to understanding expressed in student answers and labeling from one of five possible detailed relationship categories. The corpus described in this paper, which is the only one providing such detailed entailment annotations, is available as a public resource for the research community. The corpus is expected to enable application development, not only for intelligent tutoring systems, but also for general textual entailment applications, that is currently not practical. |
| Language | |
| Topics | Corpus (creation, annotation, etc.), Semantics, Dialogue & Natural Interactivity |
Full paper |
Annotating Students’ Understanding of Science Concepts |
Slides |
Annotating Students’ Understanding of Science Concepts |
| Bibtex | @InProceedings{NIELSEN08.873, author = {Rodney D. Nielsen, Wayne Ward, James Martin and Martha Palmer}, title = {Annotating Students’ Understanding of Science Concepts}, booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)}, year = {2008}, month = {may}, date = {28-30}, address = {Marrakech, Morocco}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-4-0}, note = {http://www.lrec-conf.org/proceedings/lrec2008/}, language = {english} } |