| Title | Agreement in Human Factoid Annotation for Summarization Evaluation |
| Author(s) |
Simone Teufel (1), Hans van Halteren (2)
(1) Computer Laboratory, University of Cambridge,UK; (2) Department of Language and Speech, University of Nijmegen, The Netherlands |
| Session | O39-EW |
| Abstract | Factoid analysis was introduced by vanHalteren and Teufel (2003) as an objective, yet semantics-oriented way of measuring overlap of information rather than surface strings in summaries. In this paper, we report on annotation experiments with two sets of summaries, and on a factoid-pairing program which finds correlations between factoids semi-automatically. |
| Keyword(s) | Evaluation, summarization, semantic overlap |
| Language(s) | English |
| Full Paper | 723.pdf |