LREC 2006 - Proceedings sorted by papers

Title	Using Semantic Overlap Scoring in Answering TREC Relationship Questions
Authors	G. Marton, B. Katz
Abstract	A first step in answering complex questions, such as those in the “Relationship”' task of the Text REtrieval Conference's Question Answering track (TREC/QA), is finding passages likely to contain pieces of the answer---passage retrieval. We introduce semantic overlap scoring, a new passage retrieval algorithm that facilitates credit assignment for inexact matches between query and candidate answer. Our official submission ranked best among fully automatic systems, at 23% F-measure, while the best system, with manual input, reached 28%. We use our Nuggeteer tool to robustly evaluate each component of our Relationship system post hoc. Ablation studies show that semantic overlap scoring achieves significant performance improvements over a standard passage retrieval baseline.
Keywords
Full paper	Using Semantic Overlap Scoring in Answering TREC Relationship Questions

Title

Using Semantic Overlap Scoring in Answering TREC Relationship Questions

Authors

G. Marton, B. Katz

Abstract

A first step in answering complex questions, such as those in the “Relationship”' task of the Text REtrieval Conference's Question Answering track (TREC/QA), is finding passages likely to contain pieces of the answer---passage retrieval. We introduce semantic overlap scoring, a new passage retrieval algorithm that facilitates credit assignment for inexact matches between query and candidate answer. Our official submission ranked best among fully automatic systems, at 23% F-measure, while the best system, with manual input, reached 28%. We use our Nuggeteer tool to robustly evaluate each component of our Relationship system post hoc. Ablation studies show that semantic overlap scoring achieves significant performance improvements over a standard passage retrieval baseline.

Keywords

Full paper

Using Semantic Overlap Scoring in Answering TREC Relationship Questions