SUMMARY : Session O26-W Question Answering

 

Title Using Semantic Overlap Scoring in Answering TREC Relationship Questions
Authors G. Marton, B. Katz
Abstract A first step in answering complex questions, such as those in the “Relationship”' task of the Text REtrieval Conference's Question Answering track (TREC/QA), is finding passages likely to contain pieces of the answer---passage retrieval. We introduce semantic overlap scoring, a new passage retrieval algorithm that facilitates credit assignment for inexact matches between query and candidate answer. Our official submission ranked best among fully automatic systems, at 23% F-measure, while the best system, with manual input, reached 28%. We use our Nuggeteer tool to robustly evaluate each component of our Relationship system post hoc. Ablation studies show that semantic overlap scoring achieves significant performance improvements over a standard passage retrieval baseline.
Keywords
Full paper Using Semantic Overlap Scoring in Answering TREC Relationship Questions