Title

Evaluating Web-based Question Answering Systems

Authors

Dragomir R. Radev (School of Information University of Michigan Ann Arbor,MI 48109; Department of EECS University of Michigan Ann Arbor,MI 48109)

Hong Qi (School of Information University of Michigan Ann Arbor,MI 48109)

HarrisWu (Business School University of Michigan Ann Arbor,MI 48109)

Weiguo Fan (Business School University of Michigan Ann Arbor,MI 48109)

Session

D1: DEMOS

Abstract

The official evaluation of TREC-style Q&A systems is done manually, which is quite expensive and not scalable to web-based Q&A systems. An automatic evaluation technique is needed for dynamic Q&A systems. This paper presents a set of metrics that have been implemented in our web-based Q&A system, namely NSIR. It also shows the correlations between the different metrics.

Keywords

Question answering

Full Paper

301.pdf