LREC 2006 - Proceedings sorted by papers

Title	Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering
Authors	H. Shima, M. Wang, F. Lin, T. Mitamura
Abstract	Multilingual Question Answering systems are generally very complex, integrating several sub-modules to achieve their result. Global metrics (such as average precision and recall) are insufficient when evaluating the performance of individual sub-modules and their influence on each other. In this paper, we present a modular approach to error analysis and evaluation; we use manually-constructed, gold-standard input for each module to obtain an upper-bound for the (local) performance of that module. This approach enables us to identify existing problem areas quickly, and to target improvements accordingly.
Keywords	Multilingual Question AnsweringEvaluation
Full paper	Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering

Title

Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering

Authors

H. Shima, M. Wang, F. Lin, T. Mitamura

Abstract

Multilingual Question Answering systems are generally very complex, integrating several sub-modules to achieve their result. Global metrics (such as average precision and recall) are insufficient when evaluating the performance of individual sub-modules and their influence on each other. In this paper, we present a modular approach to error analysis and evaluation; we use manually-constructed, gold-standard input for each module to obtain an upper-bound for the (local) performance of that module. This approach enables us to identify existing problem areas quickly, and to target improvements accordingly.

Keywords

Multilingual Question AnsweringEvaluation

Full paper

Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering