Summary of the paper
| Title | The QALL-ME Benchmark: a Multilingual Resource of Annotated Spoken Requests for Question Answering |
|---|---|
| Authors | Elena Cabrio, Milen Kouylekov, Bernardo Magnini, Matteo Negri, Laura Hasler, Constantin Orasan, David Tomas, Jose Luis Vicedo, Guenter Neumann and Corinna Weber |
| Abstract | This paper presents the QALL-ME benchmark, a multilingual resource of annotated spoken requests in the tourism domain, freely available for research purposes. The languages currently involved in the project are Italian, English, Spanish and German. It introduces a semantic annotation scheme for spoken information access requests, specifically derived from Question Answering (QA) research. In addition to pragmatic and semantic annotations, we propose three QA-based annotation levels: the Expected Answer Type, the Expected Answer Quantifier and the Question Topical Target of a request, to fully capture the content of a request and extract the sought-after information. The QALL-ME benchmark is developed under the EU-FP6 QALL-ME project which aims at the realization of a shared and distributed infrastructure for Question Answering (QA) systems on mobile devices (e.g. mobile phones). Questions are formulated by the users in free natural language input, and the system returns the actual sequence of words which constitutes the answer from a collection of information sources (e.g. documents, databases). Within this framework, the benchmark has the twofold purpose of training machine learning based applications for QA, and testing their actual performance with a rapid turnaround in controlled laboratory setting. |
| Language | Multiple languages |
| Topics | Question Answering, Multilinguality, Corpus (creation, annotation, etc.) |
Full paper |
The QALL-ME Benchmark: a Multilingual Resource of Annotated Spoken Requests for Question Answering |
Slides |
The QALL-ME Benchmark: a Multilingual Resource of Annotated Spoken Requests for Question Answering |
| Bibtex | @InProceedings{CABRIO08.628, author = {Elena Cabrio, Milen Kouylekov, Bernardo Magnini, Matteo Negri, Laura Hasler, Constantin Orasan, David Tomas, Jose Luis Vicedo, Guenter Neumann and Corinna Weber}, title = {The QALL-ME Benchmark: a Multilingual Resource of Annotated Spoken Requests for Question Answering}, booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)}, year = {2008}, month = {may}, date = {28-30}, address = {Marrakech, Morocco}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-4-0}, note = {http://www.lrec-conf.org/proceedings/lrec2008/}, language = {english} } |