LREC 2000 - Papers

LREC 2000 2^nd International Conference on Language Resources & Evaluation

Conference Papers

Papers by paper title: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Papers by ID number: 1-50, 51-100, 101-150, 151-200, 201-250, 251-300, 301-350, 351-377.

List of all papers and abstracts.

Previous Paper Next Paper

Title Textual Information Retrieval Systems Test: The Point of View of an Organizer and Corpuses Provider

Authors Kremer Patrick (INIST / CNRS, 2, allee de Brabois, 54514 Vandoeuvre-les- Nancy, patrick.kremer@inist.fr)
Schmitt Laurent (INIST / CNRS, 2, allee de Brabois, 54514 Vandoeuvre-les- Nancy, laurent.schmitt@inist.fr)

Keywords

Session Session EO5 - Information Retrieval and Question Answering Evaluation

Abstract Amaryllis is an evaluation programme for text retrieval systems which has been carried out as two test campaigns. The second Amaryllis campaign took place in 1998/1999. Corpuses of documents, topics, and the corresponding responses were first sent to each of the participating teams for system learning purposes. Corpuses of new documents and a set of new topics were then supplied for evaluation purposes. Two optional tracks were added for Internet and interlingual track. The first track of these contained a test via the Internet. INIST sent topics to the system and collected responses directly, thus reducing the need for conceptor manipulations. The second contained tests in different European Community language pairs. The corpuses of documents consisted of records of questions and answers from the European Commission, in parallel official language versions. Participants could use any language pair for their tests. The aim of this paper is to give the point of view of an organizer and corpus provider (INIST) on the organization of an operation of this sort. In particular, it will describe the difficulties encountered during the tests (corpus construction, translation of topics and systems evaluation ), and will suggest avenues to explore for future tests.

Title	Textual Information Retrieval Systems Test: The Point of View of an Organizer and Corpuses Provider
Authors	Kremer Patrick (INIST / CNRS, 2, allee de Brabois, 54514 Vandoeuvre-les- Nancy, patrick.kremer@inist.fr) Schmitt Laurent (INIST / CNRS, 2, allee de Brabois, 54514 Vandoeuvre-les- Nancy, laurent.schmitt@inist.fr)
Keywords
Session	Session EO5 - Information Retrieval and Question Answering Evaluation
Abstract	Amaryllis is an evaluation programme for text retrieval systems which has been carried out as two test campaigns. The second Amaryllis campaign took place in 1998/1999. Corpuses of documents, topics, and the corresponding responses were first sent to each of the participating teams for system learning purposes. Corpuses of new documents and a set of new topics were then supplied for evaluation purposes. Two optional tracks were added for Internet and interlingual track. The first track of these contained a test via the Internet. INIST sent topics to the system and collected responses directly, thus reducing the need for conceptor manipulations. The second contained tests in different European Community language pairs. The corpuses of documents consisted of records of questions and answers from the European Commission, in parallel official language versions. Participants could use any language pair for their tests. The aim of this paper is to give the point of view of an organizer and corpus provider (INIST) on the organization of an operation of this sort. In particular, it will describe the difficulties encountered during the tests (corpus construction, translation of topics and systems evaluation ), and will suggest avenues to explore for future tests.