LREC 2000 - Papers

LREC 2000 2^nd International Conference on Language Resources & Evaluation

Conference Papers

Papers by paper title: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Papers by ID number: 1-50, 51-100, 101-150, 151-200, 201-250, 251-300, 301-350, 351-377.

List of all papers and abstracts.

Previous Paper Next Paper

Title A Parallel English-Japanese Query Collection for the Evaluation of On-Line Help Systems

Authors Sutcliffe Richard F.E. (Department of Computer Science and Information Systems, University of Limerick, Limerick, Ireland, Richard.Sutcliffe@ul.ie)
Kurohashi Sadao (Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo, Kyoto 606-8501, Japan, kuro@pine.kuee.kyoto-u.ac.jp)

Keywords Parallel Multilingual Evaluation Materials

Session Session EP1 - Evaluation and Written Area

Abstract An experiment concerning the creation of parallel evaluation data for information retrieval is presented. A set of English queries was gathered for the domain of wordprocessing using Lotus Ami Pro. A set of Japanese queries was then created from these. The answers to the queries were elicited from eight respondents comprising four native speakers of each language. We first describe how the queries were created and the answers elicited. We then present analyses of the responses in each language. The results show a lower level of agreeement between respondents than was expected. We discuss a refinement of the elicitation process which is designed to address this problem as well as measuring the integrity of individual respondents.

Title	A Parallel English-Japanese Query Collection for the Evaluation of On-Line Help Systems
Authors	Sutcliffe Richard F.E. (Department of Computer Science and Information Systems, University of Limerick, Limerick, Ireland, Richard.Sutcliffe@ul.ie) Kurohashi Sadao (Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo, Kyoto 606-8501, Japan, kuro@pine.kuee.kyoto-u.ac.jp)
Keywords	Parallel Multilingual Evaluation Materials
Session	Session EP1 - Evaluation and Written Area
Abstract	An experiment concerning the creation of parallel evaluation data for information retrieval is presented. A set of English queries was gathered for the domain of wordprocessing using Lotus Ami Pro. A set of Japanese queries was then created from these. The answers to the queries were elicited from eight respondents comprising four native speakers of each language. We first describe how the queries were created and the answers elicited. We then present analyses of the responses in each language. The results show a lower level of agreeement between respondents than was expected. We discuss a refinement of the elicitation process which is designed to address this problem as well as measuring the integrity of individual respondents.