LREC 2000 - Papers

LREC 2000 2^nd International Conference on Language Resources & Evaluation

Conference Papers

Papers by paper title: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Papers by ID number: 1-50, 51-100, 101-150, 151-200, 201-250, 251-300, 301-350, 351-377.

List of all papers and abstracts.

Previous Paper Next Paper

Title Lessons Learned from a Task-based Evaluation of Speech-to-Speech Machine Translation

Authors Levin Lori (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Bartlog Boris (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Font Llitjos Ariadna (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Gates Donna (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Lavie Alon (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, alavie@cs.cmu.edu, www.is.cs.cmu.edu)
Wallace Dorcas (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Watanabe Taro (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Woszczyna Monika (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)

Keywords Speech-to-Speech Machine Translation, Task-Based Evaluation

Session Session SO5 - Evaluation of Dialogue

Abstract For several years we have been conducting Accuracy Based Evaluations (ABE) of the JANUS speech-to-speech MT system (Gates et al., 1997) which measure quality and fidelity of translation. Recently we have begun to design a Task Based Evaluation for JANUS (Thomas, 1999) which measures goal completion. This paper describes what we have learned by comparing the two types of evaluation. Both evaluations (ABE and TBE) were conducted on a common set of user studies in the semantic domain of travel planning.

ce="Verdana">

Title	Lessons Learned from a Task-based Evaluation of Speech-to-Speech Machine Translation
Authors	Levin Lori (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu) Bartlog Boris (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu) Font Llitjos Ariadna (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu) Gates Donna (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu) Lavie Alon (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, alavie@cs.cmu.edu, www.is.cs.cmu.edu) Wallace Dorcas (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu) Watanabe Taro (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu) Woszczyna Monika (Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA, lsl@cs.cmu.edu)
Keywords	Speech-to-Speech Machine Translation, Task-Based Evaluation
Session	Session SO5 - Evaluation of Dialogue
Abstract	For several years we have been conducting Accuracy Based Evaluations (ABE) of the JANUS speech-to-speech MT system (Gates et al., 1997) which measure quality and fidelity of translation. Recently we have begun to design a Task Based Evaluation for JANUS (Thomas, 1999) which measures goal completion. This paper describes what we have learned by comparing the two types of evaluation. Both evaluations (ABE and TBE) were conducted on a common set of user studies in the semantic domain of travel planning.