Summary of the paper
| Title | Improving Contextual Quality Models for MT Evaluation Based on Evaluators’ Feedback |
|---|---|
| Authors | Paula Estrella, Andrei Popescu-Belis and Maghi King |
| Abstract | The Framework for the Evaluation for Machine Translation (FEMTI) contains guidelines for building a quality model that is used to evaluate MT systems in relation to the purpose and intended context of use of the systems. Contextual quality models can thus be constructed, but entering into FEMTI the knowledge required for this operation is a complex task. An experiment has been set up in order to transfer knowledge from MT evaluation experts into the FEMTI guidelines, by polling experts about the evaluation methods they would use in a particular context, then inferring from the results generic relations between characteristics of the context of use and quality characteristics. The results of this hands-on exercise, carried out as part of a conference tutorial, have served to refine FEMTI’s “generic contextual quality model” and to obtain feedback on the FEMTI guidelines in general. |
| Language | Language-independent |
| Topics | Machine Translation, SpeechToSpeech Translation, Evaluation methodologies, Usability, user satisfaction |
Full paper |
Improving Contextual Quality Models for MT Evaluation Based on Evaluators’ Feedback |
Slides |
- |
| Bibtex | @InProceedings{ESTRELLA08.236, author = {Paula Estrella, Andrei Popescu-Belis and Maghi King}, title = {Improving Contextual Quality Models for MT Evaluation Based on Evaluators’ Feedback}, booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)}, year = {2008}, month = {may}, date = {28-30}, address = {Marrakech, Morocco}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-4-0}, note = {http://www.lrec-conf.org/proceedings/lrec2008/}, language = {english} } |