Summary of the paper
| Title | VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation |
|---|---|
| Authors | Elisabet Comelles, Jordi Atserias, Victoria Arranz, Irene Castellon and Jordi Sesé |
| Abstract | There are several MT metrics used to evaluate translation into Spanish, although most of them use partial or little linguistic information. In this paper we present the multilingual capability of VERTa, an automatic MT metric that combines linguistic information at lexical, morphological, syntactic and semantic level. In the experiments conducted we aim at identifying those linguistic features that prove the most effective to evaluate adequacy in Spanish segments. This linguistic information is tested both as independent modules (to observe what each type of feature provides) and in a combinatory fastion (where different kinds of information interact with each other). This allows us to extract the optimal combination. In addition we compare these linguistic features to those used in previous versions of VERTa aimed at evaluating adequacy for English segments. Finally, experiments show that VERTa can be easily adapted to other languages than English and that its collaborative approach correlates better with human judgements on adequacy than other well-known metrics. |
| Topics | Tools, Systems, Applications, Evaluation Methodologies |
Full paper  |
VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation |
| Bibtex | @InProceedings{COMELLES14.1032, author = {Elisabet Comelles and Jordi Atserias and Victoria Arranz and Irene Castellon and Jordi Sesé}, title = {VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation}, booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)}, year = {2014}, month = {may}, date = {26-31}, address = {Reykjavik, Iceland}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Hrafn Loftsson and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-8-4}, language = {english} } |