Summary of the paper
| Title | Improving Evaluation of English-Czech MT through Paraphrasing |
|---|---|
| Authors | Petra Barancikova, Rudolf Rosa and Ales Tamchyna |
| Abstract | In this paper, we present a method of improving the accuracy of machine translation evaluation of Czech sentences. Given a reference sentence, our algorithm transforms it by targeted paraphrasing into a new synthetic reference sentence that is closer in wording to the machine translation output, but at the same time preserves the meaning of the original reference sentence. Grammatical correctness of the new reference sentence is provided by applying Depfix on newly created paraphrases. Depfix is a system for post-editing English-to-Czech machine translation outputs. We adjusted it to fix the errors in paraphrased sentences. Due to a noisy source of our paraphrases, we experiment with adding word alignment. However, the alignment reduces the number of paraphrases found and the best results were achieved by a simple greedy method with only one-word paraphrases thanks to their intensive filtering. BLEU scores computed using these new reference sentences show significantly higher correlation with human judgment than scores computed on the original reference sentences. |
| Topics | Textual Entailment and Paraphrasing, Machine Translation, SpeechToSpeech Translation |
Full paper  |
Improving Evaluation of English-Czech MT through Paraphrasing |
| Bibtex | @InProceedings{BARANCIKOVA14.935, author = {Petra Barancikova and Rudolf Rosa and Ales Tamchyna}, title = {Improving Evaluation of English-Czech MT through Paraphrasing}, booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)}, year = {2014}, month = {may}, date = {26-31}, address = {Reykjavik, Iceland}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Hrafn Loftsson and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-8-4}, language = {english} } |