Incremental Methods to Select Test Sentences for Evaluating Translation Ability

Yasuhiro Akiba (1,2), Eiichiro Sumita (1), Hiromi Nakaiwa (1), Seiichi Yamamoto (1), Hiroshi G. Okuno (2)

(1) ATR Spoken Language Translation Research Laboratories, and (2) Graduate School of Informatics, Kyoto University

P25-EW

This paper addresses the problem of selecting test sentences for automatically evaluating language learners' translation ability within a smaller error. In this paper, the ability to translate is measured as a TOEIC score. The existing selection methods only check whether an individual test sentence contributes to the estimation of the ability to translate or that of more general academic abilities, although combinations of test sentences may be used to contribute the estimation. This paper proposes two methods that solve the selection problem. The first method selects test sentences to minimize the estimation errors of learners' TOEIC scores. The second method selects test sentences to maximize the correlation coefficient between the number of correct translations and learners' estimated TOEIC scores. The optimization technique used in both of the proposed methods is the gradient technique in mathematical programming. The proposed methods proved to be more accurate than any of the existing methods we tested, and they estimated each TOEIC score within a permissible error of 69 points.

Automatic evaluation, Translation ability, Human translation, Test set construction, Linear regression, E-learning

115.pdf