| Title |
Predictive and objective evaluation of speech understanding: the "challenge" evaluation campaign of the I3 speech workgroup of the French CNRS |
| Authors |
Jean-Yves Antoine (VALORIA, University of South Brittany, rue Y. Mainguy, F-56000 Vannes, France) Caroline Bousquet-Vernhettes (IRIT, University Paul Sabatier, 118 route de Narbonne, F-31000 Toulouse, France) Jérôme Goulian (VALORIA, University of South Brittany, rue Y. Mainguy, F-56000 Vannes, France) Mohamed Zakaria Kurdi (CLIPS-IMAG, BP 53, F-38041 Grenoble Cedex 9, France) Sophie Rosset (LIMSI-CNRS, Orsay, France) Nadine Vigouroux (IRIT, University Paul Sabatier, 118 route de Narbonne, F-31000 Toulouse, France) Jeanne Villaneau (VALORIA, University of South Brittany, rue Y. Mainguy, F-56000 Vannes, France) |
| Session |
EP1: Evaluation |
| Abstract |
This paper presents a new paradigm of "challenge" evaluation of Spoken Language Understanding. This methodology aims at a quantitative assessment with a high diagnostic power, by opposition with standard ATIS-like frameworks. This paper details the methodology as well as the results of an evaluation campaign held by the French CNRS research agency. The benefits of this methodology are also discussed. |
| Keywords |
Speech understanding |
| Full Paper |