| LREC 2000 2nd International Conference on Language Resources & Evaluation | ||||||
|
|
|
|
|
|
|
| Title | Obtaining Predictive Results with an Objective Evaluation of Spoken Dialogue Systems: Experiments with the DCR Assessment Paradigm |
| Authors | Antoine Jean-Yves (EQUIPAGE team, VALORIA, Université de Bretagne Sud, IUP Vannes, r. Y. Mainguy, F-56000 Vannes, France., Email : Jean-Yves.Antoine@univ-ubs.fr) Siroux Jacques (CORDIAL team, IRISA-LLI, ENSSAT, 6 r. de Kerampont, F-22305 Lannion, France., Email : siroux@enssat.fr) Caelen Jean (CLIPS-IMAG, BP 53, F-38041 Grenoble Cedex 9, France., Email : Jean.Caelen@univ-ubs.fr @imag.fr) Villaneau Jeanne (EQUIPAGE team, VALORIA, Université de Bretagne Sud, IUP Vannes, r. Y. Mainguy, F-56000 Vannes, France., Email : berthele@univ-ubs.fr) Goulian Jérôme (EQUIPAGE team, VALORIA, Université de Bretagne Sud, IUP Vannes, r. Y. Mainguy, F-56000 Vannes, France., Email : Jerome.Goulian@univ-ubs.fr) Ahafhaf Mohamed (CLIPS-IMAG, BP 53, F-38041 Grenoble Cedex 9, France., Email : Mohamed.Ahafhaf@univ-ubs.fr @imag.fr) |
| Keywords | DCR Evaluation, Dialogue, Genericity, Multi-Criteria Diagnosis, Spoken Language Understanding |
| Session | Session SO5 - Evaluation of Dialogue |
| Full Paper | 36.ps, 36.pdf |
| Abstract | The DCR methodology is a framework that proposes a generic and detailed evaluation of spoken dialog systems. We have already detailed (Antoine et al., 1998) the theoretical bases of this paradigm. In this paper, we present some experimental results on spoken language understanding that show the feasibility and the reliability of the DCR evaluation as well as its ability to provide a detailed diagnosis of the system’s behaviour. Finally, we highlight the extension of the DCR methodology to dialogue management. |