Summary of the paper
| Title | Practical Evaluation of Speech Recognizers for Virtual Human Dialogue Systems |
|---|---|
| Authors | Xuchen Yao, Pravin Bhutada, Kallirroi Georgila, Kenji Sagae, Ron Artstein and David Traum |
| Abstract | We perform a large-scale evaluation of multiple off-the-shelf speech recognizers across diverse domains for virtual human dialogue systems. Our evaluation is aimed at speech recognition consumers and potential consumers with limited experience with readily available recognizers. We focus on practical factors to determine what levels of performance can be expected from different available recognizers in various projects featuring different types of conversational utterances. Our results show that there is no single recognizer that outperforms all other recognizers in all domains. The performance of each recognizer may vary significantly depending on the domain, the size and perplexity of the corpus, the out-of-vocabulary rate, and whether acoustic and language model adaptation has been used or not. We expect that our evaluation will prove useful to other speech recognition consumers, especially in the dialogue community, and will shed some light on the key problem in spoken dialogue systems of selecting the most suitable available speech recognition system for a particular application, and what impact training will have. |
| Topics | Dialogue, Speech Recognition/Understanding, Evaluation methodologies |
Full paper |
Practical Evaluation of Speech Recognizers for Virtual Human Dialogue Systems |
Slides |
- |
| Bibtex | @InProceedings{YAO10.675, author = {Xuchen Yao and Pravin Bhutada and Kallirroi Georgila and Kenji Sagae and Ron Artstein and David Traum}, title = {Practical Evaluation of Speech Recognizers for Virtual Human Dialogue Systems}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |