Summary of the paper
| Title | A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let's Go Bus Information System |
|---|---|
| Authors | Alexander Schmitt, Stefan Ultes and Wolfgang Minker |
| Abstract | Standardized corpora are the foundation for spoken language research. In this work, we introduce an annotated and standardized corpus in the Spoken Dialog Systems (SDS) domain. Data from the Let's Go Bus Information System from the Carnegie Mellon University in Pittsburgh has been formatted, parameterized and annotated with quality, emotion, and task success labels containing 347 dialogs with 9,083 system-user exchanges. A total of 46 parameters have been derived automatically and semi-automatically from Automatic Speech Recognition (ASR), Spoken Language Understanding (SLU) and Dialog Manager (DM) properties. To each spoken user utterance an emotion label from the set garbage, non-angry, slightly angry, very angry has been assigned. In addition, a manual annotation of Interaction Quality (IQ) on the exchange level has been performed with three raters achieving a Kappa value of 0.54. The IQ score expresses the quality of the interaction up to each system-user exchange on a score from 1-5. The presented corpus is intended as a standardized basis for classification and evaluation tasks regarding task success prediction, dialog quality estimation or emotion recognition to foster comparability between different approaches on these fields. |
| Topics | Corpus (creation, annotation, etc.), Speech resource/database, Discourse annotation, representation and processing |
Full paper  |
A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let's Go Bus Information System |
| Bibtex | @InProceedings{SCHMITT12.333, author = {Alexander Schmitt and Stefan Ultes and Wolfgang Minker}, title = {A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let's Go Bus Information System}, booktitle = {Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12)}, year = {2012}, month = {may}, date = {23-25}, address = {Istanbul, Turkey}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Mehmet Uğur Doğan and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-7-7}, language = {english} } |