Summary of the paper
| Title | "Human Annotation of ASR Error Regions: is ""gravity"" a Sharable Concept for Human Annotators?" |
|---|---|
| Authors | Daniel Luzzati, Cyril Grouin, Ioana Vasilescu, Martine Adda-Decker, Eric Bilinski, Nathalie Camelin, Juliette Kahn, Carole Lailler, Lori Lamel and Sophie Rosset |
| Abstract | "This paper is concerned with human assessments of the severity of errors in ASR outputs. We did not design any guidelines so that each annotator involved in the study could consider the ""seriousness"" of an ASR error using their own scientific background. Eight human annotators were involved in an annotation task on three distinct corpora, one of the corpora being annotated twice, hiding this annotation in duplicate to the annotators. None of the computed results (inter-annotator agreement, edit distance, majority annotation) allow any strong correlation between the considered criteria and the level of seriousness to be shown, which underlines the difficulty for a human to determine whether a ASR error is serious or not." |
| Topics | |
Full paper  |
"Human Annotation of ASR Error Regions: is ""gravity"" a Sharable Concept for Human Annotators?" |
| Bibtex | @InProceedings{LUZZATI14.771, author = {Daniel Luzzati and Cyril Grouin and Ioana Vasilescu and Martine Adda-Decker and Eric Bilinski and Nathalie Camelin and Juliette Kahn and Carole Lailler and Lori Lamel and Sophie Rosset}, title = {"Human Annotation of ASR Error Regions: is ""gravity"" a Sharable Concept for Human Annotators?"}, booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)}, year = {2014}, month = {may}, date = {26-31}, address = {Reykjavik, Iceland}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Hrafn Loftsson and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-8-4}, language = {english} } |