Summary of the paper
| Title | Evaluation of the PIT Corpus Or What a Difference a Face Makes? |
|---|---|
| Authors | Petra-Maria Strauß, Stefan Scherer, Georg Layher and Holger Hoffmann |
| Abstract | This paper presents the evaluation of the PIT Corpus of multi-party dialogues recorded in a Wizard-of-Oz environment. An evaluation has been performed with two different foci: First, a usability evaluation was used to take a look at the overall ratings of the system. A shortened version of the SASSI questionnaire, namely the SASSISV, and the well established AttrakDiff questionnaire assessing the hedonistic and pragmatic dimension of computer systems have been analysed. In a second evaluation, the user's gaze direction was analysed in order to assess the difference in the user's (gazing) behaviour if interacting with the computer versus the other dialogue partner. Recordings have been performed in different setups of the system, e.g. with and without avatar. Thus, the presented evaluation further focuses on the difference in the interaction caused by deploying an avatar. The quantitative analysis of the gazing behaviour has resulted in several encouraging significant differences. As a possible interpretation it could be argued that users are more attentive towards systems with an avatar - the difference a face makes. |
| Topics | Evaluation methodologies, Usability, user satisfaction, Validation of LRs |
Full paper |
Evaluation of the PIT Corpus Or What a Difference a Face Makes? |
Slides |
- |
| Bibtex | @InProceedings{STRAU10.890, author = {Petra-Maria Strauß and Stefan Scherer and Georg Layher and Holger Hoffmann}, title = {Evaluation of the PIT Corpus Or What a Difference a Face Makes?}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |