| Title |
INSPIRE: Evaluation of a Smart-Home System for Infotainment Management and Device Control |
| Author(s) |
Sebastian Möller (1), Jan Krebber (1), Alexander Raake (1), Paula Smeele (2), Martin Rajman (3), Mirek Melichar (3), Vincenzo Pallotta (3), Gianna Tsakou (4), Basilis Kladis (4), Anestis Vovos (4), Jettie Hoonhout (5), Dietmar Schuchardt (6), Nikos Fakotakis (7), Todor Ganchev (7), Ilyas Potamitis (7) (1) Institute of Communication Acoustics (IKA), Ruhr-University Bochum, Germany; (2) TNO Human Factors, Soesterberg, The Netherlands; (3) LIA, Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland; (4) Knowledge SA (LogicDIS Group), Patras, Greece; (5) Philips Electronics Nederland B.V., Eindhoven, The Netherlands; (6) ABS Gesellschaft für Automatisierung, Bildverarbeitung und Software mbH, Jena, Germany; (7) Wire Communications Lab (WCL), University of Patras, Greece |
| Session |
O37-EMS |
| Abstract |
This paper gives an overview of the assessment and evaluation methods which have been used to determine the quality of the INSPIRE smart home system. The system allows different home appliances to be controlled via speech, and consists of speech and speaker recognition, speech understanding, dialogue management, and speech output components. The performance of these components is first assessed individually, and then the entire system is evaluated in an interaction experiment with test users. Initial results of the assessment and evaluation are given, in particular with respect to the transmission channel impact on speech and speaker recognition, and the assessment of speech output for different system metaphors. |
| Keyword(s) |
spoken dialogue system, evaluation methods, smart home system |
| Language(s) | English |
| Full Paper |