Design and Evaluation of a SLDS for E-Mail Access through the Telephone

Nuria Bel (Gilcub-Universitat de Barcelona Adolf Florensa s/n E-08028 Barcelona)

Javier Caminero (Telefónica Investigación y Desarrollo, Emilio Vargas 6, E-28043 Madrid)

Luis Hernández (E.T.S.I. Telecomunicación - UPM, Ciudad Universitaria s/n, E-28040 Madrid)

Montserrat Marimón (Gilcub-Universitat de Barcelona, Adolf Florensa s/n E-08028 Barcelona)

José F. Morlesín (Terra Networks, Edificio Ática, Vía Dos Castillas 33, E-28224 Madrid)

Josep M. Otero (Sail-Labs, Roger de Llúria 50, E-08009 Barcelona)

José Relaño (Telefónica Investigación y Desarrollo, Emilio Vargas 6, E-28043 Madrid)

M. Carmen Rodríguez (Telefónica Investigación y Desarrollo, Emilio Vargas 6, E-28043 Madrid)

Pedro M. Ruz ( Telefónica Móviles España, Serrano Galvache 56, E-28033 Madrid)

Daniel Tapias (Telefónica Móviles España, Serrano Galvache 56, E-28033 Madrid)

EP1: Evaluation

E-MATTER (E-Mail Access through the Telephone Using Speech Technology Resources) is a Trial EC project (IST-1999-21042) directed to make e-mail universally and seamlessly accessible to a broad population of potential users through an affordable telephone-based service. Thus, the main objective of E-MATTER was to develop a Spoken Language Dialogue System (SLDS) for an e-mail access service that uses a multilingual spoken language interface (both input and output) and that takes into account the cultural and the linguistic diversity nature of the e-mail messages. This paper addresses the different linguistic resources involved in the design and evaluation of the E-MATTER prototype. In the first part of the paper, we describe the guidelines for the design of the principal linguistic technologies involved in the development of the system: multilingual speech recognition, multilingual text-to-speech conversion, semantic parsing, dialogue management, language identification and advanced text verification. Then we present an evaluation methodology we have followed to obtain a complete analysis of both module deficiencies and global system behaviour. This methodology has been used to show us how to improve the prototype system, and we hope it will be general enough to be useful for testingother similar SLDS’s. 

E-mail reader, Multilingual spoken dialogue system, Language identification and advanced text verification, Dialogue evaluation

144.pdf