| LREC 2000 2nd International Conference on Language Resources & Evaluation | ||||||
|
|
|
|
|
|
|
| Title | Evaluation for Darpa Communicator Spoken Dialogue Systems |
| Authors | Walker Marilyn (AT& T Labs - Research, 180 Park Ave, Florham Park, N.J. 07932, U.S.A., walker@research.att.com) Hirschman Lynette (The MITRE Corporation, 202 Burlington Road;Bedford,MA 01730 USA, lynette@mitre.org) Aberdeen John (The MITRE Corporation, 202 Burlington Rd., Bedford, MA 01730, U.S.A., aberdeen@mitre.org) |
| Keywords | |
| Session | Session SO5 - Evaluation of Dialogue |
| Full Paper | 191.ps, 191.pdf |
| Abstract | The overall objective of the DARPA COMMUNICATOR project is to support rapid, cost-effective development of multi-modal speech-enabled dialogue systems with advanced conversational capabilities, such as plan optimization, explanation and negotiation. In order to make this a reality, we need to find methods for evaluating the contribution of various techniques to the users’ willingness and ability to use the system. This paper reports on the approach to spoken dialogue system evaluation that we are applying in the COMMUNICATOR program. We describe our overall approach, the experimental design, the logfile standard, and the metrics applied in the experimental evaluation planned for June of 2000. |