Evaluation and collection of proper name pronunciations online

Ariadna Font Llitjos (Language Technologies Institute, Carnegie Mellon University, 5000 Forbes Ave. Pittsburgh, PA 15213)

Alan W Black (Language Technologies Institute, Carnegie Mellon University, 5000 Forbes Ave. Pittsburgh, PA 15213)

EO2: Evaluation Methodologies

Objective evaluation allows a model to be compared with other similar models. However, automatic pronunciation models should also be extensively evaluated by humans, since the ultimate goal of any pronunciation model is to produce an accurate pronunciation as judged by most people. This paper describes an initiative to evaluate and collect proper name pronunciations online, the development of the US Pronunciation of Proper Names Site (www.pronounce-names.org), and the results obtained so far. The internet, through our web-based interface, has already proven to be a very successful medium both in terms of number of evaluations and in terms of data collection. In 5 weeks, it has brought to our site 601 users, which have evaluated 477 names and corrected 281 pronunciations. The information gathered is useful to improve our pronunciation models, as well as to (automatically) correct the pronunciations in the CMU dictionary. 

Online evaluation, Proper name pronunciation, Language origin information, Data collection

104.pdf