| Title |
A Protocol for Evaluating Analyzers of Syntax (PEAS) |
| Authors |
Véronique Gendner (LIMSI-CNRS Batiment 508 Universite Paris XI, BP 133 - 91403 ORSAY Cedex - France) Gabriel Illouz (Columbia University 2960 Broadway New York (NY)) Michèle Jardino (LIMSI-CNRS Batiment 508 Universite Paris XI, BP 133 - 91403 ORSAY Cedex - France) Laura Monceaux (LIMSI-CNRS Batiment 508 Universite Paris XI, BP 133 - 91403 ORSAY Cedex - France) Patrick Paroubek (LIMSI-CNRS Batiment 508 Universite Paris XI, BP 133 - 91403 ORSAY Cedex - France) Isabelle Robba (LIMSI-CNRS Batiment 508 Universite Paris XI, BP 133 - 91403 ORSAY Cedex - France) Anne Vilnat (LIMSI-CNRS Batiment 508 Universite Paris XI, BP 133 - 91403 ORSAY Cedex - France) |
| Session |
EP1: Evaluation |
| Abstract |
Providing a comparative framework for parsers is a task that has already been tried in the past, e.g. (Abeill´e, 1991), (Atwell and Sutcliffe, 1997), (Black et al., 1991), and studied in the literature (Black, 1993), (Black, 1994), (Carroll et al., 1998), (Gaizauskas et al., 1998), (WEPS-98, ), (Mengel and Lezius, 2000), but mainly for English. In this paper, we present PEAS: a Protocol for Evaluating Analyzers of Syntax (in French: Protocole d’Evaluation pour les Analyseurs Syntaxiques), based on an ongoing experiment at LIMSI which aims at developing and testing a generic quantitative black-box evaluation protocol for parsers of French. Two fully operational parsers will be used to test the evaluation protocol; they are: the parser (Giguet and Vergne, 1997) developed at GREYC (Caen University) and the latest version of the parser developed at Rank Xerox Research Center in Grenoble (Ait-Mokhtar and Chanod, 1997) |
| Keywords |
Analyzer |
| Full Paper |