Summary of the paper
| Title | Test Suite Design for Biomedical Ontology Concept Recognition Systems |
|---|---|
| Authors | K. Bretonnel Cohen, Christophe Roeder, William A. Baumgartner Jr., Lawrence E. Hunter and Karin Verspoor |
| Abstract | Systems that locate mentions of concepts from ontologies in free text are known as ontology concept recognition systems. This paper describes an approach to the evaluation of the workings of ontology concept recognition systems through use of a structured test suite and presents a publicly available test suite for this purpose. It is built using the principles of descriptive linguistic fieldwork and of software testing. More broadly, we also seek to investigate what general principles might inform the construction of such test suites. The test suite was found to be effective in identifying performance errors in an ontology concept recognition system. The system could not recognize 2.1% of all canonical forms and no non-canonical forms at all. Regarding the question of general principles of test suite construction, we compared this test suite to a named entity recognition test suite constructor. We found that they had twenty features in total and that seven were shared between the two models, suggesting that there is a core of feature types that may be applicable to test suite construction for any similar type of application. |
| Topics | Evaluation methodologies, Corpus (creation, annotation, etc.), Ontologies |
Full paper |
Test Suite Design for Biomedical Ontology Concept Recognition Systems |
Slides |
Test Suite Design for Biomedical Ontology Concept Recognition Systems |
| Bibtex | @InProceedings{COHEN10.31, author = {K. Bretonnel Cohen and Christophe Roeder and William A. Baumgartner Jr. and Lawrence E. Hunter and Karin Verspoor}, title = {Test Suite Design for Biomedical Ontology Concept Recognition Systems}, booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis and Mike Rosner and Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |