Summary of the paper
| Title | Evaluating Evaluation Metrics for Ontology-Based Applications: Infinite Reflection |
|---|---|
| Authors | Diana Maynard, Wim Peters and Yaoyong Li |
| Abstract | In this paper, we discuss methods of measuring the performance of ontology-based information extraction systems. We focus particularly on the Balanced Distance Metric (BDM), a new metric we have proposed which aims to take into account the more flexible nature of ontologically-based applications. We first examine why traditional Precision and Recall metrics, as used for flat information extraction tasks, are inadequate when dealing with ontologies. We then describe the Balanced Distance Metric (BDM) which takes ontological similarity into account. Finally, we discuss a range of experiments designed to test the accuracy and usefulness of the BDM when compared with traditional metrics and with a standard distance-based metric. |
| Language | Language-independent |
| Topics | Information Extraction, Information Retrieval, Named Entity recognition |
Full paper |
Evaluating Evaluation Metrics for Ontology-Based Applications: Infinite Reflection |
Slides |
- |
| Bibtex | @InProceedings{MAYNARD08.273, author = {Diana Maynard, Wim Peters and Yaoyong Li}, title = {Evaluating Evaluation Metrics for Ontology-Based Applications: Infinite Reflection}, booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)}, year = {2008}, month = {may}, date = {28-30}, address = {Marrakech, Morocco}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-4-0}, note = {http://www.lrec-conf.org/proceedings/lrec2008/}, language = {english} } |