LREC 2006 - Proceedings sorted by papers

Title	The Evolution of an Evaluation Framework for a Text Mining System
Authors	N. Underwood, A. Lisowska
Abstract	The Parmenides project developed a text mining application applied in three different domains exemplified by case studies for the three user partners in the project. During the lifetime of the project (and in parallel with the development of the system itself) an evaluation framework was developed by the authors in conjunction with the users, and was eventually applied to the system. The object of the exercise was two-fold: firstly to develop and perform a complete user-centered evaluation of the system to assess how well it answered the users' requirements and, secondly, to develop a general framework which could be applied in the context of other users' requirements and (with some modification) to similar systems. In this paper we describe not only the framework but the process of building and parameterising the quality model for each case study and, perhaps most interestingly, the way in which the quality model and users' requirements and expectations evolved over time.
Keywords	user centred evaluation, quality model, text mining
Full paper	The Evolution of an Evaluation Framework for a Text Mining System

Title

The Evolution of an Evaluation Framework for a Text Mining System

Authors

N. Underwood, A. Lisowska

Abstract

The Parmenides project developed a text mining application applied in three different domains exemplified by case studies for the three user partners in the project. During the lifetime of the project (and in parallel with the development of the system itself) an evaluation framework was developed by the authors in conjunction with the users, and was eventually applied to the system. The object of the exercise was two-fold: firstly to develop and perform a complete user-centered evaluation of the system to assess how well it answered the users' requirements and, secondly, to develop a general framework which could be applied in the context of other users' requirements and (with some modification) to similar systems. In this paper we describe not only the framework but the process of building and parameterising the quality model for each case study and, perhaps most interestingly, the way in which the quality model and users' requirements and expectations evolved over time.

Keywords

user centred evaluation, quality model, text mining

Full paper

The Evolution of an Evaluation Framework for a Text Mining System