LREC 2000 - Papers

LREC 2000 2^nd International Conference on Language Resources & Evaluation

Conference Papers

Papers by paper title: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Papers by ID number: 1-50, 51-100, 101-150, 151-200, 201-250, 251-300, 301-350, 351-377.

List of all papers and abstracts.

Previous Paper Next Paper

Title Cardinal, Nominal or Ordinal Similarity Measures in Comparative Evaluation of Information Retrieval Process

Authors Michel Christine (Laboratoire RECODOC, Universite Claude Bernard Lyon 1, 43 Bd du 11 novembre 1918, bat 721, 69622 VILLEURBANNE Cedex, Tel: + 33 (0)4 72 43 13 91, Fax: + 33 (0)4 72 43 15 59, Christine.Michel@montaigne.u-bordeaux.fr)

Keywords Evaluation, Formalism, Similarity Measure

Session Session EO5 - Information Retrieval and Question Answering Evaluation

Abstract Similarity measures are used to quantify the resemblance of two sets. Simplest ones are calculated by ratios of the document's number of the compared sets. These measures are simple and usually employed in first steps of evaluation studies, they are called cardinal measures. Others measures compare sets upon the number of common documents they have. They are usually employed in quantitative information retrieval evaluations, some examples are Jaccard, Cosine, Recall or Precision. These measures are called nominal ones. There are more or less adapted in function of the richness of the information system's answer. Indeed, in the past, they were sufficient because answers given by systems were only composed by an unordered set of documents. But usual systems improve the quality or the visibility of there answers by using a relevant ranking or a clustering presentation of documents. In this case, similarity measures aren't adapted. In this paper we present some solutions in the case of totally ordered and partially ordered answer.

Title	Cardinal, Nominal or Ordinal Similarity Measures in Comparative Evaluation of Information Retrieval Process
Authors	Michel Christine (Laboratoire RECODOC, Universite Claude Bernard Lyon 1, 43 Bd du 11 novembre 1918, bat 721, 69622 VILLEURBANNE Cedex, Tel: + 33 (0)4 72 43 13 91, Fax: + 33 (0)4 72 43 15 59, Christine.Michel@montaigne.u-bordeaux.fr)
Keywords	Evaluation, Formalism, Similarity Measure
Session	Session EO5 - Information Retrieval and Question Answering Evaluation
Abstract	Similarity measures are used to quantify the resemblance of two sets. Simplest ones are calculated by ratios of the document's number of the compared sets. These measures are simple and usually employed in first steps of evaluation studies, they are called cardinal measures. Others measures compare sets upon the number of common documents they have. They are usually employed in quantitative information retrieval evaluations, some examples are Jaccard, Cosine, Recall or Precision. These measures are called nominal ones. There are more or less adapted in function of the richness of the information system's answer. Indeed, in the past, they were sufficient because answers given by systems were only composed by an unordered set of documents. But usual systems improve the quality or the visibility of there answers by using a relevant ranking or a clustering presentation of documents. In this case, similarity measures aren't adapted. In this paper we present some solutions in the case of totally ordered and partially ordered answer.