Summary of the paper

Title Task-Based Evaluation of Meeting Browsers: from Task Elicitation to User Behavior Analysis
Authors Andrei Popescu-Belis, Mike Flynn, Pierre Wellner and Philippe Baudrion
Abstract This paper presents recent results of the application of the task-based Browser Evaluation Test (BET) to meeting browsers, that is, interfaces to multimodal databases of meeting recordings. The tasks were defined by browser-neutral BET observers. Two groups of human subjects used the Transcript-based Query and Browsing interface (TQB), and attempted to solve as many BET tasks - pairs of true/false statements to disambiguate - as possible in a fixed amount of time. Their performance was measured in terms of precision and speed. Results indicate that the browser’s annotation-based search functionality is frequently used, in particular the keyword search. A more detailed analysis of each test question for each participant confirms that despite considerable variation across strategies, the use of queries is correlated to successful performance.
Language Single language
Topics Evaluation methodologies, Multimedia annotation and processing, Tools, systems, applications
Full paper Task-Based Evaluation of Meeting Browsers: from Task Elicitation to User Behavior Analysis
Slides -
Bibtex @InProceedings{POPESCUBELIS08.580,
  author = {Andrei Popescu-Belis, Mike Flynn, Pierre Wellner and Philippe Baudrion},
  title = {Task-Based Evaluation of Meeting Browsers: from Task Elicitation to User Behavior Analysis},
  booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)},
  year = {2008},
  month = {may},
  date = {28-30},
  address = {Marrakech, Morocco},
  editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias},
  publisher = {European Language Resources Association (ELRA)},
  isbn = {2-9517408-4-0},
  note = {http://www.lrec-conf.org/proceedings/lrec2008/},
  language = {english}
  }

Powered by ELDA © 2008 ELDA/ELRA