Summary of the paper
| Title | Collaborative Development and Evaluation of Text-processing Workflows in a UIMA-supported Web-based Workbench |
|---|---|
| Authors | Rafal Rak, Andrew Rowley and Sophia Ananiadou |
| Abstract | Challenges in creating comprehensive text-processing worklows include a lack of the interoperability of individual components coming from different providers and/or a requirement imposed on the end users to know programming techniques to compose such workflows. In this paper we demonstrate Argo, a web-based system that addresses these issues in several ways. It supports the widely adopted Unstructured Information Management Architecture (UIMA), which handles the problem of interoperability; it provides a web browser-based interface for developing workflows by drawing diagrams composed of a selection of available processing components; and it provides novel user-interactive analytics such as the annotation editor which constitutes a bridge between automatic processing and manual correction. These features extend the target audience of Argo to users with a limited or no technical background. Here, we focus specifically on the construction of advanced workflows, involving multiple branching and merging points, to facilitate various comparative evalutions. Together with the use of user-collaboration capabilities supported in Argo, we demonstrate several use cases including visual inspections, comparisions of multiple processing segments or complete solutions against a reference standard, inter-annotator agreement, and shared task mass evaluations. Ultimetely, Argo emerges as a one-stop workbench for defining, processing, editing and evaluating text processing tasks. |
| Topics | Tools, systems, applications, Text mining, Evaluation methodologies |
Full paper  |
Collaborative Development and Evaluation of Text-processing Workflows in a UIMA-supported Web-based Workbench |
| Bibtex | @InProceedings{RAK12.960, author = {Rafal Rak and Andrew Rowley and Sophia Ananiadou}, title = {Collaborative Development and Evaluation of Text-processing Workflows in a UIMA-supported Web-based Workbench}, booktitle = {Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12)}, year = {2012}, month = {may}, date = {23-25}, address = {Istanbul, Turkey}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Mehmet Uğur Doğan and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-7-7}, language = {english} } |