Summary of the paper
| Title | Evaluation of Discourse Relation Annotation in the Hindi Discourse Relation Bank |
|---|---|
| Authors | Sudheer Kolachina, Rashmi Prasad, Dipti Misra Sharma and Aravind Joshi |
| Abstract | We describe our experiments on evaluating recently proposed modifications to the discourse relation annotation scheme of the Penn Discourse Treebank (PDTB), in the context of annotating discourse relations in Hindi Discourse Relation Bank (HDRB). While the proposed modifications were driven by the desire to introduce greater conceptual clarity in the PDTB scheme and to facilitate better annotation quality, our findings indicate that overall, some of the changes render the annotation task much more difficult for the annotators, as also reflected in lower inter-annotator agreement for the relevant sub-tasks. Our study emphasizes the importance of best practices in annotation task design and guidelines, given that a major goal of an annotation effort should be to achieve maximally high agreement between annotators. Based on our study, we suggest modifications to the current version of the HDRB, to be incorporated in our future annotation work. |
| Topics | Discourse annotation, representation and processing, Corpus (creation, annotation, etc.), Other |
Full paper  |
Evaluation of Discourse Relation Annotation in the Hindi Discourse Relation Bank |
| Bibtex | @InProceedings{KOLACHINA12.894, author = {Sudheer Kolachina and Rashmi Prasad and Dipti Misra Sharma and Aravind Joshi}, title = {Evaluation of Discourse Relation Annotation in the Hindi Discourse Relation Bank}, booktitle = {Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12)}, year = {2012}, month = {may}, date = {23-25}, address = {Istanbul, Turkey}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Mehmet Uğur Doğan and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-7-7}, language = {english} } |