Summary of the paper
| Title | Finding a Tradeoff between Accuracy and Rater's Workload in Grading Clustered Short Answers |
|---|---|
| Authors | andrea Horbach, Alexis Palmer and Magdalena Wolska |
| Abstract | n this paper we investigate the potential of answer clustering for semi-automatic scoring of short answer questions for German as a foreign language. We use surface features like word and character n-grams to cluster answers to listening comprehension exercises per question and simulate having human graders only label one answer per cluster and then propagating this label to all other members of the cluster. We investigate various ways to select this single item to be labeled and find that choosing the item closest to the centroid of a cluster leads to improved (simulated) grading accuracy over random item selection. Averaged over all questions, we can reduce a teacher’s workload to labeling only 40% of all different answers for a question, while still maintaining a grading accuracy of more than 85%. |
| Topics | Statistical and Machine Learning Methods, Other |
Full paper  |
Finding a Tradeoff between Accuracy and Rater's Workload in Grading Clustered Short Answers |
| Bibtex | @InProceedings{HORBACH14.887, author = {andrea Horbach and Alexis Palmer and Magdalena Wolska}, title = {Finding a Tradeoff between Accuracy and Rater's Workload in Grading Clustered Short Answers}, booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)}, year = {2014}, month = {may}, date = {26-31}, address = {Reykjavik, Iceland}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Hrafn Loftsson and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-8-4}, language = {english} } |