| Title |
Different Ways of Evaluating a Swedish Grammar Checker |
| Authors |
Rickard Domeij (Affiliation: Department of Numerical Analysis and Computer Science Royal Institute of Technology SE- 100 44 Stockholm, Sweden) Ola Knutsson (Affiliation: Department of Numerical Analysis and Computer Science Royal Institute of Technology SE- 100 44 Stockholm, Sweden) Kerstin Severinson Eklundh (Affiliation: Department of Numerical Analysis and Computer Science Royal Institute of Technology SE- 100 44 Stockholm, Sweden) |
| Session |
EO2: Evaluation Methodologies |
| Abstract |
Three different ways of evaluating a Swedish grammar checker are presented and discussed in this article. The first evaluation concerns measuring the program's detection capacity on five text genres. The measures (precision and recall) are often used in evaluating grammar checkers. However, in order to test and improve the usability of grammar checking software, they need to be complemented with user-oriented methods. Consequently, the second and the third evaluations presented in the article both involve users. The second evaluation focuses on user reactions to grammar error presentations, especially with regard to false alarms and erroneous error identification. The third and last evaluation focuses on problems in supporting users' cognitive revision processes. It also examines user motives behind choosing to correct or not to correct problems highlighted by the program. Advantages and disadvantages of the different evaluation methods are discussed. |
| Keywords |
Grammar checking, Evaluation, Evaluation methods, User studies, Swedish |
| Full Paper |