Games-With-A-Purpose (GWAP) have shown some success and promise in language resource collection. However, player recruitment and accuracy can be challenging. In this work, TileAttack, a GWAP designed to gather annotations for text segmentation, is presented to the online linguistic community, an indie gaming community and the crowdsourcing community. We evaluate the results of this experiment both through traditional accuracy measures and adapted metrics from Free-to-Play games. With the addition of a tutorial, we find a high level of recall is achieved from crowdsourced non-expert workers.
@InProceedings{MADGE18.5, author = {Chris Madge ,Massimo Poesio ,Udo Kruschwitz and Jon Chamberlain}, title = {Testing TileAttack with Three Key Audiences}, booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)}, year = {2018}, month = {may}, date = {7-12}, location = {Miyazaki, Japan}, editor = {Jon Chamberlain and Udo Kruschwitz and Karën Fort and Christopher Cieri}, publisher = {European Language Resources Association (ELRA)}, address = {Paris, France}, isbn = {979-10-95546-10-8}, language = {english} }
Summary of the paper