This article presents the RoboCorp game. RoboCorp is a game with a purpose aimed at facilitating the annotation process of a natural language data. What makes this game unique and novel is the use of various mechanisms known from the popular Free to Play model to provide a fun and attractive gameplay. These mechanisms are presented and described in detail in the context of RoboCorp. The obtained annotation results are discussed and compared to other similar annotation tools available.
Wissler L, Almashraee M, Monett D, Paschkeg A. The gold standard in corpus annotation. In: Proceedings of the 5th IEEE Germany Students Conference, Germany, 2014.
Law E, von Ahn L. Human Computation. San Rafael, California: Morgan and Claypool Publishers, 2011.
Snow R, O’Connor B, Jurafsky D, Ng AY. Cheap and fast – but is it good? Evaluating non-expert annotations for natural language tasks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Honolulu, 2008: 254–63.
Zaidan OF, Callison-Burch C. Crowdsourcing translation: professional quality from non-professionals. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, 2011: 1220–8.
Hong Y, Kwak H, Baek Y, Moon S. Tower of Babel: a crowdsourcing game building sentiment lexicons for resource-scarce languages. In: Proceedings of the 22nd International Conference on World Wide Web Companion, 2013:549–56.
von Ahn L. Games with a purpose. Computer 2006;39:92–4
von Ahn L, Dabbish L. Designing games with a purpose. Commun ACM 2008;51:58–67.
Lee T, Dugan C, Geyer W, Ratchford T, Rasmussen J, Shami N, et al. Experiments on motivational feedback for crowdsourced workers. In: Proceedings of the 7th International AAAI Conference on Weblogs and Social Media, North America, 2013:341–50.
Poesio M, Chamberlain J, Kruschwitz U, Robaldo L, Ducceschi L. Phrase detectives: utilizing collective intelligence for internet-scale language resource creation. Trans Interact Intelll Syst 2013;3:1–44.
Venhuizen N, Basile V, Evang K, Bos J. Gamification for word sense labeling. In: Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), Short Papers, 2013:397–403.
Jurgens D, Navigli R. It’s all fun and games until someone annotates: video games with a purpose for linguistic annotation. Trans Assoc Comput Linguist 2014;2:445–64.
Kleka P, Łupkowski P. Gamifying science – the issue of data validation. Homo Ludens 2014;6:45–54.
Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, et al. Predicting protein structures with a multiplayer online game. Nature 2010;466:756–60.
Bohannon J. Online video game plugs players into remote- controlled biochemistry lab. Science 2014;343:475.
Mäyrä F. Mobile games. In: Mansell R, Ang PH, editors. International Encyclopedia of Digital Communication and Society. Chichester, UK: John Wiley and Sons, Inc., 2015.
Luton W. Free2Play. Making money from games you give away. San Francisco, California: New Riders, 2013.
Waszczuk J, Głowińska K, Savary A, Przepiórkowski A, Lenart M. Annotation tools for syntax and named entities in the National Corpus of Polish. Int J Data Mining Modell Manage 2013;5:103–22.
The journal Bio-Algorithms and Med-Systems provides a forum for the exchange of information in the interdisciplinary fields of computational methods applied in medicine presenting new algorithms and databases that allow the progress in collaboration between medicine, informatics, physics, and biochemistry.