Jump to ContentJump to Main Navigation
Show Summary Details
More options …


Journal of Interactive Media

Editor-in-Chief: Ziegler, Jürgen

See all formats and pricing
More options …
Volume 17, Issue 1


Generating Structured Data by Nontechnical Experts in Research Settings

Andre Breitenfeld / Florian Berger / Ming-Tung Hong / Maximilian Mackeprang / Claudia Müller-Birn
Published Online: 2018-03-27 | DOI: https://doi.org/10.1515/icom-2018-0005


Semantic technologies provide meaning to information resources in the form of machine-accessible structured data. Research over the past two decades has commonly focused on tools and interfaces for technical experts, leading to various usability problems regarding users unfamiliar with the underlying technologies – so-called nontechnical experts. Existing approaches to semantic technologies consider mostly consumers of structured data and leave out the creation perspective. In this work, we focus on the usability of creating structured data from textual resources, especially the creation of relations between entities. The research was conducted in collaboration with scholars from the humanities. We review existing research on the usability of semantic technologies and the state of the art of annotation tools to identify shortcomings. Subsequently we use the knowledge gained to propose a new interaction design for the creation of relations between entities to create structured data in the subject-predicate-object form. We implemented our interaction design and conducted a user study which showed that the proposal performed well, making it a contribution to enhance the overall usability in this field. However, this research provides an example of how technically sophisticated technology needs to be “translated” to make it usable for nontechnical experts. We need to extend this perspective in the future by providing more insight into the internal functioning of semantic technologies.

Keywords: Semantic Web; Semantic Technologies; Structured Data; Usability


  • [1]

    W3C DATA ACTIVITY building the web of data. https://www.w3.org/2013/data/.Google Scholar

  • [2]

    T. Berners-Lee. Linked Data. https://www.w3.org/DesignIssues/LinkedData.html.Google Scholar

  • [3]

    C. L. Borgman. Big Data, Little Data, No Data. MIT Press, 2015.Google Scholar

  • [4]

    M. Burghardt. Usability recommendations for annotation tools. In Proceedings of the Sixth Linguistic Annotation Workshop, pages 104–112, Jeju, Republic of Korea, July 2012. Association for Computational Linguistics.Google Scholar

  • [5]

    M. Burghardt. Engineering annotation usability – toward usability patterns for linguistic annotation tools, September 2014. URL http://epub.uni-regensburg.de/30768/. Pattern wiki: http://www.annotation-usability.net.Google Scholar

  • [6]

    J.-P. Cahier and M. Zacklad. Socio-semantic web applications: towards a methodology based on the theory of the communities of action. resource, 8 (400), 2004.Google Scholar

  • [7]

    J. J. Carroll, I. Dickinson, C. Dollin, D. Reynolds, A. Seaborne, and K. Wilkinson. Jena: Implementing the semantic web recommendations. In Proc. of the WWW, pages 74–83. ACM, 2004.Google Scholar

  • [8]

    P. Ciccarese, M. Ocana, and T. Clark. Open semantic annotation of scientific publications using DOMEO. J. Biomedical Semantics, 3 (S-1): S1, 2012a. URL http://www.jbiomedsem.com/content/3/S1/S1.CrossrefGoogle Scholar

  • [9]

    P. Ciccarese, M. Ocana, and T. Clark. Open semantic annotation of scientific publications using DOMEO. Journal of Biomedical Semantics, 3 (Suppl 1): S1, 2012b. ISSN 2041-1480. doi:. URL http://www.jbiomedsem.com/supplements/3/S1/S1.CrossrefGoogle Scholar

  • [10]

    R. Crawford. The Techniques of Creative Thinking: How to Use Your Ideas to Achieve Success. Hawthorn Books, 1954. URL https://books.google.co.uk/books?id=BCAsAQAAMAAJ.Google Scholar

  • [11]

    A. S. Dadzie, M. Rowe, and D. Petrelli. Hide the stack: Toward usable linked data. In Lecture Notes in Computer Science, pages 93–107, 2011.Google Scholar

  • [12]

    J. Damerow, B. E. Peirson, and M. D. Laubichler. Don’t panic! a research system for network-based digital history of science. In Future of Historical Network Research Conference, 2013.Google Scholar

  • [13]

    P. Di Maio. Toward global user models for semantic technologies: Emergent perspectives. In Proc. of ASWC, volume 8, pages 141–152, 2008.Google Scholar

  • [14]

    O. Gilson, N. Silva, P. W. Grant, and M. Chen. From web data to visualization via ontology mapping. In Computer Graphics Forum, volume 27, pages 959–966, 2008.Google Scholar

  • [15]

    E. Goodman, M. Kuniavsky, and A. Moed. Observing the user experience: A practitioner’s guide to user research. IEEE TPC, 56 (3): 260–261, 2013.Google Scholar

  • [16]

    M. Grassi, C. Morbidoni, M. Nucci, S. Fonda, and F. Piazza. Pundit: Augmenting web contents with semantics. Literary and Linguistic Computing, 28 (4): 640–659, 2013.CrossrefWeb of ScienceGoogle Scholar

  • [17]

    O. Hartig. Provenance information in the web of data, 2009.

  • [18]

    S. R. Herring, B. R. Jones, and B. P. Bailey. Idea generation techniques among creative professionals. In System Sciences, 2009. HICSS ’09. 42nd Hawaii International Conference on, pages 1–10, Jan 2009. doi:.CrossrefGoogle Scholar

  • [19]

    T. Hey, S. Tansley, K. M. Tolle, et al.The fourth paradigm: data-intensive scientific discovery, volume 1. Microsoft research Redmond, WA, 2009.Google Scholar

  • [20]

    A. Hinze, R. Heese, M. Luczak-Rösch, and A. Paschke. Semantic enrichment by non-experts: usability of manual annotation tools. In ISWC, pages 165–181. Springer, 2012.Google Scholar

  • [21]

    A. Hoffmeister, F. Berger, M. Pogorzhelskiy, G. Zhang, C. Zwick, and C. Müller-Birn. Toward cyber-physical research practice based on mixed reality. In M. Burghardt, R. Wimmer, C. Wolff, and C. Womser-Hacker, editors, Mensch und Computer 2017 – Workshopband, Regensburg, 2017. Gesellschaft für Informatik e.V.Google Scholar

  • [22]

    J. Huang, R. White, and G. Buscher. User see, user point: gaze and cursor alignment in web search. In Proc. of the SIGCHI Conference, pages 1341–1350. ACM, 2012.Google Scholar

  • [23]

    N. Ide and L. Romary. International standard for a linguistic annotation framework. Nat. Lang. Eng., 10 (3–4): 211–225, Sept. 2004. ISSN 1351-3249. URL http://dx.doi.org/10.1017/S135132490400350X.CrossrefGoogle Scholar

  • [24]

    A. Jameson. Usability and the Semantic Web. In ESWC, pages 3. Springer, 2006.Google Scholar

  • [25]

    N. Karam, C. Müller-Birn, M. Gleisberg, D. Fichtmüller, R. Tolksdorf, and A. Güntsch. A Terminology Service Supporting Semantic Annotation, Integration, Discovery and Analysis of Interdisciplinary Research Data. Datenbank-Spektrum, 16 (3): 195–205, Nov. 2016. ISSN 1610-1995. URL https://doi.org/10.1007/s13222-016-0231-8.Google Scholar

  • [26]

    D. R. Karger. The semantic web and end users: What’s wrong and how to fix it. IEEE Internet Computing, 18 (6): 64–70, 2014.Web of ScienceCrossrefGoogle Scholar

  • [27]

    A. Khalili, S. Auer, and D. Hladky. The rdfa content editor – from wysiwyg to wysiwym. In Proceedings of COMPSAC 2012 – Trustworthy Software Systems for the Digital Society, July 16–20, 2012, Izmir, Turkey, 2012. URL http://svn.aksw.org/papers/2012/COMPSAC_RDFaCE/public.pdf.Google Scholar

  • [28]

    A. Kiryakov, B. Popov, I. Terziev, D. Manov, and D. Ognyanoff. Semantic annotation, indexing, and retrieval. Web Semantics: Science, Services and Agents on the World Wide Web, 2 (1): 49–79, 2004. ISSN 1570-8268. https://doi.org/10.1016/j.websem.2004.07.005. URL http://www.sciencedirect.com/science/article/pii/S1570826804000162.CrossrefGoogle Scholar

  • [29]

    S. Lohmann, S. Negru, F. Haag, and T. Ertl. Visualizing ontologies with vowl. Semantic Web, 7 (4): 399–419, 2016.CrossrefWeb of ScienceGoogle Scholar

  • [30]

    V. Lopez, M. Fernández, E. Motta, and N. Stieler. PowerAqua: Supporting users in querying and exploring the Semantic Web. Semantic Web, 3 (3): 249–265, 2012.Web of ScienceGoogle Scholar

  • [31]

    M. Marcińczuk, J. Kocoń, and B. Broda. Inforex – a web-based tool for text corpus management and semantic annotation. In N. C. C. Chair, K. Choukri, T. Declerck, M. U. Doğan, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, and S. Piperidis, editors, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey, may 2012. European Language Resources Association (ELRA). ISBN 978-2-9517408-7-7.Google Scholar

  • [32]

    M. J. McGuffin and R. Balakrishnan. Fitts’ law and expanding targets: Experimental studies and designs for user interfaces. TOCHI, 12 (4): 388–422, 2005.CrossrefGoogle Scholar

  • [33]

    E. T. Meyer and R. Schroeder. The world wide web of research and access to knowledge. Knowledge Management Research & Practice, 7 (3): 218–233, Sept. 2009. ISSN 1477-8246. URL https://doi.org/10.1057/kmrp.2009.13.CrossrefWeb of ScienceGoogle Scholar

  • [34]

    C. Müller-Birn, T. Klüwer, A. Breitenfeld, A. Schlegel, and L. Benedix. Neonion: Combining human and machine intelligence. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing, CSCW’15 Companion, pages 223–226, New York, NY, USA, 2015. ACM. ISBN 978-1-4503-2946-0. URL http://doi.acm.org/10.1145/2685553.2699012.Google Scholar

  • [35]

    J. L. Navarro-Galindo and J. Samos. The FLERSA tool: adding semantics to a web content management system. IJWIS, 8 (1): 73–126, 2012. URL http://dx.doi.org/10.1108/17440081211222609.Google Scholar

  • [36]

    M. Neves and U. Leser. A survey on annotation tools for the biomedical literature. Briefings in Bioinformatics, 2012.Web of ScienceGoogle Scholar

  • [37]

    J. Nielsen. 10 usability heuristics for user interface design. Fremont: Nielsen Norman Group. [Consult. 20 maio 2014]. Disponível na Internet, 1995.

  • [38]

    K. Ohara, T. Berners-Lee, W. Hall, and N. Shadbolt. Use of the semantic web in e-research. World Wide Research: Reshaping the Sciences and Humanities, page 130, 2010.Google Scholar

  • [39]

    D. Oldman, M. Doerr, and S. Gradmann. Zen and the art of linked data. In R. S. a. U. Susan Schreibman, editor, A New Companion to Digital Humanities, pages 251–273. John Wiley & Sons, Ltd, 2015. doi:.CrossrefGoogle Scholar

  • [40]

    J. Pratt, P. V. Radulescu, R. M. Guo, and R. A. Abrams. It’s alive! animate motion captures visual attention. Psychological Science, 21 (11): 1724–1730, 2010.CrossrefWeb of ScienceGoogle Scholar

  • [41]

    C. Preist. A conceptual architecture for semantic web services. In ISWC, pages 395–409. Springer, 2004.Google Scholar

  • [42]

    G. Rizzo and R. Troncy. Nerd: Evaluating named entity recognition tools in the web of data. 2011.

  • [43]

    A. Sears. Heuristic walkthroughs: Finding the problems without the noise. Int. J. Hum. Comput. Interaction, 9 (3): 213–234, 1997.CrossrefGoogle Scholar

  • [44]

    L. Serafini and A. Tamilin. Drago: Distributed reasoning architecture for the semantic web. In ESWC, pages 361–376. Springer, 2005.Google Scholar

  • [45]

    H.  N. Talantikite, D. Aissani, and N. Boudjlida. Semantic annotations for web services discovery and composition. Comput. Stand. Interfaces, 31 (6): 1108–1117, Nov. 2009. ISSN 0920-5489.Web of ScienceGoogle Scholar

  • [46]

    V. Uren, P. Cimiano, J. Iria, S. Handschuh, M. Vargas-Vera, E. Motta, and F. Ciravegna. Semantic annotation for knowledge management: Requirements and a survey of the state of the art. Web Semant., 4 (1): 14–28, Jan. 2006. ISSN 1570-8268.CrossrefGoogle Scholar

  • [47]

    K. van Deemter and R. Kibble. What is coreference, and what should coreference annotation be? In Proceedings of the Workshop on Coreference and Its Applications, CorefApp ’99, pages 90–96, Stroudsburg, PA, USA, 1999. Association for Computational Linguistics. URL http://dl.acm.org/citation.cfm?id=1608810.1608828.Google Scholar

  • [48]

    D. Vrandečić and M. Krötzsch. Wikidata: A Free Collaborative Knowledgebase. Commun. ACM, 2014.Web of ScienceGoogle Scholar

  • [49]

    A. Widlöcher and Y. Mathet. The glozz platform: A corpus annotation and mining tool. In Proc. of the DocEng, pages 171–180. ACM, 2012.Google Scholar

  • [50]

    M. Wilkinson and et al.The fair guiding principles for scientific data management and stewardship. Nature Scientific Data, (160018), 2016. URL http://www.nature.com/articles/sdata201618.Google Scholar

About the article

Andre Breitenfeld

Andre Breitenfeld has studied Computer Science at the Freie Universität Berlin. He is interested in interaction and service design. His current work focuses on digitalization and mobility services in the automative industry.

Florian Berger

Florian Berger is a research associate at the HCC group at the Freie Universität Berlin. His research is at the Cluster of Excellence “Image Knowledge Gestaltung – An Interdisciplinary Laboratory” at the Humboldt Universität zu Berlin, developing a mixed-reality object annotation system.

Ming-Tung Hong

Ming-Tung Hong is a PhD student in Computer Science at the HCC group at Freie Universität Berlin. Her research focuses on augmenting humans’ sensemaking processes based on semantic annotations by allowing humans to interact with machine-recommended information.

Maximilian Mackeprang

Maximilian Mackeprang has recently completed his M.Sc. in Computer Science at the Humboldt Universität zu Berlin and is now member of the HCC group at the Freie Universität Berlin. His current research interests include Human-Centered Design, Creativity Support Software and Collaborative Ideation Systems.

Claudia Müller-Birn

Claudia Müller-Birn is the head of the research group Human-Centered Computing (HCC) at the Institute of Computer Science at the Freie Universität Berlin. Her research is in the fields of Computer-Supported Cooperative Work, Human-Machine Collaboration, and Social Semantic Computing.

Published Online: 2018-03-27

Published in Print: 2018-04-25

Funding Source: Bundesministerium für Bildung und Forschung

Award identifier / Grant number: 03IO1617

We gratefully acknowledge the financial support of the Elsa-Neumann-Stipend of the state of Berlin, of the German Research Foundation (Deutsche Forschungsgemeinschaft) with the Cluster of Excellence “Image Knowledge Gestaltung: An Interdisciplinary Laboratory” (EXC 1027), and the Federal Ministry of Education and Research of Germany (Bundesministerium für Bildung und Forschung) in the framework of “Ideas to Market” (project number 03IO1617).

Citation Information: i-com, Volume 17, Issue 1, Pages 25–40, ISSN (Online) 2196-6826, ISSN (Print) 1618-162X, DOI: https://doi.org/10.1515/icom-2018-0005.

Export Citation

© 2018 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in