Jump to ContentJump to Main Navigation
Show Summary Details
More options …


Journal of Interactive Media

Editor-in-Chief: Ziegler, Jürgen

See all formats and pricing
More options …
Volume 17, Issue 2


Virtual Prototyping of Mixed Reality Interfaces with Internet of Things (IoT) Connectivity

Thies Pfeiffer / Nadine Pfeiffer-Leßmann
Published Online: 2018-08-07 | DOI: https://doi.org/10.1515/icom-2018-0025


One key aspect of the Internet of Things (IoT) is, that human machine interfaces are disentangled from the physicality of the devices. This provides designers with more freedom, but also may lead to more abstract interfaces, as they lack the natural context created by the presence of the machine. Mixed Reality (MR) on the other hand, is a key technology that enables designers to create user interfaces anywhere, either linked to a physical context (augmented reality, AR) or embedded in a virtual context (virtual reality, VR). Especially today, designing MR interfaces is a challenge, as there is not yet a common design language nor a set of standard functionalities or patterns. In addition to that, neither customers nor future users have substantial experiences in using MR interfaces.

Prototypes can contribute to overcome this gap, as they continuously provide user experiences of increasing realism along the design process. We present ExProtoVAR, a tool that supports quick and lightweight prototyping of MR interfaces for IoT using VR technology.

Keywords: prototyping; design process; augmented reality; mixed reality; virtual reality; 360∘ panorama


  • [1]

    Günter Alce, Klas Hermodsson, Mattias Wallergård, Lars Thern, and Tarik Hadzovic. A prototyping method to simulate wearable augmented reality interaction in a virtual environment—a pilot study. 01 2015.

  • [2]

    Andrew Banks and Rahul Gupta. MQTT version 3.1. 1. OASIS standard, 29, 2014.

  • [3]

    Matthias Berning, Jin Nakazawa, Takuro Yonezawa, Michael Beigl, Till Riedel, and Hide Tokuda. PARnorama: 360 degree interactive video for augmented reality prototyping. In UbiComp 2013 Adjunct—Adjunct Publication of the 2013 ACM Conference on Ubiquitous Computing, pages 1471–1474, 2013.Google Scholar

  • [4]

    Jacob Buur and Astrid Soendergaard. Video Card Game: An Augmented Environment for User Centred Design Discussions. In Proceedings of DARE 2000 on Designing Augmented Reality Environments, DARE ’00, pages 63–69, New York, NY, USA, 2000. ACM.Google Scholar

  • [5]

    Sebastian Büttner, Henrik Mucha, Markus Funk, Thomas Kosch, Mario Aehnelt, Sebastian Robert, and Carsten Röcker. In The Design Space of Augmented and Virtual Reality Applications for Assistive Environments in Manufacturing: A Visual Approach, pages 433–440, 2017. ACM Press.Google Scholar

  • [6]

    Marco de Sá and Elizabeth Churchill. Mobile Augmented Reality: Exploring Design and Prototyping Techniques. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services, MobileHCI ’12, pages 221–230, New York, NY, USA, 2012. ACM.Google Scholar

  • [7]

    Design Council. A study of the design process—The Double Diamond, 2005.

  • [8]

    Panos E. Kourouthanassis, Costas Boletsis, and George Lekakos. Demystifying the design of mobile augmented reality applications. Multimedia Tools and Applications, 74(3):1045–1066, February 2015.CrossrefWeb of ScienceGoogle Scholar

  • [9]

    C. Lee, S. Bonebrake, T. Hollerer, and D. A. Bowman. A Replication Study Testing the Validity of AR Simulation in VR for Controlled Experiments. In 2009 8th IEEE International Symposium on Mixed and Augmented Reality, pages 203–204, October 2009.Google Scholar

  • [10]

    Cha Lee, Scott Bonebrake, Doug A. Bowman, and Tobias Höllerer. The role of latency in the validity of AR simulation. In 2010 IEEE Virtual Reality Conference (VR), pages 11–18, March 2010.Google Scholar

  • [11]

    Roger A. Light. Mosquitto. https://mosquitto.org/, accessed 2018-05-01, 2009.Google Scholar

  • [12]

    Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, UIST ’04, pages 197–206, New York, NY, USA, 2004. ACM.Google Scholar

  • [13]

    Nick O’Leary and Dave Conway-Jones. Node-RED, v0.18.4. https://nodered.org/, accessed 2018-05-01, 2013.Google Scholar

  • [14]

    ProFI Project Consortium. BMBF project Prototyping for Innovation. http://www.prototyping4innovation.de, 2018.Google Scholar

  • [15]

    Eric Ragan, Curtis Wilkes, Doug A. Bowman, and Tobias Höllerer. Simulation of Augmented Reality Systems in Purely Virtual Environments. In 2009 IEEE Virtual Reality Conference, pages 287–288, March 2009.Google Scholar

  • [16]

    Patrick Renner and Thies Pfeiffer. Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. In 2017 IEEE Symposium on 3D User Interfaces (3DUI), pages 186–194, March 2017.Google Scholar

  • [17]

    Yan Shen, Sohkhim Ong, and Andrew Y. C. Nee, Augmented reality for collaborative product design and development. Design Studies, 31(2):118–145, March 2010.CrossrefWeb of ScienceGoogle Scholar

  • [18]

    Erik Steindecker, Ralph Stelzer, and Bernhard Saske. Requirements for Virtualization of AR Displays within VR Environments. In Randall Shumaker and Stephanie Lackey, editors, Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments, number 8525 in Lecture Notes in Computer Science, pages 105–116, June 2014. Springer International Publishing.Google Scholar

  • [19]

    Abou Moussa Wafaa, Nelly De Bonnefoy, Emmanuel Dubois, Patrice Torguet, and Jean-Pierre Jessel. Virtual Reality Simulation for Prototyping Augmented Reality. In 2008 International Symposium on Ubiquitous Virtual Reality, pages 55–58, July 2008.Google Scholar

About the article

Thies Pfeiffer

Thies Pfeiffer studied Informatics at Bielefeld University and obtained his doctoral degree on human-machine interaction in 2010. In 2006, he founded Mediablix, a usability consultancy, and in 2017 Raumtänzer, a specialist for mixed reality solutions. He is currently researcher in the Central Lab Facilities at CITEC, the Cluster of Excellence Cognitive Interaction Technology at Bielefeld University, and technical director of the Virtual Reality Lab and the Immersive Media Lab.

Nadine Pfeiffer-Leßmann

Nadine Pfeiffer-Leßmann studied Informatics at Bielefeld University and received her doctoral degree in 2011. She worked in different research projects concerned with situated communication, artificial intelligence and cognition, as well as software engineering for smart kitchens. Since 2017 she works at Neuland-Medien as a researcher and senior architect with a focus on mixed reality, prototyping and IoT.

Published Online: 2018-08-07

Published in Print: 2018-08-28

This work has been partly supported by the Federal Ministry of Education and Research (BMBF) of Germany in the project “ProFI – Prototyping for Innovation” (FKZ: 01IS16015), http://prototyping4innovation.de/.

Citation Information: i-com, Volume 17, Issue 2, Pages 179–186, ISSN (Online) 2196-6826, ISSN (Print) 1618-162X, DOI: https://doi.org/10.1515/icom-2018-0025.

Export Citation

© 2018 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in