Jump to ContentJump to Main Navigation
Show Summary Details
More options …


Journal of Interactive Media

Editor-in-Chief: Ziegler, Jürgen

See all formats and pricing
More options …
Volume 17, Issue 1


Multiscreen Patterns – Interactions Across the Borders of Devices

Dominick Madden / Horst Schneider / Kirstin Kohler
  • Corresponding author
  • 120168 Mannheim University of Applied Sciences, Department of Computer Science, Mannheim, Germany
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2018-03-27 | DOI: https://doi.org/10.1515/icom-2018-0008


Design patterns are solutions for common design problems used in a number of fields including architecture, software development and user experience design. We compiled a pattern library for the usage of gesture-enabled interactions between different devices with screens, the so called multiscreen context. This library provides simple and intuitive gestures for connecting and disconnecting devices wirelessly as well as gestures for the exchange of data between these devices, like swiping a document from one’s tablet to the tablets of the surrounding colleagues in the same room. The library is the result of a 2.5 years running project in cooperation with three small and medium companies. We compiled the library by conducting an intense literature survey in parallel with several iterations of practical software projects in the multiscreen context. The literature survey inspired the practice-oriented project work. Being involved in software projects enabled us to identify challenges in the design and implementation of the gesture set. Therefore, we generated valuable insight for a comprehensive description of gestural interactions.

With our set of patterns, we aim to support interaction designers to choose the appropriate gesture for their given context. The patterns serve as inspiration by showing the different possibilities but also provide guidance how to design and implement a selected gesture for a given context. They help designing the details of an interaction by breaking it down into its smallest parts. To support the developers of these interactions our pattern descriptions are enriched with an Android library containing lifecycle events and the necessary gesture recognition logic. This paper provides an overview of the pattern library. In addition, the structure and the usage of the library is described in more detail with the means of one sample pattern. The pattern library is openly accessible.

Keywords: Interaction Pattern; Pattern Library; Multiscreen; Gestural Interactions


  • [1]

    Jan O. Borchers. 2000. A pattern approach to interaction design. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (pp. 369–378). ACM.Google Scholar

  • [2]

    Matthew Chalmers and I. MacColl. 2003. Seamful and seamless design in ubiquitous computing. In Workshop at the crossroads: The interaction of HCI and systems issues in UbiComp, (Vol. 8).Google Scholar

  • [3]

    Raimund Dachselt and R. Buchholz. 2009 April. Natural throw and tilt interaction between mobile phones and distant displays. In CHI’09 extended abstracts on Human factors in computing systems (pp. 3253–3258). ACM.Google Scholar

  • [4]

    Erich Gamma, R. Helm, R. Johnson and J. Vlissides. 1995. Design pattern. Elements of Object Oriented Software. Addison Wesley, Reading, MA.Google Scholar

  • [5]

    Frank Thomas and O. Johnston. 1981. The Illusion of Life: Disney Animation.Google Scholar

  • [6]

    Google Inc. The New Multi-screen World: Understanding Cross-pllattform Consumer Behavior. 2012, http://www.google.com/think/research-studies/the-new-multi-screen-world-study.html (accessed 20. Dezember 2017).Google Scholar

  • [7]

    Google Inc., Material Design Guidelines. https://material.io/guidelines/ (access 02. Januar 2018).Google Scholar

  • [8]

    Ken Hinckley, G. Ramos, F. Guimbretiere, P. Baudisch and M. Smith. 2004 May. Stitching: pen gestures that span multiple displays. In Proceedings of the working conference on Advanced visual interfaces (pp. 23–31). ACM.Google Scholar

  • [9]

    Lars Holmquist, F. Mattern, B. Schiele, P. Alahuhta, M. Beigl and H. W. Gellersen. 2001. Smart-its friends: A technique for users to easily establish connections between smart artefacts. In Ubicomp 2001: Ubiquitous Computing (pp. 116–122). Springer, Berlin/Heidelberg.Google Scholar

  • [10]

    Jörn Hurtienne and J. H. Israel, 2007 February. Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction. In Proceedings of the 1st international conference on Tangible and embedded interaction (pp. 127–134). ACM.Google Scholar

  • [11]

    Manuel Imaz and D. Benyon. 2007. Designing with blends. MIT Press.Google Scholar

  • [12]

    Hans-Christian Jetter, H. Reiterer and F. Geyer 2014. Blended Interaction: understanding natural human–computer interaction in post-WIMP interactive spaces. Personal and Ubiquitous Computing, 18(5), pp. 1139–1158.CrossrefWeb of ScienceGoogle Scholar

  • [13]

    Nicolai Marquardt and S. Greenberg. 2015. Proxemic interactions: From theory to practice. Synthesis Lectures on Human-Centered Informatics, 8(1), pp. 1–199.CrossrefGoogle Scholar

  • [14]

    Jun Rekimoto. 1996 November. Tilting operations for small screen interfaces. In Proceedings of the 9th annual ACM symposium on User interface software and technology (pp. 167–168). ACM.Google Scholar

  • [15]

    Dan Saffer, D. 2013. Microinteractions: designing with details. O’Reilly Media, Inc.Google Scholar

  • [16]

    Lucia Terrenghi, A. Quigley, A. Dix. 2009. A taxonomy for and analysis of multi-person-display ecosystems. Personal and Ubiquitous Computing 13(8), pp. 583–598.CrossrefWeb of ScienceGoogle Scholar

  • [17]

    Mark Weiser. 1994 March. Ubiquitous computing. In ACM Conference on Computer Science (p. 418).Google Scholar

About the article

Dominick Madden

Dominick Madden studied computer science at the University of Applied Sciences Mannheim where he wrote his thesis on the subjects of animation and Microinteractions in the Multi-screen context. He is now part of the User Experience and Interaction Design lab led by professor Kirstin Kohler. There he works on research projects concerning user experience in the mobile, web development and IoT fields.

Horst Schneider

Horst Schneider worked as a research assistant at the University of Applied Sciences Mannheim from 2015 to 2017. In Sysplace he focused on technical requirements, opportunities, and constraints of the description and implementation of multiscreen interaction patterns. He created a model for describing gesture-based interactions and graduated with a Master’s Degree. He now works as a software developer in Heidelberg.

Kirstin Kohler

Prof. Kirstin Kohler is Professor for user experience design and interaction design at the University of Applied Sciences in Mannheim. With her research group she participates in several research projects. At the university she is also responsible for an interdisciplinary course on design and innovation. She is heading the new maker space. https://de.linkedin.com/in/kirstin-kohler-b6ab3b94.

Published Online: 2018-03-27

Published in Print: 2018-04-25

The research was founded by the BMBF conducted in the project SysPlace (Grant number: 01IS14018D).

Citation Information: i-com, Volume 17, Issue 1, Pages 91–98, ISSN (Online) 2196-6826, ISSN (Print) 1618-162X, DOI: https://doi.org/10.1515/icom-2018-0008.

Export Citation

© 2018 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in