Jump to ContentJump to Main Navigation
Show Summary Details
More options …


Journal of Interactive Media

Editor-in-Chief: Ziegler, Jürgen

See all formats and pricing
More options …
Volume 14, Issue 3


Generating Interactive Prototypes from Query Annotated Discourse Models

Filip Kis / Cristian Bogdan
Published Online: 2015-12-01 | DOI: https://doi.org/10.1515/icom-2015-0041


Model Based User Interface Development offers the possibility to design User Interfaces without being concerned about the underlying implementation. This is achieved by devising models at a high level of abstraction, thus creating the potential for involving users or domain experts to achieve a user-centered design process. Obtaining a running interactive application from such models usually requires several model transformations. One of the current problems is that while a user interface is generated after these transformations, other parts of the interactive system such as the application logic need to pre-exist or they must be written manually before the interface can be tested in a realistic scenario. This leaves the domain experts dependent on programmers and increases the time between iterations. In this paper we work with Query Annotations, which were previously used only for modeling at low levels and for generating fully functional interfaces, and we aim to generalize them for the high-level modeling approach called Discourse Modeling. The direct expected benefit of this generalization is the possibility to generate complete, readily testable interactive prototypes, rather than just their user interfaces. In addition, Query Annotations can serve as the mapping between the various levels of abstraction and bring to the domain experts a better understanding of the transformation process, as well as the possibility to modify the interfaces and models directly.

Keywords: Discourse Model; GUI Generation; Interactive Prototypes; Query Annotations; UI Modeling


  • Beaudouin-Lafon, M. and W. Mackay. 2003. Prototyping tools and techniques. In: (J. A. Jacko and A. Sears, eds) The Human Computer Interaction Handbook, L. Erlbaum Associates Inc., Hillsdale, NJ, USA, pp. 1006–1031.Google Scholar

  • Bergh, J. V. D., K. Luyten and K. Coninx. 2011. CAP3: Context-Sensitive Abstract User Interface Specification Expertise Centre for Digital Media. Digital Media, pp. 31–40.Google Scholar

  • Bogdan, C. and R. Mayer. 2009. Makumba: the Role of Technology or the Sustainability of Amateur Programming Practice and Community. In: Proceedings of the fourth international conference on Communities and technologies. C&T ’09, ACM Press, New York, New York, USA, pp. 205–214.Google Scholar

  • Brambilla, M., S. Comai, P. Fraternali and M. Matera. 2008. Designing Web Applications with Webml and Webratio. In: (G. Rossi, O. Pastor, D. Schwabe, and L. Olsina, eds) Web Engineering: Modelling and Implementing Web Applications, Human – Computer Interaction Series, Springer London, pp. 221–261.CrossrefGoogle Scholar

  • Calvary, G., J. Coutaz, D. Thevenin, Q. Limbourg, L. Bouillon and J. Vanderdonckt. 2003. A Unifying Reference Framework for multi-target user interfaces. Interact. Comput. 15(3): 289–308.Google Scholar

  • Constantine, L. L. 2003. Canonical Abstract Prototypes for Abstract Visual and Interaction Design. In: (J. A. Jorge, N. J. Nunes, and J. F. e Cunha, eds) Interactive Systems. Design, Specification, and Verification. LNCS, pp. 1–15.Google Scholar

  • Coyette, A., S. Faulkner, M. Kolp, Q. Limbourg and J. Vanderdonckt. 2004. SketchiXML: towards a multi-agent design tool for sketching user interfaces based on USIXML. In: Proceedings of the 3rd annual conference on Task models and diagrams. ACM, pp. 75–82.Google Scholar

  • Falb, J., H. Kaindl, H. Horacek, C. Bogdan, R. Popp and E. Arnautovic. 2006. A discourse model for interaction design based on theories of human communication. In: CHI ’06 Extended Abstracts on Human Factors in Computing Systems. CHI EA ’06, ACM Press, New York, New York, USA, pp. 754–759.Google Scholar

  • García Frey, A., E. Céret, S. Dupuy-Chessa, G. Calvary, and Y. Gabillon. 2012. UsiCOMP: an Extensible Model-Driven Composer. In: Proceedings of the fourth ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS 2012). EICS ’12, ACM Press, New York, NY, USA, pp. 263–268.Google Scholar

  • Horacek H., R. Popp, and D. Raneburger. 2011. Automated Generation of User Interfaces – A Comparison of Models and Future Prospects. In: (I. Maurtua, ed) Human-machine interaction – getting closer (Jan), InTech, Rijeka, Croatia, pp. 3–22.Google Scholar

  • Hennig, S., J. V. D. Bergh, K. Luyten and A. Braune. 2011. User Driven Evolution of User Interface Models – The FLERP Approach. In: INTERACT ’11 Proceedings of the IFIP TC13 Interantional Conference on Human – Computer Interaction. INTERACT ’13, pp. 610–627.Google Scholar

  • Kaindl, H., R. Popp, D. Raneburger, D. Ertl, J. Falb, A. Szép and C. Bogdan. 2011. Robot-Supported Cooperative Work: A Shared-Shopping Scenario. In: Proceedings of the 44th Hawaii International Conference on System Sciences. HICSS ’11, IEEE Computer Society, Washington, DC, USA, pp. 1–10.Google Scholar

  • Kavaldjian, S., J. Falb and H. Kaindl. 2009. Generating content presentation according to purpose. In: Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics. SMC ’09, IEEE Computer Society, San Antonio, TX, USA, pp. 2046–2051.Google Scholar

  • Kennard, R. and J. Leaney. 2010. Towards a general purpose architecture for UI generation. J. Syst. Software 83(10): 1896–1906.Web of ScienceCrossrefGoogle Scholar

  • Kis, F. and C. Bogdan. 2013. Lightweight Low-Level Query-Centric User Interface Modeling. In: Proceedings of the 2013 46th Hawaii International Conference on System Sciences. HICSS ’13, IEEE Computer Society, Washington, DC, USA, pp. 440–449.Google Scholar

  • Kis, F., C. Bogdan, H. Kaindl and J. Falb. 2014. Towards Fully Declarative High-level Interaction Models: An Approach Facilitating Automated GUI Generation. In: Proceedings of the 47th Hawaii International Conference on System Sciences. HICSS’14, IEEE Computer Society, Washington, DC, USA, pp. 412–421.Google Scholar

  • Limbourg, Q. and J.Vanderdonckt. 2009. Multipath Transformational Development of User Interfaces with Graph Transformations. In: (A. Seffah, J. Vanderdonckt and M. Desmarais, eds) Human-Centered Software Engineering, Human – Computer Interaction Series, Springer London, pp. 107–138.Web of ScienceCrossrefGoogle Scholar

  • Limbourg, Q., J. Vanderdonckt, B. Michotte and L. Bouillon. 2005. USIXML: A Language Supporting Multi-path Development of User Interfaces. In: (R. Bastide, P. Palanque, J. Roth, eds) Engineering Human Computer Interaction and Interactive Systems, Lecture Notes in Computer Science, Springer Berlin Heidelberg, pp. 200–220.Google Scholar

  • Luff, P., N. G. Gilbert and D. Frohlich. 1990. Computers and conversation. Academic Press, London, UK.Google Scholar

  • Mann, W. C. and S. A. Thompson, 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text – Interdisciplinary Journal for the Study of Discourse 8(3): 243–281.Google Scholar

  • Meixner, G., F. Paternò, J. Vanderdonckt, 2011. Past, Present, and Future of Model-Based User Interface Development. i-com 10(3): 2–11.Google Scholar

  • Myers, B. A., S. E. Hudson, R. Pausch. 2000. Past, present, and future of user interface software tools. ACM Transactions on Computer – Human Interaction 7(1): 3–28.Google Scholar

  • Paterno, F. 2013. End User Development: Survey of an Emerging Field for Empowering People. International Scholarly Research Notices 2013, 1–11.Google Scholar

  • Paternò, F., C. Santoro and L. D. Spano. 2009. MARIA: A Universal, Declarative, Multiple Abstraction-Level Language for Service-Oriented Applications in Ubiquitous Environments. ACM Transactions on Computer – Human Interaction 16(4): 1–30.Google Scholar

  • Paternò, F., C. Santoro and L. D. Spano.2010. Exploiting web service annotations in model-based user interface development. In: Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems. ACM, pp. 219–224.Google Scholar

  • Popp, R., H. Kaindl, S. Badalians, D. Raneburger and F. Paterno. 2014. Duality of task- and discourse-based interaction design for GUI generation. In: 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, pp. 3316–3321.Google Scholar

  • Popp, R., H. Kaindl and D. Raneburger. 2013. Connecting Interaction Models and Application Logic for Model-Driven Generation of Web-Based Graphical User Interfaces. In: 2013 20th Asia-Pacific Software Engineering Conference (APSEC). vol. 1, IEEE, pp. 215–222.Google Scholar

  • Popp, R., D. Raneburger and H. Kaindl. 2013. Tool Support for Automated Multi-device GUI Generation from Discourse-based Communication Models. In: Proceedings of the 5th ACM SIGCHI Symposium on Engineering Interactive Computing Systems. EICS’13, ACM, New York, pp. 154–150.Google Scholar

  • Raneburger, D., H. Kaindl, R. Popp, V. Šajatović and A. Armbruster. 2014. A process for facilitating interaction design through automated GUI generation. In: Proceedings of the 29th Annual ACM Symposium on Applied Computing – SAC ’14. ACM Press, New York, New York, USA, pp. 1324–1330.Google Scholar

  • Sanchez Ramon, O., J. Vanderdonckt, J. Garcia Molina, O. S. Ramón and J. G. Molina. 2013. Re-engineering graphical user interfaces from their resource files with UsiResourcer. In: Research Challenges in Information Science (RCIS), 2013 IEEE Seventh International Conference on. pp. 1–12.Google Scholar

  • Searle, J. R. 1969. Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, Cambridge, England.Google Scholar

  • Træ tteberg, H. 2008. A Hybrid Tool For User Interface Modeling And Prototyping. Computer-Aided Design Of User Interfaces V (Idi): 215–230.Google Scholar

  • Tran, V., J. Vanderdonckt, M. Kolp, S. Faulkner. 2009. Generating user interface from task, user and domain models. 2nd International Conference on Advances in Human-Oriented and Personalized Mechanisms, Technologies, and Services – CENTRIC 2009, pp. 19–26.Google Scholar

About the article

Filip Kis

Filip Kis is a PhD student at the Department of Media Technology and Interaction Design (MID), School of Computer Science and Communication (CSC) at KTH Royal Institute of Technology, Stockholm, Sweden. His research into User Interface development is focused on enabling evolutionary prototyping and end-user development. He is particularly interested in evolutionary prototyping in the context of startup environments where the emergence of lean approaches is changing the development processes and technology.

Cristian Bogdan

Cristian Bogdan is Associate Professor at the Department of Media Technology and Interaction Design (MID), School of Computer Science and Communication (CSC) at KTH Royal Institute of Technology, Stockholm, Sweden. His background is in Computer Engineering (Timisoara, Romania), his PhD is from KTH on Human-Computer Interaction, and his post-doc at TU Wien in model-based UI development based on human communication theories. His research interests revolve around the translucency of technology for its users, which he looked at e. g. for semantic technologies and robotics. Currently, translucency is investigated in the areas of energy technologies, electric vehicles, and voting systems. Relatedly, Cristian is pursuing a quest to involve users, domain experts and interaction designers in the development of interactive artifacts. His main approach in his quest is the design and evaluation of evolutionary prototyping technologies.

Published Online: 2015-12-01

Published in Print: 2015-12-01

Citation Information: i-com, Volume 14, Issue 3, Pages 205–219, ISSN (Online) 2196-6826, ISSN (Print) 1618-162X, DOI: https://doi.org/10.1515/icom-2015-0041.

Export Citation

© 2015 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in