Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter Oldenbourg April 8, 2021

Proficiency-aware systems: Designing for user reflection in context-aware systems

Jakob Karolus ORCID logo and Paweł W. Woźniak ORCID logo


In an increasingly digital world, intelligent systems support us in accomplishing many everyday tasks. With the proliferation of affordable sensing devices, inferring user states from collected physiological data paves the way to tailor-made adaptation. While estimating a user’s abilities is technically possible, such proficiency assessments are rarely employed to benefit the user’s task reflection. In our work, we investigate how to model and design for proficiency estimation as part of context-aware systems. In this paper, we present the definition and conceptual architecture of proficiency-aware systems. The concept is not only applicable to current adaptive systems but provides a stepping stone for systems which actively aid in developing user proficiency during interaction.


Funding source: European Research Council

Award Identifier / Grant number: 683008

Funding statement: Funded by the European Research Council (Horizon 2020 Programme, Grant No.: 683008 AMPLIFY).


1. Adadi, A., and Berrada, M. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6 (2018), 52138–52160.10.1109/ACCESS.2018.2870052Search in Google Scholar

2. Baumer, E. P., Khovanskaya, V., Matthews, M., Reynolds, L., Schwanda Sosik, V., and Gay, G. Reviewing reflection: On the use of reflection in interactive system design. In Proceedings of the 2014 Conference on Designing Interactive Systems, DIS ’14 (New York, NY, USA, June 2014), Association for Computing Machinery, pp. 93–102.10.1145/2598510.2598598Search in Google Scholar

3. Cechanowicz, J. E., Gutwin, C., Bateman, S., Mandryk, R., and Stavness, I. Improving player balancing in racing games. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play, CHI PLAY ’14 (New York, NY, USA, Oct. 2014), Association for Computing Machinery, pp. 47–56.10.1145/2658537.2658701Search in Google Scholar

4. Corbett, A. T., Koedinger, K. R., and Anderson, J. R. Chapter 37 – Intelligent Tutoring Systems. In Handbook of Human-Computer Interaction (Second Edition), M. G. Helander, T. K. Landauer, and P. V. Prabhu, Eds. North-Holland, Amsterdam, Jan. 1997, pp. 849–874.10.1016/B978-044481862-1.50103-5Search in Google Scholar

5. Council of Europe. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Applied Linguistics Non Series. Cambridge University Press, 2001.Search in Google Scholar

6. Dey, A. K. Understanding and Using context. Personal and Ubiquitous Computing (2001), 4.10.1007/s007790170019Search in Google Scholar

7. D’Mello, S. K., Olney, A., Williams, C., and Hays, P. Gaze tutor: A gaze-reactive intelligent tutoring system. Int. J. Hum. Comput. Stud. 70 (2012), 377–398.10.1016/j.ijhcs.2012.01.004Search in Google Scholar

8. Ewing, K. C., Fairclough, S. H., and Gilleade, K. Evaluation of an adaptive game that uses EEG measures validated during the design process as inputs to a biocybernetic loop. Front. Hum. Neurosci. 10 (May 2016).10.3389/fnhum.2016.00223Search in Google Scholar PubMed PubMed Central

9. Fischer, G. User modeling in human–computer interaction. User Modeling and User-Adapted Interaction 11, 1 (Mar. 2001), 65–86.10.1023/A:1011145532042Search in Google Scholar

10. Grindinger, T., Duchowski, A. T., and Sawyer, M. Group-wise similarity and classification of aggregate scanpaths. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, ETRA ’10 (New York, NY, USA, Mar. 2010), Association for Computing Machinery, pp. 101–104.10.1145/1743666.1743691Search in Google Scholar

11. Herbert A. Simon. Rational choice and the structure of the environment. Psychological Review, 63(2) (1956).10.1037/h0042769Search in Google Scholar PubMed

12. Karolus, J., and Schmidt, A. Proficiency-aware systems: adapting to the user’s skills and expertise. In Proceedings of the 7th ACM International Symposium on Pervasive Displays (Munich Germany, June 2018), ACM, pp. 1–2.10.1145/3205873.3210708Search in Google Scholar

13. Karolus, J., Schuff, H., Kosch, T., Wozniak, P. W., and Schmidt, A. EMGuitar: Assisting guitar playing with electromyography. In Proceedings of the 2018 Designing Interactive Systems Conference, DIS ’18 (Hong Kong, China, June 2018), Association for Computing Machinery, pp. 651–655.10.1145/3196709.3196803Search in Google Scholar

14. Karolus, J., Wozniak, P. W., Chuang, L. L., and Schmidt, A. Robust gaze features for enabling language proficiency awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17 (New York, NY, USA, 2017), ACM, pp. 2998–3010.10.1145/3025453.3025601Search in Google Scholar

15. Kruger, J., and Dunning, D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology 77 (1999), 1121–1134.10.1037/0022-3514.77.6.1121Search in Google Scholar

16. Martínez-Gómez, P., and Aizawa, A. Recognition of understanding level and language skill using measurements of reading behavior. In Proceedings of the 19th International Conference on Intelligent User Interfaces, IUI ’14 (New York, NY, USA, Feb. 2014), Association for Computing Machinery, pp. 95–104.Search in Google Scholar

17. Rayner, K. Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124, 3 (Nov. 1998), 372–422.10.1037/0033-2909.124.3.372Search in Google Scholar

18. Rogers, Y. Moving on from Weiser’s vision of calm computing: Engaging UbiComp experiences. In P. Dourish and A. Friday, Eds., UbiComp 2006: Ubiquitous Computing (Berlin, Heidelberg, 2006), Lecture Notes in Computer Science, Springer, pp. 404–421.10.1007/11853565_24Search in Google Scholar

19. Sarcar, S., Jokinen, J. P. P., Oulasvirta, A., Wang, Z., Silpasuwanchai, C., and Ren, X. Ability-based optimization of touchscreen interactions. IEEE Pervasive Computing 17, 1 (Jan. 2018), 15–26.10.1109/MPRV.2018.011591058Search in Google Scholar

20. Sarcar, S., Joklnen, J., Oulasvirta, A., Silpasuwanchai, C., Wang, Z., and Ren, X. Towards ability-based optimization for aging users. In Proceedings of the International Symposium on Interactive Technology and Ageing Populations, ITAP ’16 (New York, NY, USA, Oct. 2016), Association for Computing Machinery, pp. 77–86.10.1145/2996267.2996275Search in Google Scholar

21. Schilit, B., Adams, N., and Want, R. Context-aware computing applications. In 1994 First Workshop on Mobile Computing Systems and Applications (Dec. 1994), pp. 85–90.10.1109/WMCSA.1994.16Search in Google Scholar

22. Schmidt, A., Beigl, M., and Gellersen, H.-W. There is more to context than location. Computers & Graphics 23, 6 (Dec. 1999), 893–901.10.1016/S0097-8493(99)00120-XSearch in Google Scholar

23. Schon, D.Reflective Practitioner, 1983.Search in Google Scholar

24. Wobbrock, J. O. Situationally aware mobile devices for overcoming situational impairments. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Valencia, Spain, June 2019), ACM, pp. 1–18.10.1145/3319499.3330292Search in Google Scholar

25. Yuksel, B. F., Oleson, K. B., Harrison, L., Peck, E. M., Afergan, D., Chang, R., and Jacob, R. J. Learn piano with BACh: An adaptive learning interface that adjusts task difficulty based on brain state. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16 (New York, NY, USA, 2016), ACM, pp. 5372–5384.10.1145/2858036.2858388Search in Google Scholar

Received: 2020-10-02
Revised: 2021-03-17
Accepted: 2021-03-29
Published Online: 2021-04-08
Published in Print: 2021-07-27

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Scroll Up Arrow