Measurement and classification of human characteristics and capabilities during interaction tasks

Abstract In this paperwe address the need to design adaptive interacting systems for advanced industrial production machines. Modern production systems have become highly complex and include many subsidiary functionalities, thus making it difficult for least skilled human operators interact with them. In this regard, adapting the behavior of the machine and of the operator interface to the characteristics of the user allows a more effective interaction process, with a positive impact on manufacturing efficiency and user’s satisfaction. To this end, it is crucial to understandwhich are the user’s capabilities that influence the interaction and, hence, should be measured to provide the correct amount of adaptation.Moving along these lines, in this paper we identify groups of users that, despite having different individual capabilities and features, have common needs and response to the interaction with complex production systems. As a consequence,we define clusters of users that have the same need for adaptation. Then, adaptation rules can be defined by considering such users’ clusters, rather than addressing specific individual user’s needs.


Introduction
In the last decade, modern fabrication and production systems have been becoming increasingly complex due to market demands and technological progress. Needs for fast and flexible production have called for subsidiary machine functions, such as fault diagnosis and fast recovery, fine-tuning of process parameters, and fast reconfiguration of the machine parameters to adapt to production changes.
Although technological progress has made most of operations automatized, the presence of human operators is still fundamental; their role has changed from operational, to supervising them and taking proper action when something goes wrong. As a consequence, most of the complexity of modern production systems is transferred to human operators who are ultimately responsible for production. Additionally, human operators are often responsible even if they are not provided with enough information about the process and its past states in order to make the right decision [1]. In other words, complex production systems imply complex interaction dynamics for the operators to deal with.
Interaction with machines, to control, program and supervise them, is provided by means of human-machine interfaces (HMIs). These have a significant impact on production performance, since they are by far the main way an operator can achieve proper situation awareness about the current status of the machine and the plant [2,3]. Several methods can be found, in the literature, that propose guidelines for the development of HMIs explicitly considering the way users perceive information, process data, and take actions [4]. However, customarily, HMIs are static and do not change their behavior as a real-time function of the status of the current user [5]. This has twofold implications. On the one side, if user's characteristics change while she/he is working, the HMI does not change accordingly: as an example, if the user gets nervous or fatigued, the HMI does not offer any simplification or adaptation. On the other side, most of the times, no different user's profiles are considered: this implies that there is no distinction of functionalities enabled to users with different experience levels. Even if operator login is required, this is typically used to track operator's activity. Different characteristics of operators are regarded to without distinctions, although they affect the interaction. For example, consider the case of operators with learning disability or limited cognitive capabilities, compared to expert operators (with long standing working experience). Unavoidably, using the same HMI for them is ineffective, since they have different needs in the interaction: while the former will need support and guidance, the latter will benefit from fast navigation through the HMI, macros and customized features. Therefore, if the behavior of a machine while interacting with a human operator and its HMI can adapt to the characteristics of the user, then the overall interaction process will be more effective, with a positive impact on manufacturing efficiency and user's satisfaction [6].
In this regard, adaptive automation has been considered [7,8]. According to this design paradigm, levels of automation should vary depending on situational demands during operational use, as proposed in [9][10][11]. Adaptive user interfaces allow to change how the information is presented so that only relevant information is provided to users by including the environment and the user as part of the monitored system through adaptive HMIs.
Adaptive user interfaces have been developed and implemented in different domains, such as automotive [12][13][14], aeronautics [15] and smartphone and hand-held devices [16]. However, very few partial attempts and preliminary results on the development of adaptive HMIs for complex industrial systems have been reported [8,17]. In [6] we have recently proposed an integrated methodological approach, referred to as Measure, Adapt and Teach (MATE), which consists of devising complex interaction systems (either automatic machines or robots) that measure the current operator's status and adapt the interaction accordingly, or teach the lacking competence.
The design of adaptive interacting systems is interested by two open problems: 1. how to measure users' characteristics and capabilities: (a) what parameters reflect user's capabilities relevant from the point of view of interaction with complex systems: specifically, which critical parameters need to be monitored to reduce negative interaction experience; (b) what measurement techniques are available, considered the constraints on environment, privacy, need for freedom of movement, protective equip-ment to be worn while operating the machine, etc.; 2. how to optimally adapt the interaction to the measured capabilities.
This paper falls in the realm of the first problem (1a). Specifically, a preliminary analysis of relevant capabilities and associated parameters to measure has been presented in [18], and it will be recalled hereafter in Subsec. 2.1. Issues related to measurement techniques have been partially addressed in [6], where different measurement schemes were considered. These are recalled in this paper in Subsec. 2.2.
As regards problem (2), we refer to the fact that the interaction can be customized by adapting the behavior of the system and/or the user interface to the measured user's characteristics. On the one side, just to mention a couple of examples, the process can be adapted by changing the working mode of the machine (from manual to automatic, to switch from flexible to batch production) and varying allocation of tasks between the machine and the human operator for job rotation. On the other side, the goal of HMI design in this regard is to find the rules for adaptation and support that best match user's characteristics. While some general methodologies can be defined, such rules need to be mostly determined considering the specific application at hand. Generally speaking, in the case of procedural tasks, interaction can be adapted to user's skills by providing a varying amount of guidance in following the correct procedure: information about the list of activities to be performed can be shown on the HMI, thus preventing wrong choices. In the case of faults and malfunctioning system, the quantity and the kind of alarms that are presented to the user should be adapted, based on her/his cognitive status. For example, if difficulties in the interaction are found by user's measurements, low-priority alarms (i.e., those alarms representing non-critical issues) should be hidden, to let the user focus on high-priority ones, which represent more critical situations that can not be neglected.

Contribution and organization of the paper
Building upon discussions initiated in [18] and [6], in this paper we aim at identifying groups of users that, despite having different individual capabilities and features, have common needs and response to the interaction with complex systems. Accordingly, such an insight will allow to define users' clusters for which adaptation rules can be defined (problem (2)). On the one side, defining adaptation rules for users' clusters is much more doable than defining rules for each single user; on the other side, the way such clusters are derived guarantees that the rules addressing users' clusters accommodate the needs of any single user during the interaction with complex systems. As shown in Fig. 1, the work flow that we propose is that, given the characteristics and capabilities that influence the interaction, these are measured while (or before) the user is using the system. From the assessment carried out in this paper, it is then possible to understand if the operator is currently experiencing some difficulties and needs to be supported in the interaction task. Thus, the cluster she/he belongs to can be identified and, hence, the optimal adaptation of the behavior of the system to support her/him can be selected. Whichever adaptation is provided, it is aimed to support the user during the interaction and relieve difficulties. Although the means to provide adaptation do not fall in the scope of this paper, it worthwhile briefly mentioning that the part of the system that is most likely to be adapted is the user interface (e.g., hiding complex functions or providing descriptions and instructions for alarms), but also the behavior of the machine might be changed, for example increasing its autonomy by switching to batch production.
The paper is organized as follows: in Sec. 2 the problem of how to measure users' characteristics and capabilities (problem (1)) is discussed, with specific reference to problem (1a) and problem (1b). Then, in Sec. 3 the effect on human-machine interaction of user's characteristics and capabilities is analyzed and users' clusters are derived in Sec. 4. Finally, Sec. 5 follows with some concluding remarks.

Measurement of human capabilities
In this section, we briefly recall possible methods to measure users' characteristics and capabilities (problem (1)).
In particular, in Subsec. 2.1 an overview of the user's characteristics that are relevant from the point of view of interaction with complex industrial systems is presented, in response to problem (1a). Among them, those whose effect on the interaction could be reduced by providing specific support to workers are selected in Sec. 3 and their effect on interaction is analyzed therein. In Subsec. 2.2 we discuss a possible measurement approach (problem (1b)).

Human characteristics relevant during interaction
An extensive analysis of which human characteristics and capabilities affect the interaction with socio-technological systems has been presented in [18]. Therein, humanmachine interaction is regarded as informational work, in which the user at first receives information about the current status of the machine and work progress through the HMI. Second, an action has to be prepared after making a decision. Finally, by giving information input to the HMI, machine movements are released. These processes basically can be referred to three stages of information processing: perception, cognition, and action [18]. Additionally, informational processes strongly depend on individual's personal characteristics. According to [19], these are grouped in four different categories: constitutional characteristics, which are parameters that cannot be changed during a life cycle; dispositional characteristics, which can be changed during the life cycle, but cannot be directly influenced; qualification and competence, which describe earned capabilities that can be influenced willingly; and situational characteristics, which are the most variable since they depend on the current situation. These capabilities are shown in Fig. 2. They are, then, further specialized in terms of parameters and attributes that allow to quantify them.
An exhaustive list of parameters and attributes and a validation approach can be found in [18]. By investigating three representative production use cases (woodworking machine, robotic cell, industrial plant for bottling and labelling), we analysed the importance of the user's attributes and ranked them accordingly by pairwise comparison. We found a tendency of cognitive attributes to be estimated with the highest values, since modern production machines require high concentration and attention over a long period of time, and experience, e.g. in forms of factual knowledge. However, results also showed clearly that the variance of different use cases is high, which indicates the difficulty of providing an overall industry standard. Further, results showed that the need for supporting perceptive attributes, such as visual capabilities, tends to be estimated as least important. European Standards already consider most of these parameters. According to these results, the measurement of human capabilities regarding adaptive HMIs should focus on cognition, but also consider use case specific requirements.

Measurement approaches
To collect information about user's characteristics and capabilities, different measurement techniques need to be considered. A general scheme, proposed in [20], considers three different kinds of measurement: 1. A-priori measurements: they provide assessment tools regarding general characteristics of the user. This is a kind of off-line assessment, consisting of questionnaires regarding demographic questions, or tests for perceptive, cognitive or motoric capabilities. Moreover, skills and constitutional characteristics can queried by questioning. 2. Real-time physiological measurements: these are online assessment tools for real-time measurement of physiological indicators for mental workload causing strain, such as pupil diameter, blinking rate, skin conductance, cerebral activity, body temperature, hormonal balance and heart rate. They allow to measure the situational characteristics listed in Fig. 2. 3. Real-time performance measurements: they refer to performance indicators, such as time for decisions, executions steps for the task, mistakes, and redundancies.  Table 1 reports how the three measurement typologies support the measurement of human characteristics and capabilities identified in Fig. 2. In particular, the main difference between a-priori and real-time measurements is that the former are used to assess user's characteristics that change very slowly or, even never change, while the latter refer to a time span similar to the duration of the interaction session.

Effect of users' characteristics on the interaction
In the following, starting from the capabilities and characteristics reported in Fig. 2, we select those for which the measured value might introduce some difficulties in the interaction with industrial complex systems. In this regard, for example, worker's culture is not considered, since having one culture rather than another does influence the interaction (e.g., the reading direction or meaning of icons might change), but it does not introduce difficulties in it: Arabian workers need that the HMI is designed with reading direction from right to left, European that it is from left to right. Provided that the design of the HMI considers the specific needs of different cultures, then no difficulties are introduced in the interaction that require additional support. On the contrary, worker's age, education and other parameters discussed below make the interaction complex and call for additional support by the system.

A-priori measurements
The static users' attributes that do not change over time are: -age, -education, -computer skills, -impairments.
These attributes represent constitutional characteristics (age and impairments) or knowledge related to learned capabilities (education and computer skills). Their effect on the human-machine interaction in terms of amount of difficulty introduced is analyzed in detail for implementing users' clusters in the following.

Age
With regard to age, an adaptive interface should address potential barriers to interaction that prevent accessible working conditions to elderly people. Human-machine interaction mainly consists in informational work, and with age, several changes in physical conditions occur that are central influencing factors for human information processing. In general, informational processes can be distinguished into perception, cognition and action. Information is received by sense organs, further processed into a decision, and translated into a, e.g. verbal or motoric action. Age-related changes refer to all stages of information processing. For example, visual, auditory and haptic perception decrease, changes occur in memory functions, or poorer mobility or decreasing fine motor skills lead in lacks of strength or precision. Table 2 reports relevant age-related changes for human-machine interaction from [21,22].
Specifically, it is shown that, according to the literature, the central changes in physical conditions that cause significant impairment to people's capabilities mainly occur at two periods: -at the age of 30 to 40, when auditory perception, haptic perception, fluid intelligence and muscle strength start to decrease; -at the age of 50 to 60, when visual perception, auditory frequency perception and vibration perception, memory and learning worsen, and the mobility of the upper limbs becomes impaired.
Therefore, with respect to difficulty in human-machine interaction caused by ageing, different needs for adaptation of the interaction are to be considered, as shown in Table 3.

Education
The user's education can be assessed in a standardized way by means of the International Standard Classification of Education (ISCED) [51], which is the standard framework used to categorise and report cross-nationally comparable education statistics, established by the United  Nations Educational, Scientific and Cultural Organization (UNESCO). The ISCED provides nine education levels, ranging from level 0, which corresponds to early childhood education level, to level 8, which describes tertiary education level, for example a doctoral degree. Table 4 reports a brief description of the ISCED education levels; more details are reported in the Appendix and can be found in [51].
Since machine operators usually went through basic education level and are not required to have any advanced research education, a subset of education levels is relevant for human-machine interaction. In Table 5 we report how education affects interaction in terms of making it more difficult for less educated workers.

Computer skills
It is important to consider also worker's computer skills to avoid that the use of computerized systems is a barrier to the efficiency of workers with high experience on the task at hand. This is the case, for example, of those workers, typically elderly, who have great experience in the task, but have low computer alphabetization. Providing adequate assistance in the use of the computer, for example by means of metaphors, allows to leverage their knowledge.
According to the Organisation of Economic Cooperation and Development international research study [52], computer skills can be divided into four levels that are listed in Table 6. A detailed description of the different levels can de found in the Appendix. In Table 7 we report how different levels of the users' computer skills make interaction with complex technological systems more difficult.

Impairments
A comprehensive categorization of possible users' impairments is impossible to achieve, since they are determined by unpredictable circumstances, especially as regards physical ones. Nevertheless, it is easily understood that impairments strongly affect the interaction, making it more difficult and calling for support from the system.
The amount of difficulty introduced in the interaction depends on the kind of impairment in relation with the interacting task, and on its severity. An ordinal relationship on their effect on interaction cannot be established across different kinds of possible impairments: in other words, it cannot be concluded that having an impairment instead of another introduces greater difficulties in the interaction, since this depends on the task and environment at hand and the degree of severity of the impairment. For example, the effect of deafness on interaction is very different in the case of a noisy plant where operators have to wear protective earmuffs, or a plant where sound alarms are used very often. Moreover, different impairments with the same degree of severity have different effect on the interaction. In Table 8, we report how some of the most common impairments identified above affect users' capabilities. Their effect on interaction need to be considered case by case.

Real-time physiological measurements
Interaction is additionally influenced by strenuous situational conditions, such as noisy environments, tight schedules, the fear of job loss, and psychological pressure due to the presence of supervisors [6]. These induce mental fatigue and make interaction less efficient, increasing the risk of errors and slowing down the speed of execution of some tasks [53]. Depending on the actual user's strain level, an adaptation of the behavior of the system is beneficial, to relieve the user and limit decrease of production efficiency. This can be achieved by adapting the user interface and/or the process, that is the working mode of the machine and allocation of tasks between the operator and the machine.
Mental workload can be assessed by monitoring physiological response. Extensive surveys in this regard can be found in [53,54]. In particular, real-time physiological measurements consist in an initial recording of baseline condition of the user, which serves as a reference for measured quantities. Then, physiological parameters are recorded during the interaction task. Changes of the situational status of the user (i.e., strain, fatigue, motivation and mood) are estimated with respect to her/his presession condition. In other words, this baseline recording accounts for pre-session variability of user's status. Table 9 reports how mental workload increases the complexity of interaction. The significance of the table is related to the different levels of mental workload that can be discriminated by the considered measuring techniques. In Table 9 we consider a quite advanced case that allows discriminating among four different levels of mental workload [54]. Table 9: Amount of diflculty related to mental workload.

Real-time performance measurements
Finally, the optimal user's profile should be selected also keeping into account her/his task performance. This information can be used to develop structural knowledge maps of each operator, e.g. regarding her/his training evolution. Moreover, it would be beneficial for the user to be supported by means of HMI adaptation dependent on her/his level of experience, derived from performance indicators, such as execution time, steps, mistakes and redundancies. The degree of experience of an operator can be derived from Table 10.
Ultimately, monitoring the task parameters related to execution performance it is possible to keep track of user's experience and, accordingly, provide adequate support when she/he cannot complete the task efficiently. Table 11 reports how operator's experience relates to the complexity of interaction.

Users' clusters
Since the relationship between different users' characteristics and capabilities and their effect on the interaction with complex human-machine systems has been defined in the previous section, users' clusters can be derived. Specifically, this clusterization aims at defining which instances of different users' characteristics cause the same level of difficulty in the interaction. In other words, users in the same cluster experience the same difficulties in the interaction and, hence, share the same need for support, although their characteristics and capabilities are different. Then, the advantage of such clusters is that tailored interaction can be provided to any user, although the rules for adapting the behavior of the system (e.g., simplifying the task) or providing additional support (e.g., step-by-step guided procedures) are defined for few users' groups.
Following the distinction between a-priori measurements (which measure constant user's characteristics that reasonably will not change during the interaction) and real-time measurements (which measure user's characteristics that depend on the current interaction), the cluster to which an operator belongs might change during the interaction. In other words, distinction should be made between static and dynamic clusters. Static clusters result as an outcome of a-priori measurements and define how human-machine interaction is at the beginning of the task. During the interaction, if real-time physiological and performance measurements show that the user is experiencing additional difficulties in the interaction that were not foreseen by the initial static cluster, a new more appropriate cluster has to be dynamically selected for the user.
In Table 12 we report the clusters deriving from the parameters considered in Sec. 3. As discussed in Subsec. 3.1.4, among different possible impairments, only mild cognitive impairment is reported in the table, since the effect of the others cannot be generalized to any interaction scenario and different impairments cannot be put in an ordinal relationship with respect to difficulties introduced in the interaction.
A user belongs to a cluster whether her/his measured characteristics and capabilities match the definition for that cluster. In the case that user's features are spread over more than one clusters (e.g., age < 30 years old and high cognitive impairments), the cluster denoting the need for higher support should be selected (i.e., in this example, cluster 3). Additionally, given the amount of needed support resulting from a-priori measurements, this can be only increased by real-time measurements: in other words, if real-time measurements denote the need for lower sup-port than the current cluster, it should not be updated. The results of real-time measurements should be considered only if they highlight additional difficulties in the interaction and, hence, the need for further support.

Conclusions
In this paper we presented an approach to guide the design of adaptive interfaces for complex industrial machines, including different kinds of users with diverse capabilities. The proposed method builds upon the MATE approach (Measure, Adapt and Teach) which addresses specific user capabilities by adapting the interaction and giving additional training to necessary skills and expertise. The objective of MATE systems is to improve process productivity by reducing stress induced by the HMI, resulting in lower individual strain and improved interaction.
The implementation of adaptive interacting systems requires a classification of relevant user capabilities that can cause barriers in human-machine interaction. For this purpose, based on an analysis of user groups and requirements, we identified several user attributes relevant during interaction that may introduce difficulties in the use of interacting systems and, hence, call for adaptation to user's characteristics. For this reason, we developed users' clusters that summarize similar attributes for several users, e.g. age spans with regard to perceptive, cognitive or motor changes. However, when clustering, not all limitations of the users will be considered individually. Though, it is not suitable to consider attributes for each user individually in this regard, because the feasibility for real working environments decreases with the degree of customization of the system. Further, specific adaptation rules can be formulated, according to the cluster level specifications.
As future work, we are currently working at the combination of the different characteristics and capabilities of the user. Our idea is that of determining which characteristics are more relevant for interaction, depending on the task, and hence should be given more importance in the selection of the optimal level of adaptation. Specifically, this would be useful in the case of users having characteristics that apply to different clusters, thus calling for different levels of adaptation. To this end, user's cluster would be determined as a weighted sum of the different characteristics. The weights depend on the kind of tasks the user has to carry out.
Moreover, it would be interesting to investigate the valence of subjective reporting by the user on her/his own status in addition to objective assessment by means of the measurement approaches discussed in Subsec. 2.2.