Who’s in charge?


 Fully automated drive still lies far ahead in the future. Therefore, vehicles with multiple modes of operation will not disappear fully as many road types, traffic and weather conditions will not allow fully automated drive. Instead, fragmented trips with regard to automation will prevail, where drivers will have different levels of automation available at different times. Given this scenario and the complexity of vehicles offering multiple levels of automation with different driving modes depending on prevailing conditions, the need for drivers to understand their responsibility during the different modes becomes critical. The aim of this paper is to contribute to further understanding of how perceived control influences the driver’s mode awareness of and responsibility for the driving task by reporting on an on-road Wizard-of-Oz study under real driving conditions. The results show that when confronted with a vehicle offering both a level 2 and a level 4 driving automation system, drivers have difficulty in determining whether control is allocated to them or to the system. Further results show that perceived control and responsibility for the driving task are closely linked, and that the driver’s perception of the driving system influence how they interact with it. Finally, conclusions are drawn regarding the way perceived control influences mode awareness when interacting with a vehicle that features multiple levels of automation.


Introduction
The automotive industry is rapidly developing driving automation systems (DAS) with the aim of supporting drivers through automation of longitudinal and/or lateral control of the vehicle. According to the SAE International standard J3016 [15], these driving automation systems can be classified into five levels depending on technical system characteristics. This technical classification ranges between level 0 'No Driving Automation' to level 5 'Full Driving Automation', whereby each level describes different tasks for driver and system, i. e. who is responsible for which task during the driving activity. Nowadays, vehicles commonly offer Level 1 (longitudinal or lateral control) and Level 2 (partially longitudinal and lateral support) systems, some manufacturers even aiming at offering Level 3 'Conditional Automation' systems. Thus, the automotive industry is moving rapidly towards Level 4 'High automation', which assumes full longitudinal and lateral control of the driving task in a defined scenario. However, vehicles with multiple modes of operation will not fully disappear as many road types, traffic conditions and weather situations will not allow fully automated drive on all road types in the near future. Instead, fragmented trips with regard to automation, where drivers will have different levels of automation available at different times depending on road conditions and function availability, will remain dominant for quite some time [6].
As the complexity of vehicles equipped with multiple automated systems increases and different driving modes are offered depending on conditions, new attentional demands are created. The driver must keep track of the automated system and its modes of operation [16], as control of the driving task is distributed between driver and vehicle and new allocation strategies arise regarding control authority and responsibility [4]. Confusion may occur in this complex interaction mode when a situation or system mode is falsely classified and an action is taken that would be appropriate for the assumed system state, but not the actual [17]. In the case of driving automation systems, this could mean that the driver believes that the vehicle is handling both longitudinal and lateral control, while in fact the vehicle is only exerting longitudinal control, Figure 1: Automation levels and control transition spectrum (adapted from [4]). and this confusion may adversely affect both road safety and the driver's experience of using the system. Therefore, the need for drivers to understand their control over the vehicle and their responsibility during the different modes becomes critical.

Control and responsibility between driver and vehicle
Historically, the allocation of functions to a machine has been motivated by the wish to relieve the human from monotonous and/or strenuous tasks [18]. Initially, the primary aim behind the introduction of driving automation was to address traffic safety problems by supporting the prevention of accidents [9]. While there is a great potential for these technologies to increase traffic safety and reduce crashes, it is important that driving automation systems are designed and integrated in such a way that drivers are supported in building a mental model of the systems capabilities and limitations. Accordingly, an accurate mental model is crucial for a safe interaction with an DAS, since it reflects the driver's knowledge about the systems purpose, its function and possible driving modes [14]. Lee and Seppelt [10] have summarized the challenges of designing automated systems, stating that they come down to a technologically focused approach, disregarding the human operator who reacts to changes in feedback, changes in tasks and task structure, and the cognitive and emotional responses. All in all, discussions regarding appropriate levels of automation and the transition of control between operator and automation argue that the automation level must be adjusted with regard to the operator's involvement, designed for collaboration with and the building of appropriate trust in the automated system [8,10,11]. Specifically, in the context of driving, the allocation of control becomes critical, bearing in mind the dynamic and continuous nature of the driving task [5,7].
Flemisch et al. [4] therefore proposed a taxonomy for levels of automation, taking into consideration the authority to change control allocation. Their model describes different levels of automation in general, depicting a range from manual control, through low automation and semi-automation, to high automation and finally full automation. Further, Flemisch et al. [4] illustrate how control can be transferred between the human and the system, mapping the control authority and level of involvement. Figure 1 illustrates how the human has control authority only at one end of the spectrum and the machine only at the other end. It is noted that within the different levels of automation, control allocation can vary and must be defined in terms of the system functions and the responsibility that users have for the task. The arrows indicate who has the authority to pass on control. According to Flemisch et al. [4], the human is allowed to change control allocation in both directions, while the machine is only allowed to propose a change in control, but not actually implement the change. One important differentiation Flemisch and colleagues [4] make with their description is that even though different regions of control authority can be identified for both human and machine, each actor has certain abilities and therefore not every control allocation might be possible. Instead, they propose contextsensitive control allocation, based on whether the human operator or the machine has the ability to handle different situations. This distinction is particularly relevant in the context of driving automation. This dynamic shift of control between driver and vehicle depending on the context makes the assessment of control authority between driver and vehicle more complex. Therefore, it is important to distinguish actual control from the control that the driver perceives he/she has. For this reason, the term 'perceived control' will be used throughout this paper to refer to the driver's perception of control of the driving task.
Additionally, perceived control influences the drivers' perception of their responsibility for the driving task [4]. However, there are many factors influencing when we feel responsible; particularly in the context of driving, there are many laws and regulations that define the realm of responsibility and suitable actions for the driver. Therefore, it is helpful to divide responsibility into subjective responsibility and objective responsibility. Subjective responsibility refers to the perceived responsibilities of the driver while objective responsibility is described in the instruction manual or from a legal point of view [4]. This paper focusses on subjective responsibility, which is defined as the driver's perceived moral and legal obligations that are present in the context of automated driving.
The complexity of a vehicle offering multiple levels of automation can cause a number of issues if drivers do not understand their role. The interaction between driver and vehicle must therefore work satisfactorily if the automation is to contribute to increased traffic safety and to a positive user experience. Thus, it is of considerable importance to address the perceived control and responsibility that drivers have when using automated technologies in order to support the development of driving automation systems that clarify mode awareness and driver responsibility for the driving task under different conditions and in different driving modes.
Monk [12] differentiated between two types of mode awareness. First, the awareness of the existence of different levels of automation and second, the awareness of the currently active mode. Both types of mode awareness are particularly interesting for the focus of this paper, as they give conclusion about the users' understanding of the automated vehicle and their responsibility for the driving task during the interaction with a vehicle offering several levels of automation.

Research questions and hypothesis
This study investigates how driver perception of control influences their feeling of responsibility for the driving task in a vehicle with multiple levels of automation, i. e. level 2 and level 4 automation. Data from interviews conducted after an on-road driving session was used to understand the factors influencing users' feelings of responsibility and their understanding of the different driving modes. It is assumed that: -Perceived control influences the driver's responsibility over the driving task. -Perceived control influences the driver's mode awareness.

Methodology
This paper is based on insights from an empirical on-road study which took place in the San Francisco Bay Area in the United States in June 2019. In the study the participants experienced two different driving modes, a level 2 partial automation system and a level 4 high automation system [15], in a Wizard-of-Oz (WOz) car. As this paper focuses on exploring how the driver perceives their responsibility over the driving task, during the initial engagement with a vehicle offering different levels of automation, the authors have decided to conduct semi-structured in-depth interviews. Interviews as a data collection method are a reliable choice, when exploring different perspectives and reflections about the users' perception and the users' mental model of the driving automation [2].

Equipment Vehicle
In order to simulate a realistic use experience for the participants and to be able to test two levels of automation a WOz setup was created. The test vehicle was a modified Volvo XC90, where a test driver could take over control of the vehicle with concealed vehicle control devices from the back seat, and thereby, simulate the level 4 automation system. The vehicle was modified according to all relevant road permission standards and was approved for road testing by the local authorities, which enabled testing in a real driving context.

Systems
The study included a vehicle with two modes of operation, defined as automation level 2 and automation level 4 according to the SAE [15] standard. The level 2 system was an assistive function that helped in maintaining speed, automatically adjusting the vehicle's speed in response to other objects moving in front and also provided lane keeping support. It could be activated at any time by the driver by pushing a button on the existing steering wheel interface. When active, the system status was indicated by an icon on the instrument cluster display and if the vehicle registered that the driver was not keeping their hands on the steering wheel, they were prompted with a visual and auditory cue. The level 4 system was only operational in dense traffic conditions. When these conditions were met it would offer to take control of the driving task, through auditory and visual cues, making it possible for the driver to perform other tasks. To activate the system, the driver had to make a sustained press on a button on the existing steering wheel interface. When the level 4 system was activated the instrument cluster display changed appearance, the centre panel HMI changed view to display apps, such as YouTube, and the steering wheel was locked. Once the conditions were no longer met, the driver was asked to take back control through a visual and auditory cue and the driver had a maximum of one minute to take over control.

Participants
The study involved 20 participants, 11 females and 9 males, whose ages ranged from 22 to 62 years (Mean = 42, SD = 14). They were recruited and reimbursed through a local agency. All participants had to be holders of a valid driver's license, and they had to drive a car equipped with Common Cruise Control (Driver Assistance System which maintains a steady set speed) and an automatic gearbox. All the participants were frequent drivers and all but one commuted to work by car daily.

Study design and procedure
During the drive all 20 participants experienced both modes. However, while the participants could choose to engage with the level 2 system whenever they wanted, they could only use the level 4 system when they received a prompt, indicating that the necessary traffic congestion conditions were fulfilled, and the system was ready. The study consisted of three phases during which data was collected, applying a mixed methods approach. This comprised personal interviews and on-road observations, and the interviews were analysed for this paper.

Phase 1
Before the driving session, the participants were introduced to the study and provided their informed consent in accordance with the General Data Protection Regulations (GDPR). The participants were interviewed before the drive regarding their expectations from vehicles with highly automated systems and received written and verbal information about the vehicle's two modes of operation, along with an explanation of the capabilities of the respective modes, how to interact with them, and their responsibility in each driving mode.

Phase 2
The driving session took approximately 90 minutes and included highways and urban areas in the San Francisco Bay area (see Figure 2), during which the participants were encouraged to try out the functions as much as they wanted and to think out loud, i. e. verbalizing their thoughts about the interaction with the system. All the drives were conducted during rush hour, either in the morning or the evening, to ensure both congested traffic and free-flowing traffic for all the participants.

Phase 3
After the driving session, the participants were interviewed regarding the control they perceived when driving with the different systems. The interviews were semistructured, with a set of open questions that all participants were asked. The questions investigated if the participants felt in control and responsible when using the systems, as well as what made it clear when they were responsible. Afterwards, the participants were informed that the level 4 system was simulated using a WOz approach, something they were not aware of during the test.

Coding and analysis
As a first step, the interviews were transcribed verbatim. From the transcribed material, statements were extracted in which participants described the control they perceived in each driving mode, and whether they always felt responsible for the driving task. A thematic analysis was then conducted using an inductive coding approach, in order to explore how the participants experienced their control and responsibility. The statements were coded into two groups: 1. factors affecting the perceived responsibility, i. e. statements explaining why participants felt high or low levels of responsibility; and 2. links between perceived control and responsibility, i. e. statements discussing the connection between control and responsibility. The statements regarding why participants felt high or low levels of responsibility were further analysed by dividing the state-  (3) System is steering, accelerating, braking (1) Monitoring, Steering (9) System performance very good, does not need driver input (1) Being in the driver's seat (2) Hands-on-wheel icon (2)

Level 4 High Automation
Activating/Deactivating the System (4) System taking over complete driving task (6) Being in the driver's seat (10) System tells driver when intervention needed (3) ments based on whether they related to the level 2 or level 4 system, and how they perceived their responsibility, in order to identify and compare factors that had a positive or negative impact on the participants perceived responsibility when using each system.

Findings
During the analysis, several patterns were identified describing how drivers perceived control and responsibility. Their general view of responsibility is presented along with different factors influencing driver perception.

Perceived responsibility
Based on the answers to the interview questions, driver perception of whether the driver or the system was responsible for the driving task during the two different driving modes was analysed. This resulted in categorization of the perceived responsibility into high and low responsibility and identification of factors influencing that perception. The level 2 system is a form of assistance and requires the driver to be in control and responsible for the driving task at all times. The level 4 system, on the other hand, completely takes over the driving task under certain conditions so the driver is not responsible for the driving activities during that period. When enquiring about their responsibility during use of the two systems, the preferred outcome would be that the objective responsibility and subjective responsibility match. The results, however, showed a discrepancy resulting in only two drivers whose perceived (subjective) responsibility matched the objective responsibility, i. e. level 2 -driver responsible; level 4 -vehicle responsible. The majority of participants felt responsible for both levels and there were indications of ambiguity over who had control during which driving mode.
A summary of the factors influencing the drivers' perceived responsibility during usage of the level 2 and level 4 system is presented in Table 1.
The feeling of responsibility was partially motivated by the fact that the drivers were still in the driver's seat and that they had the ability to activate and deactivate the system, or simply the fact that a driver is always responsible for the vehicle and what happens around it. This points to a more moral and legal accountability that drivers see and accounts for this feeling of responsibility. Other reasons why drivers felt responsible was the amount of control they had over the driving task. In level 2, for example, the feeling of responsibility was perceived because they were still steering and monitoring, although some participants explained that the steering assistance was so good, they thought they did not need to intervene. Hence the ambiguity of responsibility in this driving mode. A further indicator for responsibility was the hands-on-wheel icon, which accompanied by auditory feedback tells the driver that they have to steer the vehicle in case they let go of the steering wheel. However, this was overlooked by many and often only recognized after the audible warning chimed.

Linking responsibility and control
The ambiguity and uncertainty most drivers experienced about their responsibility were influenced by the level of control drivers perceived to have over the vehicle. While the ability to activate and deactivate the systems at any time enhanced their feeling of control and responsibility, the perception of the system being extremely capable and not needing human input decreased their feeling of responsibility. The illustration in Figure 3 provides an overview of how the participants perceived the systems' control of the driving task and how this related to whom they felt was responsible. Especially regarding the level 2 system, the participants expressed a high level of trust in its performance, leading them to feel less responsible. This perception was due to the system taking over acceleration, braking and maintenance of distance to vehicles in front, as well as the steering support, which led many participants to feel as if they were sharing control with the car. In some cases, the drivers felt as if they were competing with the car for control as the steering support gave them a feeling that the car wanted to take over control and that they could therefore disengage. Figure 3 illustrates how perceived control affects responsibility and highlights the fact that the participants were ambiguous about the perceived control in level 2 driving mode. As indicated by the vertical arrows, if the system was perceived to have high control, the driver was perceived to be less responsible and vice versa. The participants expressed that they interpreted the system's feedback as though it was controlling some of the driving task, but it was difficult to interpret how much of the driving task was handled by the system and how much control they themselves were supposed to have.
When the level 4 driving mode was activated the steering wheel was locked and the vehicle completely took over the driving task, which caused heightened ambiguity as to who had control. Most drivers perceived the system to have high or full control over the driving. However, there was a big difference in how the drivers perceived their responsibility. Many still felt responsible even if they did not feel like they were in control owing to lack of trust, and also because as the driver they were able to activate and deactivate the system. This led to a mismatch, as illustrated by the diagonal arrow in Figure 3. Some drivers felt responsible even though the system was perceived to have high control, saying that they always had responsibility when driving and also because they still had the ability to turn off the system.
In summary, in level 2 it seems to be difficult to judge the degree to which the system controls the driving task, while in level 4 drivers had difficulty in letting go of the feeling of responsibility. Ultimately, control allocation and authority during the different driving modes caused confusion, which led to the perceived subjective responsibility not matching with the objective responsibility. Such confusion is risky and indicates a lack of mode awareness or understanding of the systems.

Discussion
This paper focuses on the effect perceived control has on driver responsibility and mode awareness. The experiment was run under real-life driving conditions imitating a level 2 and level 4 driving automation system through a WOz approach. The results show a high level of interconnectedness between the concepts of control, responsibility and mode awareness.
Perceived control influences driver responsibility over the driving task From the results it seems that perceived control affects perceived responsibility in different ways for the different driving modes. In level 2, the high control that users assume the system to have, and the good system performance, led to the conclusion that they were not responsible for the driving task. This connection between the level of perceived control and responsibility is partly supported by Siebert et al. [19] who found that perceived responsibility was influenced by participants' perceived ability to generally control the vehicle. The factors identified through the interviews show that aspects like the system's steering support and feedback through icons are direct influencers of the perceived control that users feel they have. Furthermore, the results suggest that there is a conflict within the feedback that drivers receive. Specifically, the hands-on-wheel icon competing with the system offering advanced support, for instance steering, accelerating and braking, which causes ambiguity over control authority. The same observation was reported by Wilson et al. [20], who found that a hands-on-wheel system alone is not adequate to maintain driver engagement, as there were other factors which indicated they could disengage from the driving task, thus contributing to mode confusion. These assumptions are supported by Flemisch and colleagues [4], who describe the triple bind between responsibility, control and authority and point out that the balance between these concepts needs to be designed with consistency in mind across the different automation levels in order to avoid mode ambiguity. Thus, suggesting a transparent system design regarding the nature of the driver's role is essential in supporting understanding of control authority and therefore driver responsibility in the different driving modes.
Even though control is a major indicator, it is not the only factor supporting drivers' understanding of their responsibility for the driving task. In level 4 it can be seen that when the system exerts a high degree of control it supports users in understanding that they are not responsible for the driving task, even though other factors nevertheless made them feel responsible. These factors were not technical or cognitive but rather underlying and indirect influencers, which can be described as legal, ethical or individual, and need further investigation. The results show that some drivers felt in control even though they understood that they were not in charge of the driving task, because they had activated the function or because they could take over at any time. However, this also made it unclear for some drivers where their responsibility lies and whether they could disengage completely or were still responsible for monitoring the system, for example.

Perceived control influences the driver's mode awareness
The results suggest that control and system understanding are highly interconnected, as the perceived control communicates to the driver what their role is during the act of driving and therefore supports an understanding of the system modes. However, prior results showed that when describing the two driving modes, the respondents were not able to really differentiate between the different modes based on the perceived control as there was considerable ambiguity [13]. These findings further indicated that users' understanding of driving modes is dependent on the extent of control they perceive. This was also reported by Endsley [3] who during a six-month trial of a vehicle with a level 2 system noted several accounts of mode confusion based on driving mode changes that offered no clear feedback to the driver. Consequently, one may assume that if users have to interact with a vehicle featuring multiple levels of automation which are not clearly distinguishable to them, they will not be able to develop an accurate mental model. The ambiguity of perceived control over the system can therefore result in mode confusion. This implies that mode awareness is the result of a correct mental model, making users' understanding of the system absolutely critical.

Mode awareness clarifies responsibility
The lack of mode awareness can lead to inadequate and unsafe usage strategies, which will result in serious consequences. Frequent reports of drivers misusing driving automation systems by not paying attention to their driving, or assuming they can release the steering wheel during partially automated driving modes, show the implications of an ambiguously designed system: complacency and over-trust [1,3]. Looking back at Monks' definition of mode awareness, there seems to be the need to broaden that definition by including not only the awareness of the existence of different levels of automation, or the awareness of the currently active mode [12], but also the understanding of what each driving mode is capable of. The understanding of who is in charge of what, in each driving mode, means users understand their responsibility for the driving task at any given point in time.

Conclusion
The introduction of automated driving systems into nowadays vehicles comes with challenges regarding the control allocation between driver and vehicle. This study aimed to investigate the connection between perceived control and responsibility over the driving task, and how the driver's mode awareness is influenced by this.
The authors concluded that perceived control influenced the drivers feeling of responsibility over the driving task, during the initial usage of a driving automation system. Generally, the more control the system exerted, the less responsible the driver felt, due to the impression that the system had authority over the driving task. During the study it was found that this impression is influenced by visual and haptic feedback the drivers perceive from the system. Further, it was concluded that perceived control influenced the driver's mode awareness. The results showed that only 10 % of the participants understood the different modes and their responsibility correctly, showing that there is a mismatch between their mental model and the system design. Broadly translated, this implies that mode awareness is not only the awareness of the mode active, but also the awareness of what each mode of the system does, and how this affects my responsibility over the driving task.
In summary, we conclude that perceived control affected drivers' subjective responsibility for the driving task. Thus, a transparent system design, clear feedback, and coherent system behaviour throughout the different driving modes could inform the user's mental model and lead to a better understanding of their responsibility for the driving task. However, there are also indirect influencers, such as ethical and individual factors, which affect drivers' perception of their responsibility, and these need deeper investigation.
Consequently, in order to support the development of an accurate mental model, mode awareness and responsibility, further work should investigate how to design transparent feedback, which supports interaction with driving automation. Furthermore, since this study focuses on the initial usage of automated systems, future research should explore how long-term usage affects the driver's perception of their responsibility and their mode awareness.