User interfaces for automated vehicles

In the area of automotive user interface research, a lot has changed during the last years and since the last it – Information Technology special issue on automotive user interfaces in 2012 [3]. With the shift towards assisted and automated driving, the rising importance of climate change, and the appearance of novel mobility forms, new challenges and opportunities for research have appeared since then. A lot of research activities shifted from investigating how to support interactions in and around manually-driven cars towards understanding how to design vehicles in the transition towards automation and with alternative propulsion technologies from a human perspective. This relates both to interaction aspects inside and outside of the vehicle.

Looking at the interaction within the car, novel human-computer interaction (HCI) aspects refer, for instance, to the design of control transitions between the driver and the automated vehicle and how to support non-driving-related tasks [2] once the driver can assume a passenger role in the (partially) automated car. Especially with a gradual transition towards automated driving, we see many challenges on how to shift responsibilities between the car and the driver, or how to implement shared control [4]. This also involves aspects on creating and maintaining situational awareness and especially finding solutions to establish and maintain trust while leveraging user acceptance of automation technology.

Furthermore, the role of infotainment becomes even more important as we can imagine that the interior may convert into a mobile office, living room, dining room, or maybe even support relaxation and sleeping – at least in the far future [1]. This inevitably translates to a need for rethinking the interior, finding novel interaction concepts, and investigating how, for instance, novel materials, interaction methods, or user interfaces can assist drivers and passengers and make the ride more comfortable.

Once the car takes over control, another arising challenge is the design of the communication with other road users such as pedestrians, cyclists or drivers of manually driven cars: As the driver does not have to monitor the road and may be involved in non-driving-related activities, other road users may lose the ability to communicate with the driver (e. g., through gestures and gaze) and technical solutions may be needed to replace this traditional form of communication. The influence of such communication concepts on the driving behavior, traffic flow, and their social acceptability can play an important role in future traffic situations.

The major research direction for automotive user interfaces has significantly changed from manual driving towards automated driving modes since the last it special issue on this topic. Thus, with this current special issue on automotive user interfaces for automated vehicles, we want to share insights into hot topics in this field and outline the current challenges and research directions.

While the reality of automated driving is approaching fast, there will be a transition period where partial automation will be the norm. There will be traffic situations, weather conditions, or other circumstances where a partially automated vehicle may not be able to cope and would call upon the assistance of the human driver. Within the span of a journey in an automated vehicle, there may be different segments that require different levels of engagement and input from the human driver. In this context, it is critical that the driver understands the mode and the level of automation the vehicle is currently operating on, and their corresponding responsibilities and tasks. In their paper “Who’s in Charge?” Fjollë Novakazi, Mikael Johansson, Gustav Erhardsson, and Linnéa Lidander study in a real-world experiment with 20 participants how the perceived control can influence a driver’s awareness of automation mode and responsibility. They also show that typically it is difficult for drivers to understand which aspects of vehicle control lie with themselves. This work contributes by highlighting the need for transparent system design and clear feedback in the development of automated vehicles.

Take-over situations in automated driving context are a crucial and timely topic. Drivers need to shift their attention back to the driving context and get ready for maneuvering. Several researchers have conducted studies investigating drivers’ behavior and responses at takeover situations. However, little is known about the underlying cognitive factors or the driver’s experience. The article “When Does the Driver Feel Ready to Drive Again after Automated Driving? – A Qualitative Approach” by Lara Scatturin, Rainer Erbach, and Martin Baumann presents the results of a simulator study, in which 28 participants experienced three transitions – two non-critical and one critical, and were asked, “when they felt ready to drive” after the takeover retrospectively. By content analysis, their statements were categorized to different themes describing driving task, automated system, cognition, situation, motor task, and perceived duration at takeover. The results show that the feeling of being able to drive is not solely related to motor or driving tasks, but also on temporal and cognitive factors, and that the distribution of categories differs between the criticality of the investigated events. These results suggest that driver availability is a subjectively and situationally influenced concept determined by a variety of parameters. Identifying the relevant factors contributes to the development of tailored support during the transition.

Until automated vehicles are publicly available on roads, researchers explore drivers’ responses to different situations and stimuli in controlled experiments where participants often experience the technology only for a short amount of time. This limits the ability to explore drivers’ behavioral changes over time and may limit the validity of the gained results in situation beyond first-time encounters with the automated system. To address this, Jonas Andersson, Azra Habibovic, and Daban Rizgary present in their paper “First Encounter Effects in Testing of Highly Automated Vehicles – The Need for Recurrent Testing” the results of a field study in which each driver experienced the test conditions in two different occasions with an interval of one week between sessions. They collected qualitative data through interviews and questionnaires and quantitative data such as gaze behavior. Their results revealed that habituation effects were attenuated over time. While in the first encounter, the drivers were exhilarated and had variations in their perceived safety, they became more neutral and had a steadier perception of safety during the second encounter. These results indicate that exposing drivers to HAVs on multiple occasions may provide more informative insights into driver behavior and experience as compared to only one occasion.

Trust in automation is still one of the major barriers for future automated cars. With this regard, Philipp Wintersberger, Frederica Janotta, Jakob Peintner, Andreas Löcken, and Andreas Riener investigated in their paper “Evaluating Feedback Requirements for Trust Calibration in Automated Vehicles” the feedback demands of users of automated vehicles and how the automotive user interface can provide such information. In a first study, they investigated subjective trust issues of (front seat) automated vehicle passengers and which feedback modalities and content they prefer. The second study deepened the aspects of (user interface) feedback demands. Their findings and recommendations show that interfaces of automated vehicles should inform their users about upcoming actions and how to do so in order to raise trust and improve acceptance of vehicle automation.

Lastly, another factor that will be critical in ensuring the acceptance of automated vehicles is the aspect of interactions between the vehicle and pedestrians and other road users. To address the communication gap that arises from the inability to communicate with a driver inside, external Human-Machine Interfaces (eHMI) have been proposed as a means to facilitate smooth interactions between automated vehicles and pedestrians. However, the extent of the effectiveness of eHMIs are not fully known. In their article “Communicating the Intention of an Automated Vehicle to Pedestrians: The Contributions of eHMI and Vehicle Behavior” Debargha Dey, Andrii Matviienko, Melanie Berger, Bastian Pfleging, Marieke Martens, and Jacques Terken explore how universal communication from eHMIs are, and how the driving behavior of the vehicle plays a role in pedestrians’ willingness to cross in front of an approaching automated vehicle. Their study with 24 participants reveals that pedestrians do not blindly trust the message from an eHMI. Furthermore, when an eHMI contradicts the vehicle’s apparent intention as observed from its behavior, pedestrians fall back to interpreting the vehicle’s behavior. They conclude that maximum effectiveness of an eHMI can be achieved when its message is in alignment with the vehicle’s driving behavior.

The papers in this issue provide another stepping stone in adding to the corpus of knowledge and pushing forward the boundaries of automotive user interface research towards the future of driving automation. As the field develops and matures, we expect that more nuances of these and other research questions will continue to surface and pave the way for a more seamless and successful integration of automated vehicles into traffic and daily life. We thank the authors for their research contributions. Additionally, we would like to express our special thanks to all of our reviewers who supported the selection process and who made very helpful suggestions to the authors on how to improve their articles.