Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter Oldenbourg April 10, 2021

First encounter effects in testing of highly automated vehicles during two experimental occasions – The need for recurrent testing

Jonas Andersson ORCID logo, Azra Habibovic and Daban Rizgary

Abstract

To explore driver behavior in highly automated vehicles (HAVs), independent researchers are mainly conducting short experiments. This limits the ability to explore drivers’ behavioral changes over time, which is crucial when research has the intention to reveal human behavior beyond the first-time use. The current paper shows the methodological importance of repeated testing in experience and behavior related studies of HAVs. The study combined quantitative and qualitative data to capture effects of repeated interaction between drivers and HAVs. Each driver (n=8) participated in the experiment on two different occasions (∼90 minutes) with one-week interval. On both occasions, the drivers traveled approximately 40 km on a rural road at AstaZero proving grounds in Sweden and encountered various traffic situations. The participants could use automated driving (SAE level 4) or choose to drive manually. Examples of data collected include gaze behavior, perceived safety, as well as interviews and questionnaires capturing general impressions, trust and acceptance. The analysis shows that habituation effects were attenuated over time. The drivers went from being exhilarated on the first occasion, to a more neutral behavior on the second occasion. Furthermore, there were smaller variations in drivers’ self-assessed perceived safety on the second occasion, and drivers were faster to engage in non-driving related activities and become relaxed (e. g., they spent more time glancing off road and could focus more on non-driving related activities such as reading). These findings suggest that exposing drivers to HAVs on two (or more) successive occasions may provide more informative and realistic insights into driver behavior and experience as compared to only one occasion. Repeating an experiment on several occasions is of course a balance between the cost and added value, and future research should investigate in more detail which studies need to be repeated on several occasions and to what extent.

ACM CCS:

Funding source: VINNOVA

Award Identifier / Grant number: 2017-03058

Funding statement: The project was funded by SAFER Open Research at AstaZero (Grant No. A-0018), the VINNOVA Strategic Vehicle Research and Innovation program (project Trust in Intelligent Cars - Grant No. 2017-03058), and the Knowledge Foundation (project Action Intention Recognition - Grant No. 20140220)

Acknowledgment

We want to thank our colleagues Maria Klingegård, Alexey Voronov, David Lindström, Victor Malmsten-Lundgren and Emma Asker for their work at the AstaZero proving grounds, and with data analysis.

Appendix A

Table A.1

Scenarios at the test track demonstrating HAV capabilities.

ScenarioDescription
RoadworksA static roadworks is marked by cones and a roadwork sign. It is visible from distance. The roadworks demonstrates that the HAV can handle zigzag driving
Upcoming & OvertakingThe HAV catches up a slow-moving vehicle (ca 40 km/h) in the same lane. The road section is straight, and visibility is good. The HAV decides to either follow the slow-moving vehicle (too short distance to intersection) or perform an overtake. This demonstrates that the HAV can handle overtaking
IntersectionA vehicle is about to enter the priority road (i. e. it should yield). Before entering it stops. To demonstrate that it has understood the situation and possible risk, the HAV slightly adapts its speed
Oncoming trafficThe HAV encounters a vehicle travelling in the adjacent lane in the opposite direction

Table A.2

The level of safety categorized into safe (i. e., TPs used the Swedish word “trygg” in their expression), and quite safe (i. e., TPs used expression such as “ganska trygg”). No stronger expression of uncertainty or unsafety were identified.

Test participantOverall safety – Occasion 1Progress over time during Occasion 1Overall safety – Occasion 2Progress over time during Occasion 2
TP2SafeIncreasen/aNo change
TP3Quite safen/aSafeIncrease
TP4Safen/aSafeNo change
TP5Quite safeNo changeSafen/a
TP6Quite safen/aQuite safen/a
TP7SafeIncreaseSafeDecrease, increased
TP8SafeNo changeSafe, quite safeNo change
TP9SafeNo changeSafeNo Change
Figure A.1 TPs’ answer to the post drive question “Would you feel safe to test the AD in real traffic?”

Figure A.1

TPs’ answer to the post drive question “Would you feel safe to test the AD in real traffic?”

References

1. SAE International, “Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (J3016_201806),” 2018.Search in Google Scholar

2. V. Melcher, S. Rauh, F. Diederichs, H. Widlroither, and W. Bauer, “Take-over requests for automated driving,” Procedia Manufacturing, vol. 3, pp. 2867–2873, 2015, doi: 10.1016/j.promfg.2015.07.788.Search in Google Scholar

3. A. Habibovic, J. Andersson, J. Nilsson, M. Nilsson, and C. Edgren, “Command-Based Driving for Tactical Control of Highly Automated Vehicles,” in Advances in Human Aspects of Transportation: Proceedings of the AHFE 2016 International Conference on Human Factors in Transportation, July 27–31, 2016, Walt Disney World®, Florida, USA, N. A. Stanton, S. Landry, G. Di Bucchianico, and A. Vallicelli, Eds. Cham: Springer International Publishing, 2017, pp. 499–510.10.1007/978-3-319-41682-3_42Search in Google Scholar

4. D. R. Large, G. Burnett, D. Salanitri, A. Lawson, and E. Box, “A Longitudinal Simulator Study to Explore Drivers’ Behaviour in Level 3 Automated Vehicles,” in Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2019, pp. 222–232, doi: 10.1145/3342197.3344519.Search in Google Scholar

5. A. H. Jamson, N. Merat, O. M. J. Carsten, and F. C. H. Lai, “Behavioural changes in drivers experiencing highly-automated vehicle control in varying traffic conditions,” Transportation Research Part C: Emerging Technologies, vol. 30, pp. 116–125, May 2013, doi: 10.1016/j.trc.2013.02.008.Search in Google Scholar

6. N. Merat, A. H. Jamson, F. C. H. Lai, M. Daly, and O. M. J. Carsten, “Transition to manual: Driver behaviour when resuming control from a highly automated vehicle,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 27, no. PB, pp. 274–282, 2014, doi: 10.1016/j.trf.2014.09.005.Search in Google Scholar

7. C. Gold, M. Körber, C. Hohenberger, D. Lechner, and K. Bengler, “Trust in automation – before and after the experience of take-over scenarios in a highly automated vehicle,” Procedia Manufacturing, vol. 3, no. Ahfe, pp. 3025–3032, 2015, doi: 10.1016/j.promfg.2015.07.847.Search in Google Scholar

8. B. K.-J. Mok, D. Sirkin, S. Sibi, D. B. Miller, and W. Ju, “Understanding driver-automated vehicle interactions through Wizard of Oz design improvisation,” no. October, pp. 380–386, 2015, doi: 10.17077/drivingassessment.1598.10.17077/drivingassessment.1598Search in Google Scholar

9. A. Habibovic et al., “Communicating intent of automated vehicles to pedestrians,” Front. Psychol., 2018, doi: 10.3389/fpsyg.2018.01336.Search in Google Scholar PubMed PubMed Central

10. S. Kim et al., “Autonomous taxi service design and user experience,” International Journal of Human–Computer Interaction, vol. 36, no. 5, pp. 429–448, 2020, doi: 10.1080/10447318.2019.1653556.Search in Google Scholar

11. K. Osz, A. Rydström, V. Fors, S. Pink, and R. Broström, “Building collaborative test practices: Design ethnography and WOz in autonomous driving research,” Interaction Design and Architecture(s), vol. 37, pp. 12–20, 2018.10.55612/s-5002-037-001Search in Google Scholar

12. O. Jarosch, S. Paradies, D. Feiner, and K. Bengler, “Effects of non-driving related tasks in prolonged conditional automated driving – A Wizard of Oz on-road approach in real traffic environment,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 65, pp. 292–305, 2019, doi: 10.1016/j.trf.2019.07.023.Search in Google Scholar

13. P. Lai, “The literature review of technology adoption models and theories for the novelty technology,” Journal of Information Systems and Technology Management, vol. 14, no. 1, pp. 21–38, 2017, doi: 10.4301/s1807-17752017000100002.Search in Google Scholar

14. E. Karapanos, J. Zimmerman, J. Forlizzi, and J.-B. Martens, “User Experience Over Time: An Initial Framework,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009, pp. 729–738, doi: 10.1145/1518701.1518814.Search in Google Scholar

15. V. A. Banks and N. A. Stanton, “Keep the driver in control: Automating automobiles of the future,” Applied Ergonomics, vol. 53, pp. 389–395, 2016, doi: 10.1016/j.apergo.2015.06.020.Search in Google Scholar PubMed

16. R. Broström, Annie Rydström, Christoffer Kopp and “Drivers Quickly Trust Autonomous Cars,” in T. Ahram and C. Falcão, Eds. Advances in Usability, User Experience and Assistive Technology, AHFE 2018. Advances in Intelligent Systems and Computing, vol. 794. Cham: Springer, Cham, 2019, pp. 699–705.10.1007/978-3-319-94947-5_69Search in Google Scholar

17. J. Hedman and G. Gimpel, “The adoption of hyped technologies: a qualitative study,” Information Technology and Management, vol. 11, no. 4, pp. 161–175, Dec. 2010, doi: 10.1007/s10799-010-0075-0.Search in Google Scholar

18. A. G. Mirnig, M. Gärtner, A. Meschtscherjakov, and M. Tscheligi, “Blinded by Novelty: A Reflection on Participant Curiosity and Novelty in Automated Vehicle Studies Based on Experiences from the Field,” in Proceedings of the Conference on Mensch Und Computer, 2020, pp. 373–381, doi: 10.1145/3404983.3405593.Search in Google Scholar

19. N. Du et al., “Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving,” Transportation Research Part C: Emerging Technologies, vol. 112, pp. 78–87, 2020, doi: 10.1016/j.trc.2020.01.006.Search in Google Scholar

20. E. Nilsson et al., in “Vehicle Driver Monitoring – Sleepiness and Cognitive load,” Linköping, Sweden, 2017.Search in Google Scholar

21. J. C. F. de Winter, R. Happee, M. H. Martens, and N. A. Stanton, “Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 27, pp. 196–217, 2014, doi: 10.1016/j.trf.2014.06.016.Search in Google Scholar

22. F. Naujoks, C. Purucker, and A. Neukum, “Secondary task engagement and vehicle automation – Comparing the effects of different automation levels in an on-road experiment,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 38, pp. 67–82, 2016, doi: 10.1016/j.trf.2016.01.011.Search in Google Scholar

23. F. Walker, W. Verwey, and M. Martens, “Gaze Behaviour as a Measure of Trust in Automated Vehicles,” 2018.10.1155/2018/1045186Search in Google Scholar

24. L. Montoro, S. A. Useche, F. Alonso, I. Lijarcio, P. Bosó-Seguí, and A. Martí-Belda, “Perceived safety and attributed value as predictors of the intention to use autonomous vehicles: A national study with Spanish drivers,” Safety Science, vol. 120, pp. 865–876, 2019, doi: 10.1016/j.ssci.2019.07.041.Search in Google Scholar

25. T. Zhang, D. Tao, X. Qu, X. Zhang, R. Lin, and W. Zhang, “The roles of initial trust and perceived risk in public’s acceptance of automated vehicles,” Transportation Research Part C: Emerging Technologies, vol. 98, pp. 207–220, 2019, doi: 10.1016/j.trc.2018.11.018.Search in Google Scholar

26. J. K. Choi and Y. G. Ji, “Investigating the importance of trust on adopting an autonomous vehicle,” International Journal of Human–Computer Interaction, vol. 31, no. 10, pp. 692–702, 2015, doi: 10.1080/10447318.2015.1070549.Search in Google Scholar

27. B. D. Seppelt and T. W. Victor, “Potential Solutions to Human Factors Challenges in Road Vehicle Automation,” in Road Vehicle Automation 3, G. Meyer and S. Beiker, Eds. Cham: Springer International Publishing, 2016, pp. 131–148.10.1007/978-3-319-40503-2_11Search in Google Scholar

28. A. D. McDonald et al., “Toward computational simulations of behavior during automated driving takeovers: A review of the empirical and modeling literatures,” Human Factors, vol. 61, no. 4, pp. 642–688, 2019, doi: 10.1177/0018720819829572.Search in Google Scholar PubMed

29. S. Brandenburg and F. Roche, “Behavioral changes to repeated takeovers in automated driving: The drivers’ ability to transfer knowledge and the effects of takeover request process,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 73, pp. 15–28, 2020, doi: 10.1016/j.trf.2020.06.002.Search in Google Scholar

30. S. Hergeth, L. Lorenz, R. Vilimek, and J. F. Krems, “Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving,” Human Factors, vol. 58, no. 3, pp. 509–519, 2016, doi: 10.1177/0018720815625744.Search in Google Scholar PubMed

31. J. Navarro, F. Osiurak, M. Ovigue, L. Charrier, and E. Reynaud, “Highly automated driving impact on drivers’ gaze behaviors during a car-following task,” International Journal of Human–Computer Interaction, vol. 35, no. 11, pp. 1008–1017, 2019, doi: 10.1080/10447318.2018.1561788.Search in Google Scholar

32. K. Lee Koay, D. S. Syrdal, M. L. Walters, and K. Dautenhahn, “Living with Robots: Investigating the Habituation Effect in Participants’ Preferences During a Longitudinal Human-Robot Interaction Study,” in RO-MAN 2007 – The 16th IEEE International Symposium on Robot and Human Interactive Communication, Aug. 2007, pp. 564–569, doi: 10.1109/ROMAN.2007.4415149.Search in Google Scholar

33. C. Teddlie and A. Tashakkori, Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. SAGE Publications, 2009.10.4135/9781483348858.n9Search in Google Scholar

34. D. L. Morgan, “Motivations for Using Mixed Methods Research,” in Integrating Qualitative and Quantitative Methods: A Pragmatic Approach, 2014, pp. 63–84.10.4135/9781544304533.n4Search in Google Scholar

35. D. D. Heikoop, J. C. de Winter, B. van Arem, and N. A. Stanton, “Acclimatizing to automation: Driver workload and stress during partially automated car following in real traffic,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 65, pp. 503–517, 2019.10.1016/j.trf.2019.07.024Search in Google Scholar

Received: 2020-07-10
Revised: 2021-03-21
Accepted: 2021-03-29
Published Online: 2021-04-10
Published in Print: 2021-06-25

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Scroll Up Arrow