Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Current Directions in Biomedical Engineering

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Buzug, Thorsten M. / Haueisen, Jens / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Kraft, Marc / Lenarz, Thomas / Leonhardt, Steffen / Malberg, Hagen / Penzel, Thomas / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Urban, Gerald A.


CiteScore 2018: 0.47

Source Normalized Impact per Paper (SNIP) 2018: 0.377

Open Access
Online
ISSN
2364-5504
See all formats and pricing
More options …

Application of head-mounted devices with eye-tracking in virtual reality therapy

Otto Hans-Martin Lutz
  • Corresponding author
  • Chair of Industrial Automation Technology, Technische Universität Berlin, PTZ 5, Pascalstr. 8-9, 10587 Berlin, Germany
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Charlotte Burmeister / Luara Ferreira dos Santos / Nadine Morkisch / Christian Dohle / Jörg Krüger
Published Online: 2017-03-08 | DOI: https://doi.org/10.1515/cdbme-2017-0012

Abstract:

Using eye-tracking to assess visual attention in head-mounted devices (HMD) opens up many possibilities for virtual reality (VR)-based therapy. Existing therapy concepts where attention plays a major role can be transferred to VR. Furthermore, they can be expanded to a precise real-time attention assessment, which can serve as a foundation for new therapy approaches. Utilizing HMDs and eye-tracking in a clinical environment is challenging because of hygiene issues and requirements of patients with heterogeneous cognitive and motor impairments. In this paper, we provide an overview of those challenges, discuss possible solutions and present preliminary results of a study with patients.

Keywords: eye-tracking; head-mounted devices; hygiene; virtual reality therapy

1 Introduction

Virtual Reality (VR) is a promising field for new forms of therapy. It is a flexible tool for therapists to provide an immersive and interactive environment for patients. VR has been used in the treatment of phobias [1], therapy of addictions [2] and motor rehabilitation [3]. Implementing eye-tracking into head-mounted devices (HMD) introduces a precise real-time assessment of attention and opens up new possibilities for VR-based therapy, e.g. analyzing cognitive processes [4] or diagnosing medical conditions [5]. Additionally to common VR-related issues, specific conditions persist when using HMDs in a clinical context. Strict hygiene requirements have to be met to protect patients, particularly considering the prevalence of multi-resistant organisms.

To achieve sufficient positional accuracy of the eye-tracking, a re-calibration is required every time the HMD has been moved [6]. Hence, eye-tracking creates additional workload for patients and therapists through the calibration procedure. Adequate procedures have to be found to minimize this additional workload in order to be applicable for patients with cognitive deficits.

Setting up eye-tracking systems in head-mounted devices in a way that it (i) meets the hygiene requirements for use in clinical applications, (ii) is suitable for patients with heterogeneous motor and cognitive limitations and (iii) still yields valid and reliable attention data throughout a therapy session is a complex challenge we like to address in this paper.

2 Potential of eye-tracking in VR therapy

Adding eye-tracking to VR therapy enhances the possibility to accurately measure attention in real-time. Especially in therapy methods where the focus of attention plays a mayor role, e.g. mirror therapy for stroke patients [7] or treatment of social anxiety [8], therapists strongly benefit from an objective measurement of patients’ attention. Stimuli can be rendered depending on the focus of attention. Renaud et al. developed an environment which changes when the patients gaze behavior is indicating visual avoidance [9], as this would decrease therapy effectiveness. The visual field can be masked partially according to the gaze position for rehabilitation of hemispatial neglect [10]. Furthermore, continuous assessment of attention opens up new potential for realizing adaptive shaping of therapy difficulty, not only based on performance measures but also depending on patients’ cognitive abilities at any given moment [11].

Another advantage of eye-tracking in VR is hands-free interaction. Users found gaze based interaction to be easier and more intuitive than pointing in VR [12]. This overcomes the challenges of common human-computer interaction (e.g. pressing a key) while wearing a HMD.

VR cover and eye-tracking integration.
Figure 1

VR cover and eye-tracking integration.

3 Head mounted devices in therapy applications

3.1 Hygiene aspects

When using HMDs in a therapeutic setting with different patients, it has to be considered that the device can serve as fomite for diseases [13]. Especially the foam, which is pressed against the user’s face, gets dirty quickly, might transmit diseases and acquire uncomfortable smells. Therefore employing a defined hygiene routine ensures both a sanitary environment and patients’ comfort. There is little literature on the specific topic of HMD hygiene in therapy context, most of it are guidelines by manufacturers of specific solutions [14].

Most HMDs have to be modified to comply with hygiene regulations. All textile, foam and Velcro components have to be enclosed, removed or replaced by components with an even, cleanable surface. Inside the HMD, caution has to be exercised not to accidentally short-circuit electric components with the disinfectant. Coating all circuitry with a protective finish is a suitable solution, but the coating must not be solvable by the disinfectant. This applies in particular when adding third-party electrical components, e.g. an eye-tracker (see Figure 1B). Lenses can be protected against the disinfectant with a self-adhesive foil. Using covers either for the HMD or on the patient’s face prevents disease transmission by the foam pads. Reusable covers (see Figure 1A) and wipeable replacement foam pads are commercially available [14]. A solution for sanitizable head straps is not yet available, but these can be manually replaced by adjustable rubber bands found in diving goggles (see Figure 2).

All hygiene requirements and modifications should be discussed and agreed upon with the person in charge (e.g. hygiene officer) of the institution conducting the VR therapy. Patients need to be informed about potential risks and safety guidelines.

Custom rubber strap solution.
Figure 2

Custom rubber strap solution.

3.2 Cyber sickness

Cyber and simulator sickness are known issues since the beginning of VR applications [15]. Amongst others, the sensory mismatch theory is the most established explanation of cyber sickness [16]. The discrepancy in movement perception between proprioceptive and visual modalities can lead to effects similar to motion sickness. Therefore, precautions against cyber sickness have to be taken both technologically (low-latency head tracking, rendering and displaying of the VR scene) and in the design of VR environments. Movements in the VR need to be uniform with actual body movements. A seated setting is less prone to cause cyber sickness than a standing or moving setting. Locomotion by teleporting can be effective for reducing cyber sickness, but may cause disorientation [17].

We conducted a study with 12 patients after first-ever stroke, nine male and three female (mean age M = 67.26, SD = 7.96). Patients with visuospatial deficits, hemispatial neglect, aphasia or with constraints or pain in their non-affected arm or hand were excluded from the study. Subjects performed motor tasks in VR on two days. In each session they performed three tasks, simple pointing movements toward virtual cubes with their non-affected hand. One VR-task had a duration of approximately two to three minutes. Tasks were performed seated, there was no locomotion through the VR scene. Patients were asked to take off the HMD between tasks. Participants rated their discomfort on simulator sickness questionnaire (SSQ) [18] immediately before the first and after the last VR-task. A paired-samples t-test of the mean SSQ score was conducted to compare the pre and post VR session results. There were no significant differences between the pre and post assessment in the 1st (pre: M = 1.22, SD = 0.25; post: M = 1.18, SD = 0.16; t(11) = 0.62, p = 0.543, effect size d = 0.18) or in the 2nd session (pre: M = 1.11, SD = 1.16; post: M = 1.11, SD = 0.11; t(10) = 0.00, p = 1.00, d = 0.00).

4 Eye-tracking in HMDs

4.1 Technology

Current eye-tracking in HMDs is video-based, i.e. a camera films the eyes either directly or via a hot mirror, which reflects infrared light, but transmits the visible spectrum. Several solutions for equipping existing HMDs with eye cameras and components for illuminating the eyes by infrared LEDs (see Figure 1B) are commercially available. Gaze detection is performed by matching the black pupil ellipse in each video frame and calculating its center. Positions of the LEDs’ corneal reflections can be utilized additionally to achieve more precise data [6]. Mapping of gaze to screen coordinates is performed either in 2D, via fitting a polynomial equation or in 3D, by fitting eye models and gaze vectors in the 3D space of the VR scene [19].

4.2 Requirements for clinical application

For clinical applications of eye-tracking in HMDs, several additional aspects have to be considered in comparison to studies in controlled laboratory environments. Hygiene requirements were already mentioned, but a number of issues regarding human-technology interaction have to be examined as well. The system needs to be designed comprehensible for both therapists and patients. The overall functionality of the system, basics of eye-tracking and potential issues (e.g. concerning makeup, eyelashes, fit of the HMD and corresponding camera perspective) should be communicated in a suitable way.

Adequate feedback needs to be given appropriately, understandable and at the right time. If the eyes are not tracked, a warning should be displayed indicating that the system does not work as intended. Information on short losses of eye detection may be hidden from the patient, but still needs to be presented to the therapist interpreting the recorded data.

In clinical context, both therapists and patients can not be considered as proficient in the handling of eye-tracking technology. Therefore, the system has to be designed robust against human error.

In several studies at our department, users tended to unconsciously readjust the fit of the HMD through a VR session or to move it by touching their faces (e.g. scratching the nose). This happend even if users were asked to avoid touching the HMD. Furthermore, the HMD might slowly move downwards over time depending on the fit on the face. However, even small displacements of the HMD change the position of the camera relative to the eyes and thus the computed gaze position. A movement detection and slip compensation should be implemented to achieve robustness against these possible sources of error [20].

4.2.1 Calibration

As every user has unique physiological properties, individual calibration is indispensable. At the same time, user acceptance of calibration procedures is low, particularly in cases where a re-calibration is needed [21]. Re-calibration is necessary each time the HMD has been moved, which can happen several times during a therapy session. On the contrary, workload for patients induced by eye-tracking should be minimized. Hence, the system should employ a fast set-up process and a quick comprehensible calibration procedure.

Automatic acceptance of calibration points by a combination of fixation detection and spatial proximity (i.e. the gaze has been stable for a relatively long time close to the target position) is not applicable in HMDs as the spatial proximity is heavily depending on the fit of the HMD. One approach to efficient calibration in therapy context is combining an assisted regular calibration done once per user with an automatic one-point calibration performed each time the HMD is put on. A solution which does not reduce calibration workload per se, but might present a less tedious alternative, is pursuit calibration [21]. The user’s gaze follows a moving stimulus, i.e. smooth pursuit of eye movements. This approach can be integrated in a playful way, which may add a motivational benefit counterbalancing the additional workload.

5 Conclusion

Integrating eye-tracking in HMDs is a promising approach for new VR therapy concepts. It offers the possibility of a precise real-time assessment of attention. But particularly for clinical applications, several aspects have to be considered. We presented obstacles and practical solutions in regard to hygiene, cyber sickness, and human-machine interaction and applied these solutions in a pilot study with patients. Additional research is required to achieve both high eye-tracking accuracy and good usability for patients. Systems need to be designed robust against human error and easily comprehensible by both therapists and patients.

Author’s Statement

Research funding: The research was funded by the German Federal Ministry of Education and Research as part of the regional innovation cluster “BeMobil: Regain mobility and motivity”. Conflict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use complies with all the relevant national regulations, institutional policies and was performed in accordance with the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.

References

  • Bouchard S, Côté S, St-Jacques J, Robillard G, Renaud P. Effectiveness of virtual reality exposure in the treatment of arachnophobia using 3D games. Technol Health Care 2006;14:19–27. Google Scholar

  • Gamito P, Oliveira J, Baptista A, Morais D, Lopes P, Rosa P, et al. Eliciting nicotine craving with virtual smoking cues. Cyberpsychol Behav Soc Netw. 2014;17:556–61. Google Scholar

  • Archambault P, Norouzi NG, Kairy D, Solomon JM, Levin MF. Towards establishing clinical guidelines for a virtual reality arm rehabilitation system. Replace, repair, restore, relieve - Bridging clinical and engineering solutions in neurorehabilitation, Vol. 7, Biosystems and Robotics; 2014. p. 263–71. Google Scholar

  • Vidal M, Turner J, Bulling A, Gellersen H. Wearable eye tracking for mental health monitoring. Comp Commun. 2012;35:1306–11. Google Scholar

  • Tsirlin I, Dupierrix E, Chokron S, Coquillart S, Ohlmann T. Uses of virtual reality for diagnosis, rehabilitation and study of unilateral spatial neglect: review and analysis. Cyberpsychol Behav. 2009;12:175–81. Google Scholar

  • Holmqvist K, Nyström M, Andersson R, Dewhurst R, Halszka J, van de Weijer J. Eye tracking : a comprehensive guide to methods and measures. New York: Oxford University Press; 2011. Google Scholar

  • Ramachandran VS, Rogers-Ramachandran D, Cobb S. Touching the phantom limb. Nature 1995;377:489–90. Google Scholar

  • Grillon H, Riquier F, Herbelin B, Thalmann D. Use of virtual reality as therapeutic tool for behavioural exposure in the ambit of social anxiety disorder treatment. In: Proc 6th Intl Conf Disability, Virtual Reality & Assoc Tech; 2006. p. 105–12. Google Scholar

  • Renaud P, Joyal C, Stoleru S, Goyette M, Weiskopf N, Birbaumer N. Real-time functional magnetic imaging-brain-computer interface and virtual reality. promising tools for the treatment of pedophilia. Prog Brain Res. 2011;192:263–72. Google Scholar

  • Baheux K, Yoshizawa M, Tanaka A, Seki K, Handa Y. Diagnosis and rehabilitation of hemispatial neglect patients with virtual reality technology. Technol Health Care 2005;13:245–60. Google Scholar

  • Squeri V, Basteris A, Sanguineti V. Adaptive regulation of assistance ’as needed’ in robot-assisted motor skill learning and neuro-rehabilitation. In: Proc. IEEE International Conference on Rehabilitation Robotics; 2011. Google Scholar

  • Tanriverdi V, Jacob RJK. Interacting with eye movements in virtual environments. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’00; 2000;2:265–72. Google Scholar

  • Costello P. Health and safety issues associated with virtual reality - a review of current literature. Advisory Group on Computer Graphics (AGOGG) Techical Reports; 1997. p. 1–23. Google Scholar

  • Gauss Labs Ltd. Virtual reality headset hygiene best practices. [Online]. Available at: https://vrcover.com/virtual-reality-headset-hygiene-best-practices. Google Scholar

  • Kolasinski EM. Simulator Sickness in Virtual Environments. Alexandria, VA: Tech. Rep.; 1995. Google Scholar

  • Rebenitsch L, Owen C. Review on cybersickness in applications and visual displays. Virtual Reality. 2016;20:101–25. Google Scholar

  • Bowman DA, Koller D, Hodges LF. Travel in immersive virtual environments: an evaluation motion control techniques of viewpoint. Proc. IEEE Virtual Reality Annual Symposium; 1997. p. 45–52. Google Scholar

  • Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator sickness questionnaire : an enhanced method for quantifying simulator sickness. Int J Aviation Psychol. 1993. p. 203–20. Google Scholar

  • Świrski L, Dodgson N. A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting. In Proc. PETMEI; 2013. Google Scholar

  • Rolland J, Hua H. Displays: head-mounted. In: Encyclopedia of Optical Engineering; 2005. Google Scholar

  • Pfeuffer K, Vidal M, Turner J. Pursuit calibration: making gaze calibration less tedious and more flexible. Proceedings of the 26th annual ACM symposium on User interface software and technology; 2013. p. 261–69. Google Scholar

About the article

Published Online: 2017-03-08

Published in Print: 2017-03-01


Citation Information: Current Directions in Biomedical Engineering, Volume 3, Issue 1, Pages 53–56, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2017-0012.

Export Citation

©2017 Otto Hans-Martin Lutz et al., licensee De Gruyter.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in