Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Current Directions in Biomedical Engineering

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Buzug, Thorsten M. / Haueisen, Jens / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Kraft, Marc / Lenarz, Thomas / Leonhardt, Steffen / Malberg, Hagen / Penzel, Thomas / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Urban, Gerald A.

1 Issue per year

Open Access
Online
ISSN
2364-5504
See all formats and pricing
More options …

Pseudo-haptic feedback in medical teleoperation

Carsten Neupert
  • Corresponding author
  • Institute of Electromechanical Design, Technische Universität Darmstadt, Merckstraße 25, 64283 Darmstadt
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Sebastian Matich
  • Institute of Electromechanical Design, Technische Universität Darmstadt, Merckstraße 25, 64283 Darmstadt
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Peter P. Pott
  • Institute of Electromechanical Design, Technische Universität Darmstadt, Merckstraße 25, 64283 Darmstadt
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Christian Hatzfeld
  • Institute of Electromechanical Design, Technische Universität Darmstadt, Merckstraße 25, 64283 Darmstadt
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Roland Werthschützky
  • Institute of Electromechanical Design, Technische Universität Darmstadt, Merckstraße 25, 64283 Darmstadt
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2015-09-12 | DOI: https://doi.org/10.1515/cdbme-2015-0040

Abstract

Pseudo-haptic feedback is a haptic illusion based on a mismatch of haptic and visual perception. It is well known from applications in virtual environments. In this work, we discuss the usabiliy of the principle of pseudo-haptic feedback for teleoperation. Using pseudo-haptic feedback can ease the design of haptic medical tele-operation systems.

Thereby a user’s grasping force at an isometric user interface is used to control the closing angle of an end effector of a surgical robot. To provide a realistic haptic feedback, the coupling characteristic of grasping force and end effector closing angle is changed depending on acting end effector interaction forces.

With an experiment, we show the usability of pseudo-haptic feedback for discriminating compliances, comparable to the mechanical characteristic of muscles relaxed and contracted. The provided results base upon the data of 10 subjects, and 300 trails.

Keywords: Haptics; Haptic feedback; Teleoperation; Telerobotics

1 Motivation

Haptic feedback can improve the performance of telerobotic surgery in terms of decreasing the time of a surgery and the trauma [1, 2]. Also, providing haptic feedback for teleoperation allows a surgeon to use his haptic sensation, instead of using vision only, to estimate the interaction forces of an end effector in situ. This is supposed to reduce cognitive load of the surgeon.

State of the art teleoperation systems including haptic feedback are commonly based on an impedance-admittance system architecture [3]. This implies a masterslave structure. A user controls the slave robot by interacting with the master unit. The user sets a position or velocity at the master. The slave follows the user’s input. The force between the end effector and tissue, resulting from the performed manipulation in situ, is measured. The measured forces are used to reflect haptic feedback to a user at the master unit.

In this case, it is required that the dexterity of the master unit fits the dexterity of the slave robot. At least one sensor at the slave and one actuator at the master unit is required to provide haptic feedback for each degree of freedom of the teleoperation system. The integration of force sensors to the end effector and implementing up to 7 actuators to the master input device, that are needed to provide full dexterity, enhances the complicity of a system. Also, the requirements concerning the dynamic behavior of the actuators rise with rising dexterity and workspace of the telerobotic system.

With our work, we introduce the principle of pseudo-haptic feedback as an option of reflecting haptic feedback of grasping forces of an end effector without the necessity of a moving or active part in the master unit.

2 Application

Pseudo haptic feedback is well known from applications in virtual environments [46]. Different kinds of interaction and haptic behavior of virtual object can be simulated and fed back to the user by for example changing the resulting moving characteristics of a mouse curser dependent on special tasks or interaction. The haptic sensation of pseudo haptics bases on a haptic illusion resulting from the mismatch of haptic- and visual perception [46].

For the pseudo-haptic teleoperation, we assume a stiff interface as input device (Figure 1). The input device is sensitive to the grasping force of the user (Fg). The closing angle of the end effector (φ) is controlled by the user’s grasping force. The acting interaction force at the end effector (Fe) is measured.

The relation between the grasping force and the end effector is designed as a linear spring for Fe = 0 to present the characteristics of an surgical forceps (Figure 2). Based on interaction forces acting at the end effector, the coupling characteristic of grasping force and end effector closing angle is altered. For example the interaction with a linear compliance (Fe = 1/n · φ) results in a drop of the slope of the characteristic curve. Dependent on mechanical properties, the interaction with any kind of tissue leads to unique characteristic curves. The change of the coupling characteristic, depending on the case of interaction, causes a realistic haptic feedback.

Parts of a teleoperation system concerning the grasping degree of freedom. Principle of an isometric user interface in the upper part of the figure and the principle of the end effector in the lower part of the figure.
Figure 1

Parts of a teleoperation system concerning the grasping degree of freedom. Principle of an isometric user interface in the upper part of the figure and the principle of the end effector in the lower part of the figure.

Characteristic curve of the relation of the grasping force and the end effector closing angle. Basis couple characteristic for idle case (Fe = 0 N) (solid), Interaction with a linear compliance (Fe = 1/n · φ) and interaction with any environment (Fe(φ, environment)) (dashed).
Figure 2

Characteristic curve of the relation of the grasping force and the end effector closing angle. Basis couple characteristic for idle case (Fe = 0 N) (solid), Interaction with a linear compliance (Fe = 1/n · φ) and interaction with any environment (Fe(φ, environment)) (dashed).

The haptic feedback perceived by the user results from the perceived difference of grasping forces at the user interface needed to reach the same end effector closing angle. The intensity and the rate of realism of the perceived sensation can be altered by using different coupling characteristics [8].

3 Experiments

A compliance identification experiment is performed to verify the principle of pseudo-haptic teleoperation. An adjustable compliance is used as specimen to simulate different properties of tissue (Figure 3, b, c). For the experiment the specimen’s compliance is set to n1 = 8 mm/N, n2 = 4 mm/N and n3 = 2 mm/N. The values are chosen with respect to the properties of muscles, relaxed (n ≈ 8 mm/N) and contracted (n ≈ 2mm/N) [7].

A load cell is used as isometric user interface (Figure 3, a). The end effector displacement is controlled by the user’s force excitation at the load cell. The end effector bases on a linear moving axis with a force sensor at its tip. The signals of the force sensor at the end effector is used to change the couple characteristic between the user’s grasping force and the end effector displacement.

Experimental setup. Isometric user-interface based on a load cell (a). End effector in interaction with an adjustable compliance, simulating the stiffness of relaxed and contracted muscles (b, c).
Figure 3

Experimental setup. Isometric user-interface based on a load cell (a). End effector in interaction with an adjustable compliance, simulating the stiffness of relaxed and contracted muscles (b, c).

While performing the experiment, the subjects were asked to identify the three compliances (n1 . . . n3) only by analyzing their excited grasping forces to the load cell and the visual information of the resulting end effector movement at the specimen.

The experiment was performed with ten subjects, all of them were unprejudiced to what pseudo-haptic feedback is. Every subject was asked to perform 30 trails. The sequence of presented compliances was chosen randomly. The results of the experiment are presented in a confusion matrix (Table 1). The confusion matrix describes the number of perceived values in respect to the presented stimuli [9, 10].

Table 1

Results of the experiment presented as confusion matrix.

The experiment shows that 77.3 % of all presented stimuli were perceived correctly. 47 % of the wrong identified stimuli were perceived as weaker and 53 % as harder than the stimuli presented. All subjects were able to differ between n1 and n3 without any mistake.

In addition to the compliance identification experiment with the teleoperation system, all subjects were asked to repeat the experiment by using their finger directly at the compliance, to derive a reference value. Therefore, the subjects touched and displaced the compliance (Fig. 3 b, c) by their finger directly, instead of the end effector. With direct interaction 99.3 % of all presented stimuli were perceived correctly [11]. The result of the identification experiment without any technical devices in the perceptual channel, can be seen as gold standard for the maximum reachable performance for the identification of compliances with the presented setup.

4 Discussion and further work

The experiment shows the usability of pseudo haptic feedback for the discrimination of compliances. The compliances, chosen as stimuli in the experiment, derived from muscles compliances, allow the conclusion that the discrimination of different tissue is possible, too. While for first experiments the chosen setup with defined and time constant values is best to reach meaningful results, future experiments under more realistic conditions with tissue need to be performed.

Compliance values of muscles were used for first experiments. In future studies weaker material like liver or spleen with compliances up to 15 mm/N should be taken into account to derive information about the usability of pseudo-haptic feedback for very compliant tissue [12]. To give a surgeon the possibility to differ between characteristic material or tissue, the coupling characteristic of the grasping force at the user interface and the resulting end effector closing angle can be adapted.

Since the visual feedback is a very important factor of providing pseudo-haptic feedback, future studies need to take the intra operative sight in situ into account. In contrast to the presented setup of this work, during real surgery the end effector might be covered for some time. Also different kinds of viewing angles and a scaling of displacements need to be considered.

Funding

The work on this project is funded by the Deutsche Forschungsgemeinschaft (DFG) under grant WE 2308/13-1

References

  • [1]

    King, C. H.; Culjat, M. O.; Franco M. L.; Lewis, C. E.; Dutson, E. P.; Grundfest, W. S. & Bisley, J. W.: Tactile feedback induces reduced grasping force in robot-assisted surgery, Haptics, IEEE Transactions on, vol. 2, pp. 103-110, 2009. Google Scholar

  • [2]

    Okamura, A. M.: Methods for haptic feedback in teleoperated robotassisted surgery, Industrial Robot: An International Journal, vol. 31, pp. 499-508, 2004. Google Scholar

  • [3]

    Matich, S.; Neupert, C.; Kirschniak, A.; Werthschützky, R.; Schlaak, H. F. & Pott, P. P.: Teleoperation system with haptic feedback for single insicion surgery - concept ans system design, In: proceedings of cars, 2013. Google Scholar

  • [4]

    Lecuyer, A.: Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback, Presence: Teleoperators and Virtual Environments, MIT Press, 2009, 18, 39-53. Google Scholar

  • [5]

    Lecuyer, A.; Coquillart, S.; Kheddar, A.; Richard, P. & Coiffet, P.: Pseudo-haptic feedback: can isometric input devices simulate force feedback? Virtual Reality, 2000. Proceedings. IEEE, 2000, 83-90.Google Scholar

  • [6]

    Lecuyer, A.; Cuquillart, S. & Coiffet, P.: Simulating haptic information with haptic illusions in virtual environments, DTIC Document, 2001. Google Scholar

  • [7]

    Zheng, Y.-P.; Mak, A. F. & Lue, B.: Objective assessment of limb tissue elasticity: development of a manual indentation procedure, Journal of rehabilitation research and development, Rehabilitation Research and Development Service, US Department of Veterans Affairs, 1999. Google Scholar

  • [8]

    Neupert, C.; Hatzfeld, C.; Matich, S.: Teleoperationssystem mit intrinsischem haptischen Feedback durch dynamische Kennlinienanpassung für Greifkraft und Endeffektorkoordinaten, patent pending (2015011915374800DE), 2015. Google Scholar

  • [9]

    Jones, L. A. & Tan, H. Z.: Application of psychophysical techniques to haptic research, Haptics, IEEE Transactions on, 2013, 6, 268-284. Google Scholar

  • [10]

    Hatzfeld, C. & Kern, T. A.: Engineering Haptic Devices, A Beginners Guide, Second Edition Springer, 2014. Google Scholar

  • [11]

    Neupert, C.; Matich, S.; Hatzfeld, C.; Kupnik, M. & Werthschützky, R.: Investigation of the Usability of Pseudo-Haptic Feedback in Teleoperation, IEEE Worldhaptics Conference 2015, Evanston, USA, 2015. Google Scholar

  • [12]

    Carter, F. J.; Frank, T. G.; Davis, P. J.; McLean, D. & Cuschieri, A.: Measurements and modelling of the compliance of human and porcine organs. Medical Image Analysis 5.4, 2001, 231-236. Google Scholar

About the article

Published Online: 2015-09-12

Published in Print: 2015-09-01


Funding Source: Deutsche Forschungsgemeinschaft

Award identifier / Grant number: WE 2308/13-1

The work on this project is funded by the Deutsche Forschungsgemeinschaft (DFG) under grant WE 2308/13-1


Author's Statement

Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.


Citation Information: Current Directions in Biomedical Engineering, Volume 1, Issue 1, Pages 160–163, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2015-0040.

Export Citation

© 2015 by Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Lina Becerra, Thaiany Pedrozo Campos Antunes, Heather M. Capel, Sandra A. Wiebe, and Kim D. Adams
Assistive Technology, 2018, Page 1
[2]
Carsten Neupert, Sebastian Matich, Nick Scherping, Mario Kupnik, Roland Werthschutzky, and Christian Hatzfeld
IEEE Transactions on Haptics, 2016, Volume 9, Number 3, Page 397

Comments (0)

Please log in or register to comment.
Log in