Haptic feedback can improve the performance of telerobotic surgery in terms of decreasing the time of a surgery and the trauma [1, 2]. Also, providing haptic feedback for teleoperation allows a surgeon to use his haptic sensation, instead of using vision only, to estimate the interaction forces of an end effector in situ. This is supposed to reduce cognitive load of the surgeon.
State of the art teleoperation systems including haptic feedback are commonly based on an impedance-admittance system architecture . This implies a masterslave structure. A user controls the slave robot by interacting with the master unit. The user sets a position or velocity at the master. The slave follows the user’s input. The force between the end effector and tissue, resulting from the performed manipulation in situ, is measured. The measured forces are used to reflect haptic feedback to a user at the master unit.
In this case, it is required that the dexterity of the master unit fits the dexterity of the slave robot. At least one sensor at the slave and one actuator at the master unit is required to provide haptic feedback for each degree of freedom of the teleoperation system. The integration of force sensors to the end effector and implementing up to 7 actuators to the master input device, that are needed to provide full dexterity, enhances the complicity of a system. Also, the requirements concerning the dynamic behavior of the actuators rise with rising dexterity and workspace of the telerobotic system.
With our work, we introduce the principle of pseudo-haptic feedback as an option of reflecting haptic feedback of grasping forces of an end effector without the necessity of a moving or active part in the master unit.
Pseudo haptic feedback is well known from applications in virtual environments [4–6]. Different kinds of interaction and haptic behavior of virtual object can be simulated and fed back to the user by for example changing the resulting moving characteristics of a mouse curser dependent on special tasks or interaction. The haptic sensation of pseudo haptics bases on a haptic illusion resulting from the mismatch of haptic- and visual perception [4–6].
For the pseudo-haptic teleoperation, we assume a stiff interface as input device (Figure 1). The input device is sensitive to the grasping force of the user (Fg). The closing angle of the end effector (φ) is controlled by the user’s grasping force. The acting interaction force at the end effector (Fe) is measured.
The relation between the grasping force and the end effector is designed as a linear spring for Fe = 0 to present the characteristics of an surgical forceps (Figure 2). Based on interaction forces acting at the end effector, the coupling characteristic of grasping force and end effector closing angle is altered. For example the interaction with a linear compliance (Fe = 1/n · φ) results in a drop of the slope of the characteristic curve. Dependent on mechanical properties, the interaction with any kind of tissue leads to unique characteristic curves. The change of the coupling characteristic, depending on the case of interaction, causes a realistic haptic feedback.
The haptic feedback perceived by the user results from the perceived difference of grasping forces at the user interface needed to reach the same end effector closing angle. The intensity and the rate of realism of the perceived sensation can be altered by using different coupling characteristics .
A compliance identification experiment is performed to verify the principle of pseudo-haptic teleoperation. An adjustable compliance is used as specimen to simulate different properties of tissue (Figure 3, b, c). For the experiment the specimen’s compliance is set to n1 = 8 mm/N, n2 = 4 mm/N and n3 = 2 mm/N. The values are chosen with respect to the properties of muscles, relaxed (n ≈ 8 mm/N) and contracted (n ≈ 2mm/N) .
A load cell is used as isometric user interface (Figure 3, a). The end effector displacement is controlled by the user’s force excitation at the load cell. The end effector bases on a linear moving axis with a force sensor at its tip. The signals of the force sensor at the end effector is used to change the couple characteristic between the user’s grasping force and the end effector displacement.
While performing the experiment, the subjects were asked to identify the three compliances (n1 . . . n3) only by analyzing their excited grasping forces to the load cell and the visual information of the resulting end effector movement at the specimen.
The experiment was performed with ten subjects, all of them were unprejudiced to what pseudo-haptic feedback is. Every subject was asked to perform 30 trails. The sequence of presented compliances was chosen randomly. The results of the experiment are presented in a confusion matrix (Table 1). The confusion matrix describes the number of perceived values in respect to the presented stimuli [9, 10].
Results of the experiment presented as confusion matrix.
The experiment shows that 77.3 % of all presented stimuli were perceived correctly. 47 % of the wrong identified stimuli were perceived as weaker and 53 % as harder than the stimuli presented. All subjects were able to differ between n1 and n3 without any mistake.
In addition to the compliance identification experiment with the teleoperation system, all subjects were asked to repeat the experiment by using their finger directly at the compliance, to derive a reference value. Therefore, the subjects touched and displaced the compliance (Fig. 3 b, c) by their finger directly, instead of the end effector. With direct interaction 99.3 % of all presented stimuli were perceived correctly . The result of the identification experiment without any technical devices in the perceptual channel, can be seen as gold standard for the maximum reachable performance for the identification of compliances with the presented setup.
4 Discussion and further work
The experiment shows the usability of pseudo haptic feedback for the discrimination of compliances. The compliances, chosen as stimuli in the experiment, derived from muscles compliances, allow the conclusion that the discrimination of different tissue is possible, too. While for first experiments the chosen setup with defined and time constant values is best to reach meaningful results, future experiments under more realistic conditions with tissue need to be performed.
Compliance values of muscles were used for first experiments. In future studies weaker material like liver or spleen with compliances up to 15 mm/N should be taken into account to derive information about the usability of pseudo-haptic feedback for very compliant tissue . To give a surgeon the possibility to differ between characteristic material or tissue, the coupling characteristic of the grasping force at the user interface and the resulting end effector closing angle can be adapted.
Since the visual feedback is a very important factor of providing pseudo-haptic feedback, future studies need to take the intra operative sight in situ into account. In contrast to the presented setup of this work, during real surgery the end effector might be covered for some time. Also different kinds of viewing angles and a scaling of displacements need to be considered.
The work on this project is funded by the Deutsche Forschungsgemeinschaft (DFG) under grant WE 2308/13-1
Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.
King, C. H.; Culjat, M. O.; Franco M. L.; Lewis, C. E.; Dutson, E. P.; Grundfest, W. S. & Bisley, J. W.: Tactile feedback induces reduced grasping force in robot-assisted surgery, Haptics, IEEE Transactions on, vol. 2, pp. 103-110, 2009.
Okamura, A. M.: Methods for haptic feedback in teleoperated robotassisted surgery, Industrial Robot: An International Journal, vol. 31, pp. 499-508, 2004.
Matich, S.; Neupert, C.; Kirschniak, A.; Werthschützky, R.; Schlaak, H. F. & Pott, P. P.: Teleoperation system with haptic feedback for single insicion surgery - concept ans system design, In: proceedings of cars, 2013.
Lecuyer, A.: Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback, Presence: Teleoperators and Virtual Environments, MIT Press, 2009, 18, 39-53.
Lecuyer, A.; Coquillart, S.; Kheddar, A.; Richard, P. & Coiffet, P.: Pseudo-haptic feedback: can isometric input devices simulate force feedback? Virtual Reality, 2000. Proceedings. IEEE, 2000, 83-90.
Lecuyer, A.; Cuquillart, S. & Coiffet, P.: Simulating haptic information with haptic illusions in virtual environments, DTIC Document, 2001.
Zheng, Y.-P.; Mak, A. F. & Lue, B.: Objective assessment of limb tissue elasticity: development of a manual indentation procedure, Journal of rehabilitation research and development, Rehabilitation Research and Development Service, US Department of Veterans Affairs, 1999.
Neupert, C.; Hatzfeld, C.; Matich, S.: Teleoperationssystem mit intrinsischem haptischen Feedback durch dynamische Kennlinienanpassung für Greifkraft und Endeffektorkoordinaten, patent pending (2015011915374800DE), 2015.
Jones, L. A. & Tan, H. Z.: Application of psychophysical techniques to haptic research, Haptics, IEEE Transactions on, 2013, 6, 268-284.
Hatzfeld, C. & Kern, T. A.: Engineering Haptic Devices, A Beginners Guide, Second Edition Springer, 2014.
Neupert, C.; Matich, S.; Hatzfeld, C.; Kupnik, M. & Werthschützky, R.: Investigation of the Usability of Pseudo-Haptic Feedback in Teleoperation, IEEE Worldhaptics Conference 2015, Evanston, USA, 2015.
Carter, F. J.; Frank, T. G.; Davis, P. J.; McLean, D. & Cuschieri, A.: Measurements and modelling of the compliance of human and porcine organs. Medical Image Analysis 5.4, 2001, 231-236.