Physiotherapy is a therapy, in which mechanical force and movements are applied by a specialized therapist to prevent and cure injuries. The manual therapy consisting of specific motions and exercises are applied by the therapist and executed by the patient. Therapeutic exercise programs are designed by the therapist to fit the unique needs of an individual patient. Besides manual therapy and exercises applied during appointments with the therapist, the patient is usually instructed to perform additional exercises at home to support the rehabilitation process. To achieve satisfying results, the patient has to perform the exercises on a daily basis, which, exercising at home alone, the patient often is not motivated enough to perform the exercises on a regular basis . Additionally, the patient needs to pay close attention to both the body posture and the range of motions, repeating the exercises the exact same way as shown by the therapist to prevent unnecessary strain to the joints and muscles, possibly leading to further injuries .
Game consoles employing various ways of user interaction have nowadays been established as a form of home-based exercise. Systems such as Nintendo Wii (Nintendo AG, Kyołto, Japan), PlayStation Eye (Sony Computer Entertainment Inc., Tokyo, Japan) or Microsoft Kinect (Microsoft Corporation, Redmond, USA) allow the user to interact with a computer game by either using a special controller with acceleration and tilt sensors (Wii) or through motion recognition (Eye, Kinect).
Additionally, tablet computers have replaced desktop computers and notebooks in a wide area of applications. The large amount of Apps available for such systems helps users with limited computer knowledge to easily use tablets in daily life.
Combining a tablet computer with a camera based controller such as the Microsoft Kinect system would allow to create a novel physiotherapy system for home use. It would enable the therapist to select or record an individual set of exercises, which the patient should execute at home. The physiotherapist would have the chance to control and improve the patient’s movements, repetitions and most importantly the body posture based on the recordings of the exercises whereas the patient would be able to stay motivated. We thus propose a system that employs a Kinect camera system to track the movements of human extremities and that allows a patient to execute various exercises. Figure 1 depicts a schematic representation of the intended setup. Infrared depth sensors are used to track the patient’s movements while the RGB camera allows the user to reflect the motions on the tablet screen. The tablet software tells the user which exercises to execute and – if necessary – to correct the current posture.
2.1 Kinect system
The Kinect camera is a compact system, consisting of a VGA camera and a pair of infrared sensors, which serve as depth sensors. The system may be connected to a computer via USB and is usable with various programming languages, e.g.
C# (Microsoft Corporation) or C++. The Kinect system is able to track up to six people under normal conditions with two people being active, i.e. moving, at the same time. Based on the depth information it tracks movements of the human extremities by computing the position of various skeleton joints. The origin of the skeleton structure is the hip centre, from which the skeletal tracking algorithm builds the whole skeleton. The position of all joints can be accessed by calling the respective joint functions provided in the Kinect code library. In total, the position of 20 joints or body parts can be accessed. Figure 2 shows all available joints.
2.2 Calculation of extremity lengths
To individualize physiotherapy treatment it might be necessary to calculate certain body measurements as different body types may lead to individual workouts. In the proposed system, the length between two joints A and B is calculated as:(1)
with x, y and z denoting the Cartesian coordinates of joints A and B in three dimensional space.
2.3 Calculation of angles
Calculating the angle in which the extremities are abducted from the body is an important measure to evaluatev correctness of the exercised workouts. All angles can be calculated through triangle relations, using three given sides. Figure 3 gives an example on how to calculate the angle in which the elbow is adducted.
Solving the triangular equation, the elbow angle can be calculated as:(2)
with a denoting the length of the humerus (AB), b denoting the length of the ulna (BC) and c being the distance between wrist and shoulder (CA).
2.4 Calculating distance between floor and body joints
Some physiotherapy exercises (e.g. push-ups or squats) might need to use the distance between the floor and certain body joints to determine whether the exercise is done correctly. The Kinect systems allows to read the floor coordinates, thus distance between a joint J and the floor (F) can be calculated as :(3)
with A, B, C and D denoting the floor plane coordinates and xJ, yJ and zJ describing the coordinates of the joint.
Figure 4 shows an exemplary user interface to control and record physiotherapy exercises. On the left it provides a live image of the patient showing the current position of all skeleton joints. Upper and lower spine joint as well as the hip center are marked in yellow, shoulder and hip joints are shown in red, wrists and ankles are marked in pink, while the feet and the head are shown in blue and light blue, respectively. The presented example instructs the user to do five squats while keeping lower legs in parallel to reduce strain to the knee ligaments. A pictographic image shows a schematic overview of the exercise. The user is able to start the exercise by waving the left hand. During exercise, the user is instructed to bend and extend the knees, indicated by an arrow. A repetition is counted as valid when the knee angle is decreased below 120°. During exercise, the distance between knee joints is controlled to be equal to the distance between ankles to ensure parallelism of the lower legs. If the lower legs are not kept in parallel, the current repetition is not counted and the user has to restart that specific repetition. After five repetitions, the user is informed that the exercise is finished. Additionally, the user is enabled to save the exercise data. The recorded data can later be analyzed by a physiotherapist. The recorded data includes the position of all joints along with the length of all extremities during the exercises.
The proposed system allows tracking body movements during exercise and calculating the position of body joints, the length of extremities and the distance between a body joint and the floor. That information may be used to instruct and control home-based physiotherapy exercise.
The use of the Microsoft Kinect system as a tool for rehabilitation or physiotherapy has previously been reported in several publications. Luna-Oliva et al. reported on the use of such a system on children with cerebral palsy to train various motor and process skills . Chang et al. showed that using a Kinect-based system for physical rehabilitation in young adults helps in increasing motivation during exercises and thus improving exercise performance . Kinect-based physiotherapy systems currently available on the market (e.g. ) use patient-computer interaction to provide individualized exercises and an encouraging rehabilitation routine. However, aside from counting the number of exercise executions they do not control the exact posture during the workout. The presented system intends to include a tight control of body posture and to give feedback to the patient during exercise.
The employed Kinect camera is a low-cost platform allowing implementing the proposed system into current physiotherapy routine without stressing health insurance costs. However, due to its focus on being used in a computer game environment, where the exact tracking of all joints might not be the primary goal of the Kinect system, the calculated joint coordinates might be not exact enough to be used in a physiotherapy setting. Thus, a general analysis of the exactness of joint positions computed by the Kinect might be necessary. Preliminary tests employing a high-resolution 3D tracking system with eight infrared cameras and reflective markers showed an acceptable accuracy of the Kinect calculations.
The presented user interface currently includes only a limited number of exercises. The implementation of additional exercises and the possibility to compose individual workout plans through physiotherapists is planned for future work. Additionally, an evaluation including a greater number of participants to assess both the effectiveness of the posture control and the improvement of the outcome of the physical therapy is planned for future work.
This work was supported by the Bundesministerium für Bildung und Forschung (TREFFER, Grant 01PL11008).
D. Lloyd-Jones, R. J. Adams, T. M. Brown, M. Carnethon, S. Dai, G. De Simone, T. B. Ferguson, E. Ford, K. Furie, et al., ”Heart disease and stroke statistics–2010 update: a report from the American Heart Association,” Circulation, vol. 121 (7), pp. e46-e215, 2010. Google Scholar
C. Kisner and L. A. Colby, Therapeutic Exercise - Foundations and Techniques, 6 ed. Philadelphia: F.A. Davis Company, 2012. Google Scholar
Z. A. Mundher and J. Zhong, ”A Real-Time Fall Detection System in Elderly Care Using Mobile Robot and Kinect Sensor,” Int J Mater Mech Man, vol. 2 (2), pp. 133-138, 2014. Google Scholar
L. Luna-Oliva, R. M. Ortiz-Gutierrez, R. Cano-de la Cuerda, R. M. Piedrola, I. M. Alguacil-Diego, C. Sanchez-Camarero, and C. Martinez Culebras Mdel, ”Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: a preliminary study,” NeuroRehabilitation, vol. 33 (4), pp. 513-21, 2013. Google Scholar
Y. J. Chang, S. F. Chen, and J. D. Huang, ”A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities,” Res Dev Disabil, vol. 32 (6), pp. 2566-70, 2011. Web of ScienceGoogle Scholar
RespondWell Rehab Platform, Available under: www.respondwell.com. Google Scholar
About the article
Published Online: 2015-09-12
Published in Print: 2015-09-01
Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.