Abstract
In radiotherapy of abdominal and thoracic tumors, respiratory motion is a problem for an accurate treatment. Most current motion compensation techniques rely on externally acquired breathing signals of the patient. The systems in clinical use usually work with 1D surface motion signals to describe internal structure respiratory motion patterns. As a 1D signal is not able to describe complex motion patterns and breathing variations, in this work the Microsoft Kinect, which can record multidimensional respiratory surface motion signals, is proposed to be used instead. For the Kinect, a clinically acceptable measurement setup is designed and Kinect measurements are compared to the Varian RPM system (clinical standard). The results show that the signals are well aligned. An additional comparison of Kinect signals from different regions of interest on the chest further reveals variations between them. This illustrates that the use of a system that provides multidimensional signals is worthwhile; the knowledge about breathing variations could be applied for optimization of current clinical workflows.
1 Introduction
Radiation therapy is – besides surgery and chemotherapy – one important component in tumor therapy. For tumors in abdominal or thoracic regions, respiratory motion is a major challenge during therapy. It leads to a movement of the tumor during the respiratory cycle up to two centimetres [1]. One way to compensate for tumor motion is to increase safety margins to cover the complete motion range. Technical solutions for motion compensation are tumor tracking [2] and gating [3]. Tumor tracking is an active process with the radiation beam following the tumor motion. Gating means that radiation is only active when the tumor is located at a predefined position window. For the use of this technique, a breathing signal from the patient is required. Similarly, breathing signals are required for reconstruction of 4D-CT data, which often form the basis of treatment planning for abdominal and thoracic tumors. There are two different kinds of breathing signals an internal or external breathing signal. An example for an internal breathing signal is the movement of the diaphragm captured with fluoroscopy imaging. By fluoroscopy imaging, the patient is exposed to extra radiation, so mostly external breathing signals are used. External signals used in clinical practise are often one-dimensional surrogate signals like spirometers, abdominal belts or the Varian RPM system. The RPM system tracks the surface motion at one certain point of the chest, and a possible change from, e.g., abdominal to thoracic breathing can not be captured. This kind of changes in respiratory motion is a problem during imaging and radiation. To describe these motion variations, it is necessary to use multidimensional breathing signals [4]. Multidimensional breathing signals can for example be acquired using range image systems, which usually work with structured light [5] (e.g. Microsoft Kinect) or with Time-of-Flight (ToF) [6]. Such systems do not need any additional marker on the surface, and it is possible to simultaneously extract signals from different surface regions.
The equipment for recording respiratory surface motion in the University Medical Center Hamburg-Eppendorf is the RPM system. In this work, a setup for recording multidimensional breathing signals with a low cost range imaging camera – here the Microsoft Kinect – is designed, evaluated and signals acquired with the Kinect are compared to the RPM system signals.
Depth map from a test person, acquired by the Microsoft Kinect. The colormap show the Kinect raw values. The test person is located in front of the CT on the CT couch. Areas coloured in dark red indicate that the Kinect does not receive any dot patterns; for these areas, no depth information is available.
2 Material and methods
2.1 Hard and software equipment
2.1.1 Microsoft Kinect
2.1.1.1 Camera system
The Microsoft Kinect is a low cost camera system, which is equipped with three different devices: a colour camera, an infrared camera and an infrared projector. The infrared sensor of the Kinect has a resolution of 1200 x 960 pixels but for further processing and bandwidth limitation of the USB port the resolution is reduced to 640x480 pixels. The opening angle of the camera is 43° in vertical and 57° in horizontal. The framerate of the Kinect is 30 frames/second and the principle of the depth measurement of the sensor is structured light. The Kinect emits a infrared dot pattern that is reflected from an object in front of the camera and is captured by the infrared camera. By comparison of the received dot pattern to a reference pattern, the distance to the object is calculated. For further information see [7]. To connect the Kinect with a computer, we use the libfreenect software developed by the OpenKinect Project [8] and an in-house developed Python program. Recorded Kinect signals and captured depth maps (see Figure 1) are saved as Matlab matrices. For the signal extraction, a Matlab program is used, which averages the distance values from a predefined region of interest in every recorded time step. It is possible to select several regions of interest of the entire field of view for evaluation purposes.
2.1.1.2 Calibration of the depth measurement
To calibrate the Kinect raw disparity values draw to the real distance D of an object to the camera, the mathematical model of [7] is applied: There exists a linear relation between the inverse of the real distance D to the measured value draw of the Kinect: D–1 = a · draw + b
To determine the parameters a and b, a linear regression is performed based on a series of measurements for known object to sensor distances.
2.1.2 Varian RPM system and 4D motion platform
The Varian RPM system is an established clinical system to measure external respiratory motion signals. For measurement, a marker block with six reflective markers is placed on the patient’s surface and the movement of this block is captured in all spatial directions by an infrared camera. This infrared camera captures the reflected infrared light from the marker block. From the marker block position over the time, a 1D motion signal is calculated [9].
For a first analysis of the potential of the Kinect as a respiratory motion sensor, a 4D motion platform is used [10]. This platform is computer–controlled and able to move in all three spatial directions whereby it is possible to reproducibly simulate arbitrary motion trajectories. This motion trajectories can be simple sinusoidal trajectories or real patient trajectories, using for example an extracted RPM signal.
2.2 Study design
2.2.1 Measurement setup
In the first part of our study, the measurement setup is installed in the CT room with relation to a potential application of the Kinect measurements to reconstruction of 4D-CT data sets. The goal was to find a camera position on condition that the setup does not interfere with the clinical workflow. Thus, one important clinical restriction was to place the Kinect camera in such a way that the RPM system and signals were not influenced. Furthermore, the Kinect itself has some restrictions, which should be noted during the design of the setup: At first, the distance from the measured surface to the camera should be between 0.7 and 1 m to obtain a signal with reasonable noise ratio. A second parameter is the angle between the optical axis of the Kinect to the measured surface; large angles result in an increased fraction of reflected light from the surface to be captured by the camera. Corresponding setups for recording respiratory motion with Kinect or ToF cameras [5, 6] had also an influence on the design.
2.2.2 Experiments
At first, a Kinect calibration measurement series was performed and the calibration parameters were computed (cf. section 2.1.1.2). The second experiment was a comparison between the RPM and Kinect signals for a sinusoidal movement of the 4D motion platform and test person runs. In the third experiment, we compared Kinect signals that were extracted from different regions on the upper body of a test person, e.g. a thoracic region vs. a abdominal region.
3 Results and discussion
The general requirements for the Kinect measurement setup in the CT room were described in section 2.2.1. According to these, the Kinect was positioned centrally over the CT table. In this way, the Kinect field of view covers the whole chest of a person and the RPM system is not affected as the RPM system field of view is always below the Kinect mounting; see Figure 2. The construction is clamped to the table. The advantage of this setting is that it is not necessary to compensate for the table movement during (spiral) CT scanning. The height of the camera above the person is individually adjustable to make sure that the gantry of the CT does not interfere with the field of view of the Kinect. Test runs with this setup showed that neither the table movement or in-room lasers used for patient positioning influence the Kinect image quality.
For the Kinect calibration, depth images for 19 well-known object to sensor distances from 0.7 m to 1.22 m were acquired. The analysis of the linear fitting delivered the two calibration parameters: a = −2.74 · 10−6 and b = 0.0031. These parameters were used in the subsequent experiments for conversion from the Kinect raw disparity values to real world distance values in millimetres.
3.1 Comparison Kinect to RPM signals
A first comparison of breathing signals acquired by the Kinect and the RPM system has been conducted using the 4D motion platform. 12 sinusoidal motion trajectories in anterior-posterior (AP) direction with amplitudes from 8 to 24 mm were programmed. The comparison between the Kinect and the RPM signals revealed that they were temporally as well as regarding the measured motion amplitudes very well aligned and sufficiently capture the phantom motion (see Figure 3). The average overall amplitude difference between RPM and Kinect signals for the phantom experiments was 0.50 ± 0.35 mm (comparison of signal peak-to-peak deviations).
The Kinect setup implemented for this study.
Comparison of Kinect and RPM signals for a sinusoidal motion phantom movement of 12 millimeter amplitude. The two signals are very well aligned.
A similar comparison has been performed for test person measurements. Again, the RPM signal has been recorded in parallel to the Kinect. The comparison between the two signals (RPM signal; Kinect signal from the chest wall region with the RPM marker block) results in high correlation coefficients up to 0.98 and an average difference in motion amplitude of 0.92 ± 0.29 mm.
Both phantom and test person experiments therefore indicate that the Kinect has the potential to reliably capture respiratory surface motion.
3.2 Comparison of Kinect signals from different anatomical positions
A major advantage of the Kinect compared to the RPM system is the possibility to extract breathing signals from different anatomical regions by a single measurement. In this work, breathing signals from thoracic and abdominal regions of the upper part of the body were analysed. It can be observed that regions of interest (ROIs) that are close together yield almost identical signals (correlation coefficients of signals between of 0.97 and 0.99). But for regions with larger distances from another – like ROIs at the abdomen and the thorax –, the correlation coefficient decreases to 0.55. Figure 4 shows two different signals simultaneously acquired from the abdomen and the thorax of a test person for illustration purposes. The most notable difference between the two signals is their amplitude. In the first three breathing periods, the test person was asked to perform normal breathing. Then, the breathing pattern has been asked to be changed to thoracic breathing, which is observable by the strong increase of the amplitude of the signal from the thoracic region. For normal breathing, the average amplitude from all test persons for the thoracic ROI is 4.63 ± 0.78 mm; for the abdominal ROI it is 7.67 ± 0.62 mm. For forced thoracic breathing the measured motion amplitudes are 7.49 ± 1.68 mm (thorax) and 10.25 ± 1.81 mm (abdomen), respectively. A possible global phase shift between the signal from the different ROIs could not be detected. Only some specific breathing cycles exhibited a phase shift up 0.3 sec between the two signals.
Comparison of two Kinect signals extracted from thoracic and abdominal regions of the chest. The signal of the thoracic region has a lower motion amplitude than the signal from the abdomen – with the exception of a period of forced thoracic breathing of the test person (starting from t=13 sec).
4 Conclusion
In this work, we analysed the feasibility of an implementation of a clinical setup for respiratory surface motion measurement by the Microsoft Kinect. The results of the experiments indicate that the Kinect has the potential to measure respiratory motion in a clinical environment. The comparison of the signals of different chest wall regions reveal variations between the signals. This fact is important as the clinically applied RPM system is limited to the measurement of the surface movement at one single point of the chest wall; potential complex breathing pattern changes during treatment can be assumed to be insufficiently reflected by a single point.
Future investigations will focus on a detailed analysis of the variability and differences between breathing curves acquired for different regions of the chest wall and the potential influence on, e.g., 4D-CT reconstruction.
Author's Statement
Conflict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.
References
[1] Seppenwoolde Y, Shirato H, Kitamura K, et al. Precise and real-time measurement of 3D tumor motion in lung due to breathing and heartbeat, measured during radiotherapy. Int J Radiat Oncol Biol Phys 2002, 53: 822–834.10.1016/S0360-3016(02)02803-1Search in Google Scholar
[2] Schweikard A, Glosser G, Bodduluri M, Murphy MJ, Adler JR. Robotic motion compensation for respiratory movement during radiosurgery. Computer Aided Surg 2000, 5: 263–277.10.3109/10929080009148894Search in Google Scholar
[3] Kubo HD, Hill BC. Respiration gated radiotherapy treatment: a technical study. Phys Med Biol 1996, 41: 83–91.10.1088/0031-9155/41/1/007Search in Google Scholar PubMed
[4] McClelland JR, Hawkes DJ, Schaeffter T, King AP. Respiratory motion models: A review. Med Image Anal 2013, 17: 19–42.10.1016/j.media.2012.09.005Search in Google Scholar PubMed
[5] Tahavori F, Alnowami M, Wells K. Markerless respiratory motion modeling using the Microsoft Kinect for Windows. Proc. SPIE 2014, 9036: 90360K.10.1117/12.2043569Search in Google Scholar
[6] Wentz T, Fayad H, Bert J, et al. Accuracy of dynamic patient surface monitoring using a time-of-flight camera and B-spline modeling for respiratory motion characterization. Phys Med Biol 2012, 57: 4175–4193.10.1088/0031-9155/57/13/4175Search in Google Scholar PubMed
[7] Khoshelham K, Elberrink SO. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors 2012, 12: 1437–1454.10.3390/s120201437Search in Google Scholar PubMed PubMed Central
[8] Openkinect. http://openkinect.org/wiki/Main_Page, accessed 2015-01-12.Search in Google Scholar
[9] Varian Medical Systems: RPM Respiratory Gating System Reference Guide, Version 1.7, 2012.Search in Google Scholar
[10] Grohmann C, Frenzel T, Werner R, Cremers F. Design, performance characteristics and application examples of a new 4D motion platform. Z Med Phys 2014 [epub ahead of print].10.1016/j.zemedi.2014.09.003Search in Google Scholar PubMed
© 2015 by Walter de Gruyter GmbH, Berlin/Boston
This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
