Abstract
Videos of the human skin contain subtle color variations associated with the blood volume pulse. This remote photoplethysmography signal can be used for heart rate monitoring and represents an alternative to signals obtained from contact-based hardware. We developed an algorithm that estimates the heart rate in real-time from photoplethysmography signals and evaluate its performance in the context of ultra-high-field magnetic resonance imaging. We compare its accuracy to heart rate values estimated from electrocardiography and finger pulse oximetry triggers, obtained from MR vendor-provided hardware. For eight subjects, two experiments are conducted with the patient table outside and inside a 7 Tesla scanner. During both 5 min setups, heart rates from the algorithm and contact-based methods are stored. Their comparison suggests technical feasibility of the contactless method but that it is inferior in accuracy compared to contact-based hardware and that low heart rates (≤50 beats per minute) and adequate illumination are major challenges for practical feasibility.
1 Introduction
It has been shown recently that videos of the human skin can be used to estimate cardiac activity [1, 2], which applies as well to videos recorded by MRI-compatible cameras [3, 4]. The remotely obtained photoplethysmography signal is comparable to one obtained by pulse oximetry (PO) and allows estimation of the heart rate (HR). Therefore, it offers an alternative to contact-based hardware for HR monitoring such as PO or electrocardiography (ECG), which both are subject to several limitations. In the context of ultra-high-field MRI, the sensitivity of the ECG to magneto- hydrodynamic interferences increases [5]. Additionally, application of the ECG electrodes may result in patient discomfort. The PO probe can only be applied to certain areas of the body (fingertip, earlobe), is very sensitive to movement, and bears the risk of failing because of low perfusion during long-duration examinations [6]. These disadvantages can be overcome by using a MR-compatible camera: It is not affected by magnetohydrodynamic interferences, not restricted to possibly low-perfused region of the body, and does not cause additional stress to the patient.
Recently, elaborate algorithms for offline HR estimation from remote photoplethysmography signals recorded with color cameras in fitness [7] or office settings [8–10] have been presented. They are primarily concerned with motion compensation [7, 8], analysis of camera color channels [9], and estimation of morphological parameters [10].
In the context of MRI in-bore HR monitoring, these results cannot be applied without adjustments because the HR has to be computed online and compatible cameras have significantly lower performance than the ones used in [7–10]. Furthermore, the illumination in the bore as well as the distance to the subject represent additional challenges. So far, it has been shown that using frequency filtering [3] and magnification [4], offline HR estimation from videos recorded with MRI in-bore cameras is possible. Results have not been validated in a larger group of subjects, and validation of online computation has not been done.
In this work, we compare the accuracy of online HR measurement using remote photoplethysmography, ECG, and PO in the context of ultra-high-field MRI for a population of eight healthy subjects. We developed a basic, realtime algorithm that computes the HR from spectral information of mean pixel variations on a subject’s forehead. We designed two experiments inside and outside the MR bore to analyze the influence of illumination on the algorithm’s accuracy.
2 Method
2.1 Study population
Eight healthy volunteers (gender: 2 female, age: 32.0±4.5 years, weight: 74.5±11.5 kg, height: 179.1±8.5 cm; mean±std) with different ethnic origins (6 Europe, 1 East Asia, 1 South Asia) were examined. Each volunteer was informed about the examination procedure by a physician and a written informed consent was obtained before the examination. The study was approved by the Ethics Commission of the Medical Faculty of the University Duisburg-Essen (Study number: 11-4898-BO) and was conducted in conformance with the Helsinki Declaration.
2.2 Experimental setup
A MRI-compatible camera (12M-i, MRC Systems, Heidelberg, Germany; B/W, 720x576 pixel, 25 frames-per-second) was used in an ultra-high-field scanner room (Magnetom 7T, Siemens, Erlangen, Germany). The camera was installed above the patient’s head using a custom-built stand made from MR-compatible material that was fixed beneath the patient’s head (see Figure 1). For each volunteer, we used two experimental setups with a duration of 300s each: (1) Patient table in home position with room illumination only (approximately 0.3T); (2) Patient table in scanning position inside the MR (7T). As the illumination inside the bore is significantly lower, we installed an off-the-shelf video projector to provide additional illumination. Subjects were positioned feet-first and no imaging was performed.
Experimental setup inside the bore A) MR-compatible camera B) Camera stand C) Light from video projector.
During both examinations the time points of PO and ECG triggers from the MR vendor-provided system were stored individually. ECG triggers were computed from the peaks of R waves (depolarization of the ventricles), while the PO triggers were computed from the maximum of the pulse curve (arrival of the pulse wave in the periphery). The raw ECG signal was sampled at 400 Hz and the raw PO signal at 50 Hz.
2.3 Algorithm
Using ROOT [11] and OpenCV [12] libraries, a real-time C++ algorithm was developed which assumes a heart rate between 48 and 180 beats per minute (bpm). A region-of-interest (ROI) that covered most of the subject’s forehead was manually defined before the algorithm was launched.
The algorithm continuously estimated the current HR from a signal f(n) which represents the mean pixel intensities inside the ROI over the last five seconds (N=125 frames):
Update of f(n)
Normalization of the amplitudes of f(n) to the range [0,1]
Zero padding of f(n) to N=256 values. Zeros are added equally before and after the original values of f(n).
Application of a Hamming window
to f(n) by element-wise multiplication in order to minimize the effect of spectral leakage in the next step.
Computation of the Fast Fourier Transform (FFT) of f(n)
HR estimation from the Fourier spectrum of f(n) by choosing the bin with the largest magnitude within a frequency interval of 0.8-3Hz, which corresponds to the considered heart rates of 48-180 bpm. For example, if the largest magnitude was found in a bin corresponding to a frequency of 1Hz, the HR was assumed to be 60 bpm. After computation of the HR, the value was stored together with a time-stamp and the algorithm returned to step (0).
2.4 Data analysis
A moving average filter (size: 10s shift: 1s) was used to compute the signals MAECG, MAPO and MAVID. The signals MAPO and MAECG contain averaged reciprocal interval lengths between successive PO and ECG triggers, respectively. MAVID contains averaged HR values computed by our algorithm. These low-pass filtered versions of the raw data allow to compare the results of the different measurement modalities despite different signal properties, e.g. the sample rate. All three averaged signals contain T=291 values that are compared via root-mean-square error:
3 Results
Table 1 shows the obtained RMSE values during both experiments. In the first setup, the ECG is assumed to be the most accurate modality and therefore MAECG is used as ground truth. In the second setup, the ECG is expected to be corrupted by magnetohydrodynamic interferences and therefore MAPO is used as ground truth. In both cases, the root-mean-square error between the remaining signals and ground truth is computed. Figures 2 and 3 display example results for two subjects in different setups.
RMSE results of MAECG, MAPO and MAVID. The results corresponding to the data shown in Figures 2/3 are highlighted.
| Setup 1) | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 |
|---|---|---|---|---|---|---|---|---|
| RMSE(MAECG, MAPO) | 0.6 | 0.3 | 3.3 | 4.3 | 4.4 | 3.0 | 1.9 | 0.3 |
| RMSE(MAECG, MAVID) | 6.1 | 4.9 | 3.3 | 1.3 | 5.8 | 5.0 | 1.0 | 1.2 |
| Mean HR bpm (ECG) | 52.5 | 52.0 | 66.4 | 58.7 | 53.6 | 50.4 | 57.0 | 60.2 |
| Setup 2) | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 |
|---|---|---|---|---|---|---|---|---|
| RMSE(MAPO, MAECG) | 56.8 | 36.3 | 47.0 | 56.0 | 13.2 | 58.7 | 60.0 | 65.4 |
| RMSE(MAPO, MAVID) | 8.9 | 12.3 | 6.0 | 4.4 | 9.0 | 3.4 | 2.3 | 1.8 |
| Mean HR bpm (PO) | 49.7 | 52.0 | 65.5 | 57.1 | 49.7 | 51.5 | 56.0 | 59.5 |
Results of subject seven in first setup. As can be seen, MAVID does not capture the fine short-time HR variations. The PO probe missed triggers around 100 and 250 seconds.
Results of subject four in second setup. MAECG is heavily distorted by magnetohydrodynamic interactions. Variations of MAPO around 50 and 120 s are incorrectly detected by the PO probe, presumably because of patient finger movement. The HR results of MAVID can be assumed to be more accurate.
As expected, the deviation between MAECG and MAPO is only minimal in the first setup. Larger deviations are a result of some missed PO triggers as can be seen in the examples of Figure 2 and 3 as well. The error between MAECG and MAVID is much larger and in most cases when the HR of the subject is near 50 bpm. In the second setup the ECG is corrupted by magnetohydrodynamic interferences resulting in many false additional triggers and therefore a HR that is too high. The accuracy of the algorithm is decreased inside the bore as well; in seven out eight cases, the error between MAPO and MAVID is larger than the error between MAECG and MAVID in the first setup.
4 Conclusion
The results obtained allow to draw several conclusions: – Online HR estimation based on photoplethysmography signals from the forehead using the described setup is technically feasible but inferior to contact-based methods such as ECG and PO. – As the error of the video-based approach compared to ground truth is in most cases inferior in the second setup compared to the first setup, we conclude that low illumination inside the bore poses a challenge for video-based HR monitoring. We increased illumination by using a video projector from the back of the bore, which may not be sufficient. – 50% of the subjects had a mean HR near to 50 bpm and presumably exhibited intervals where the HR was below the considered lower border of the algorithm (48 bpm). Therefore we suppose that the interval of assumed heart rates has to be adjusted. We leave this challenge for future work because the spectrum of the photoplethysmography signal contains low frequency components which are not associated with the blood volume pulse. This prohibits to increase the interval of considered heart rates in the current implementation because the algorithm would always wrongly choose a low frequency component due to the large magnitudes of these components. – In most cases the video-based HR monitoring approach is able to reproduce the general trend of the HR but the low frames-per-second performance of the camera prevents more accurate results. As can be seen in Figures 2 and 3, fine details of the HR variations are not captured. This is a result of the low number of frames used and the resulting low resolution of the spectrum’s frequency axis. The current algorithm uses frames from the last five seconds for HR estimation. Increasing this number would improve the resolution of the spectrum mathematically but would average the computed HR over a larger number of heart cycles, thus low-pass filtering the results and removing HR variations as well.
In summary, using photoplethysmography signals is a promising new approach for real-time HR monitoring and technical feasibility under certain conditions has been shown but more validation and development is needed.
Author’s Statement
Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.
References
[1] Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmo-graphic imaging using ambient light. Optics Express 2008; 16(26)10.1364/OE.16.021434Search in Google Scholar
[2] Poh MZ, McDuff DJ, Picard RW. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering 2011; 18(1)10.1109/TBME.2010.2086456Search in Google Scholar
[3] Maclaren J, Aksoy M, Ehrl J, Saranathan M, Bammer R. Simultaneous monitoring of cardiac and respiratory signals using a markerless optical system. Proc. Intl. Soc. Mag. Reson. Med 2014; 890.Search in Google Scholar
[4] Spicher N, Brumann C, Kukuk M, Ladd ME, Maderwald S. Eulerian video magnification for heart pulse measurement in MRI scanners. Proc. Intl. Soc. Mag. Reson. Med 2014; 4824.Search in Google Scholar
[5] Snyder CJ et al. Initial results of cardiac imaging at 7T. Magnetic Resonance in Medicine 2009; 6110.1002/mrm.21895Search in Google Scholar
[6] Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 2007; 2810.1088/0967-3334/28/3/R01Search in Google Scholar
[7] de Haan G, Jeanne V. Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering 2013; 6010.1109/TBME.2013.2266196Search in Google Scholar
[8] Wang W, Stuijk S, de Haan G. Exploiting spatial redundancy of image sensor for motion robust rPPG. IEEE Transactions on Biomedical Engineering 2015; 6210.1109/TBME.2014.2356291Search in Google Scholar
[9] McDuff DJ, Gontarekand S, Picard RW. Improvements in remote cardiopulmonary measurement using a five band digital camera. IEEE Transactions on Biomedical Engineering 2014; 6110.1109/TBME.2014.2323695Search in Google Scholar
[10] McDuff DJ, Gontarekand S, Picard RW. Improvements in remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Transactions on Biomedical Engineering 2014; 6110.1109/TBME.2014.2340991Search in Google Scholar
[11] Brun R, Rademarkers F. ROOT - an object oriented data analysis framework. Proc. AIHENP 1996 Workshop.10.1016/S0168-9002(97)00048-XSearch in Google Scholar
[12] Bradski G. OpenCV is an open-source, computer-vision library for extracting and processing meaningful data from images. Dr. Dobb’s Journal of Software Tools 2000Search in Google Scholar
© 2015 by Walter de Gruyter GmbH, Berlin/Boston
This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
