Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Current Directions in Biomedical Engineering

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Buzug, Thorsten M. / Haueisen, Jens / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Kraft, Marc / Lenarz, Thomas / Leonhardt, Steffen / Malberg, Hagen / Penzel, Thomas / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Urban, Gerald A.


CiteScore 2018: 0.47

Source Normalized Impact per Paper (SNIP) 2018: 0.377

Open Access
Online
ISSN
2364-5504
See all formats and pricing
More options …

Intraoperative identification of somato-sensory brain areas using optical imaging and standard RGB camera equipment – a feasibility study

M. Oelschlägel
  • Corresponding author
  • Technische Universität Dresden, Institut für Biomedizinische Technik, 01062 Dresden +49 351 46332118, +49 351 46336026
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ T. Meyer / S. B. Sobottka / M. Kirsch / G. Schackert / U. Morgenstern
Published Online: 2015-09-12 | DOI: https://doi.org/10.1515/cdbme-2015-0066

Abstract

Intraoperative Optical Imaging (IOI) is a neuro-imaging technique that allows the visualization of changes in optical properties of the brain cortex. Recent developments enhanced the method regarding the robustness under intraoperative conditions. However, the necessity of additional hardware still limits the use in the operating room (OR). Since modern surgical microscopes are potentially equipped with all required hardware for imaging, we investigated the possible use of such standard RGB camera for IOI. Measurements were performed on eight patients. Changes in optical properties of the cortical surface were acquired with a monochrome CCD camera (AxioCam MRm) and simultaneously with a standard RGB camera (Trio 610). Maps of cortical activity were calculated from the image data and the quality of these maps was assessed with a spatial signal-to-noise ratio. Activity maps calculated from AxioCam MRm data showed highest SNR in six out of eight patients. In two patients the activity map calculated from Trio 610 red channel performed best overall. The Trio 610 maps calculated from red channel data performed best in three out of eight cases like the activity maps calculated from green channel data, whereas the activity map calculated from blue channel data performed best in only two cases. If the color channel with the highest SNR is chosen in each patient for comparison to AxioCam MRm, the median of the SNR (SNRAxioCam/SNRBestColorChannel) is 84 % (Quartile 1 (Q1): 78 %, Quartile 3 (Q3): 99%). Results reveal that the integration of the Intraoperative Optical Imaging method into the OR and surgical workflow can be further improved by using RGB camera equipment. A robust identification of somato-sensory areas seems possible. Due to the gain of information from different wavelength bands the need for intelligent evaluation algorithms is increased and should therefore be topic of future research.

Keywords: Optical Imaging; Sensory Cortex Localization; Brain Tumor Surgery

1 Introduction

Intraoperative Optical Imaging (IOI) is a neuro-imaging technique that can be utilized for visualization of functional brain areas during neurosurgical procedures. Subtle changes in the optical properties of the cortical brain surface are induced by stimulation of the corresponding specific function and acquired with a camera attached to the surgical microscope. In recent publications a robust stimulation protocol was introduced for detection of the median nerve area on the postcentral gyrus under intraoperative conditions in clinical routine [1, 4]. The protocol that is therefore used comprises image acquisition over nine minutes with alternating 30 s rest and 30 s median nerve stimulation periods. For image acquisition during this research the hardware setup consisted of a highly sensitive CCD camera for scientific application with 12-Bit digitalization. Light wavelength filtering was performed within the optical path of the camera with a 568 nm interference filter, aiming at the detection of blood volume changes. Even though the used hardware setup is manageable in the operating room, for further integration into clinical routine it is desirable to reduce additional imaging hardware, especially since modern surgical microscopes are potentially equipped with the required hardware for acquisition of image data. The aim of this work was to investigate if localization of functional brain areas is possible using a standard microscope-integrated RGB camera (24 Bit) in connection with the stimulation protocol describe in [1]. In the past, research groups showed especially in animal model that optical imaging using a multispectral approach with evaluation of different wavelength bands is promising since different physiological information is included in each wavelength band [2, 3]. Nevertheless, measurements on humans are still not existing, and a systematic investigation especially regarding the localization of functional areas under intraoperative conditions is still missing.

2 Methods

2.1 Imaging setup

Imaging was performed using two cameras (AxioCam MRm, Carl Zeiss MicroImaging, Germany; Trio 610, Carl Zeiss Meditec AG, Germany) attached via a beam splitter (50/50) to the surgical microscope (OPMI Pentero, Carl Zeiss Meditec AG, Germany). The AxioCam MRm was connected via FireWire IEEE1394 to a laptop (Lenovo T61p, Lenovo Ltd., USA) where the images were acquired with a self-programmed capturing tool, based on the AxioCam Software Development Kit (SDK). Within the optical path of the AxioCam MRm a band pass interference filter ( = 568 nm ± 5 nm, Edmund Optics GmbH, Germany) was installed. The Trio 610 camera head was connected to the corresponding camera controller unit (Trio 600 CCU, Carl Zeiss Meditec, Germany). The signal was grabbed from the controller unit with a Digital Video (DV) recorder connected via IEEE1394 to a second laptop (Lenovo T430, Lenovo Ltd., USA) where the images were acquired with a recording software (WinDV Software, Petr Mourek, Czech Republic).

2.2 Image acquisition

The image acquisition with both cameras was started manually. Synchronization of image data was performed postoperatively. Images of the AxioCam MRm were acquired with 4 frames per second (fps) with a resolution of 620 x 592 pixels, running the camera in 2x2 binning mode. Shutter time was adjusted to 50 ms. Images were saved as separate TIFF files. The data of the Trio 610 was acquired according to the DV standard in AVI container format (spatial resolution 720 x 576 pixel). Shutter time was adjusted on the controller unit to 1/1000 second, the red channel level was set to -5 to avoid saturated pixel values in this color channel. The light source intensity of the surgical microscope (300 W xenon) was adjusted for each measurement, so that both camera images were well illuminated. Image acquisition was performed over nine minutes.

2.3 Patients

Measurements were performed on eight patients undergoing tumor resection after given informed consent. Craniotomy and dura opening were performed according to standard surgical procedure. Detailed patient and pathology information are shown in Table 1.

Table 1

Patient characteristics.

2.4 Data evaluation

2.4.1 Pre-processing

In a first step, images within the AVI container with the Trio 610 image data were extracted. A synchronization frame was used to calculate an absolute delay between the image data of both cameras. The framerate of the Trio 610 image stream was reduced to 4 fps by extracting only the images with the lowest delay to the corresponding AxioCam MRm images respectively timestamps. The RGB images of the Trio 610 were split into intensity based images for each color channel. An elastic registration of each color channel image data respectively of the AxioCam MRm image data was performed using a modified Demon’s algorithm. From each registered dataset the measured intensity time course was pixel wise extracted and transformed by Fourier decomposition. Maps of cortical activity where calculated using an algorithm that was already described in [4] (see Fig. 1).

AxioCam MRm activity map for patient 8. Segmentation borders of postcentral gyrus are shown in green, borders of trepa-nation in red. Strength of activity is color coded from blue to red.
Figure 1

AxioCam MRm activity map for patient 8. Segmentation borders of postcentral gyrus are shown in green, borders of trepa-nation in red. Strength of activity is color coded from blue to red.

The postcentral gyrus was segmented by an experienced neurosurgeon for each patient. Therefore a white light image of the cortical surface, intraoperative acquired functional data from electrophysiological measurements, and preoperative acquired MRI data were used. At last segmentation was then overlayed on each map of cortical activity.

2.4.2 SNR, Dice coeflcient, and mass centre distance calculation

To assess the visual quality of each calculated activity map, a spatial signal-to-noise ratio was used. Therefore initially the mean power P sc and standard deviation PstdSC of the activity map within the region that was segmented as postcentral gyrus was calculated. The map was segmented using

t=P¯SC+PstdSC

as threshold. The final spatial SNR was calculated as

SNR=P¯SegScPStdSurround

with as mean power of the thresholded activity map within sensory cortex area and PStdSurround as standard deviation of the activity map power outside the segmented sensory cortex area. For comparison of the spatial extent of activity maps the Dice coefficient of each color channel map was calculated in comparison to AxioCam MRm. Furthermore the distances between the centers of mass of the thresholded activity maps within the region of the segmented postcentral gyrus were compared.

3 Results

The results of the SNR calculation for each patient and each activity map are shown in Figure 2. The activity maps calculated from AxioCam MRm data showed highest SNR in six out of the eight patients (patient no. 2, 3, 4, 5, 7, 8). In two patients (patient no. 1, 6) the activity map calculated from Trio 610 red channel performed best even in comparison to the AxioCam data. The Trio 610 color channel comparison reveals, that the activity maps calculated from red channel data performed best in three out of eight cases (patient no. 1, 2, 6) like the activity maps calculated from green channel data (patient no. 3, 4, 5), whereas the activity map calculated from blue channel data performed best in only two cases (patient no. 7, 8). If the color channel with the highest SNR is chosen in each patient for comparison to AxioCam MRm, the median of the SNR (SNRAxioCam/SNRBestColorChannel) is 84 % (Quartile 1 (Q1): 78 %, Quartile 3 (Q3): 99%). Figure 3 shows the calculated Dice coefficients for each color channel activity map in comparison to AxioCam MRm and Figure 4 shows the Euclidian distance of the center of mass for the activity maps. The green channel shows in five of the eight patients highest Dice coefficient, followed by blue channel with three out of eight and red channel with one out of eight cases. Corresponding to this results, the center of mass for activity maps calculated from red channel data is in most cases more distant to center of mass from AxioCam MRm activity maps than the green and blue channel activity map mass centers.

SNR comparison of activity maps calculated from both cameras. ~X, Q1, Q3: AxioCam MRm: 6.9, 5.7, 7.5; Trio 610 green channel: 4.9, 3.9, 5.0; Trio 610 blue channel: 5.2, 3.7, 5.9; Trio 610 red channel: 5.4, 3.15, 6.7.
Figure 2

SNR comparison of activity maps calculated from both cameras. ~X, Q1, Q3: AxioCam MRm: 6.9, 5.7, 7.5; Trio 610 green channel: 4.9, 3.9, 5.0; Trio 610 blue channel: 5.2, 3.7, 5.9; Trio 610 red channel: 5.4, 3.15, 6.7.

Dice coeflcients of the activity maps calculated from each color channel in comparison to AxioCam MRm activity map. ~X, Q1, Q3: Trio 610 green channel: 0.40, 0.31, 0.45; Trio 610 blue channel: 0.40, 0.29, 0.45; Trio 610 red channel: 0.21, 0.17, 0.29.
Figure 3

Dice coeflcients of the activity maps calculated from each color channel in comparison to AxioCam MRm activity map. ~X, Q1, Q3: Trio 610 green channel: 0.40, 0.31, 0.45; Trio 610 blue channel: 0.40, 0.29, 0.45; Trio 610 red channel: 0.21, 0.17, 0.29.

Distances (in numbers of pixels) between mass centers of the activity maps calculated from each color channel and the AxioCam MRm activity map. ~X, Q1, Q3: Trio 610 green channel: 13.8, 7.5, 20.6; Trio 610 blue channel: 13.2, 10.2, 20.6; Trio 610 red channel: 30.4, 16.8, 44.9.
Figure 4

Distances (in numbers of pixels) between mass centers of the activity maps calculated from each color channel and the AxioCam MRm activity map. ~X, Q1, Q3: Trio 610 green channel: 13.8, 7.5, 20.6; Trio 610 blue channel: 13.2, 10.2, 20.6; Trio 610 red channel: 30.4, 16.8, 44.9.

4 Discussion

The results reveal, that in general the identification of somato-sensory brain areas is possible with a microscope integrated standard RGB camera. In comparison to the AxioCam MRm in connection with a light wavelength filter, the SNR of the calculated activity maps is in most cases only slightly lower if the color channel with the highest SNR is chosen for comparison. Nevertheless a physiological interpretation of maps calculated from image data acquired over a wide range of light wavelength is much more difficult since different physiological phenomena are contributing with a different weighting to the measured signal. As shown in the results, the maps calculated from green channel (~500 nm – 600 nm) and blue channel (~400 nm – 500 nm) are often very similar regarding to the SNR of the calculated maps. Furthermore, the maps do have a higher Dice coefficient and lower mass center distances in nearly all patients than the red channel in comparison to AxioCam MRm. This is an indicator for the main physiological component that is responsible for optical changes. The main physiological component in green and blue channel is the change of blood volume like in the measurements performed at 568 nm (AxioCam MRm). The SNR of maps calculated from red channel data (~550 nm – 680 nm) shows a higher variability over the patient measurements. Furthermore the Dice coefficient is in nearly all cases lower and the distance of the mass centers is higher in activity maps computed from red channel data than in activity maps calculated from green/blue channel data. This might be due to the fact that in this wavelength band the absorption of oxyhemoglobin and deoxyhemoglobin differs more than in the other both wavelength bands and therefore changes in oxygenation are predominantly responsible for the optical changes that are made visible with the optical imaging technique. In two patients the maps calculated from red channel data showed even higher SNR’s than the maps calculated from AxioCam MRm data. This implies that under some circumstances the aiming at blood volume changes might not be ideal for localization of the somato-sensory cortex area. The multispectral approach with an RGB camera or even an approach with hyperspectral imaging might furthermore improve the imaging technique. The results of this study are promising but nevertheless it does have some limitations. The number of patients investigated is still small and includes a wide variety of tumor localization. Therefore, statistically significant estimations are not yet possible. Furthermore, the parameters that were used for comparison (SNR, Dice coefficient, mass center distance) are not suited for an inter-individual comparison since they are strongly dependent on the size of the area of the somato-sensory cortex that was exposed during neurosurgical procedure. If a big part of sensory cortex is exposed the mean power of the map and also the SNR is lower than if only a small area is exposed (under the assumption that the size of the active area within median nerve region has the same size in both cases). Nevertheless, for the aim of this study, the comparison of different cameras and color channels, the calculated parameters are well suited.

5 Conclusion

The results reveal that the integration of the Intraoperative Optical Imaging method into the OR and surgical workflow can be further processed by using RGB camera equipment e.g. the one that is already part of many modern surgical microscopes. A robust identification of somato-sensory areas seems to be possible. Due to the gain of information from different wavelength bands the need for intelligent evaluation algorithms is strongly increased and should therefore be topic of future research.

Funding

This work was financially supported by Carl Zeiss Meditec AG, Oberkochen, Germany.

References

  • [1]

    Sobottka SB, Meyer T, Kirsch M et. al.: Intraoperative optical imaging of intrinsic signals: a reliable method for visualizing stimulated functional brain areas during surgery. In: J Neurosurg. 119(4), pp. 853-63, 2013. Google Scholar

  • [2]

    Steimers A, Gramer M, Ebert B et. al.: Imaging of cortical hemoglobin concentration with RGB reflectometry. In: Proc. SPIE 7368, Clinical and Biomedical Spectroscopy, 736813. Munich, Germany, 2009. Google Scholar

  • [3]

    Prakash N, Biag JD, Sheth SA et al.: Temporal profiles and 2-dimensional oxy-, deoxy-, and total-hemoglobin somatosensory maps in rat versus mouse cortex. In:NeuroImage, 37(1), pp. 27 - 36, 2007. Google Scholar

  • [4]

    Oelschlägel M, Meyer T, Wahl H et. al.: Evaluation of intraoperative optical imaging analysis methods by phantom and patient measurements. In:Biomedizinische Technik/Biomedical Engineering . 58(4), pp. 257–267, 2013 Google Scholar

About the article

Published Online: 2015-09-12

Published in Print: 2015-09-01


Author's Statement

Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.


Citation Information: Current Directions in Biomedical Engineering, Volume 1, Issue 1, Pages 265–269, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2015-0066.

Export Citation

© 2015 by Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in