Abstract
Intraoperative thermal neuroimaging is a novel intraoperative imaging technique for the characterization of perfusion disorders, neural activity and other pathological changes of the brain. It bases on the correlation of (sub-)cortical metabolism and perfusion with the emitted heat of the cortical surface. In order to minimize required computational resources and prevent unwanted artefacts in subsequent data analysis workflows foreground detection is a important preprocessing technique to differentiate pixels representing the cerebral cortex from background objects. We propose an efficient classification framework that integrates characteristic dynamic thermal behaviour into this classification task to include additional discriminative features. The first stage of our framework consists of learning this representation of characteristic thermal time-frequency behaviour. This representation models latent interconnections in the time-frequency domain that cover specific, yet a priori unknown, thermal properties of the cortex. In a second stage these features are then used to classify each pixel’s state with conditional random fields. We quantitatively evaluate several approaches to learning high-level features and their impact to the overall prediction accuracy. The introduction of high-level features leads to a significant accuracy improvement compared to a baseline classifier.
1 Introduction
Intraoperative thermal neuroimaging denotes an non-invasive and contactless imaging technique that measures the emitted temperature radiation of tissue during neurosurgical interventions. In neurosurgery, temperature gradients derive from heat transfers being caused by (sub-)cortical perfusion and neuronal activity. Gorbach et al. [1] and Shevelev et al. [2] showed the application of intraoperative thermography for tumour diagnostics. Steiner et. al. [3] demonstrated the detection of a ice-cold saline solution applied through a central line as a tool for perfusion diagnostics.
High-level feature representations are extracted from data in time-frequency domain so that they unveil time-dependent thermal behaviour of the human cortex. We further evaluate two common approaches to this feature representation learning task and finally analyse their influence to the overall performance. Latter is also compared to a simple baseline method.
2 Methods
Dynamic thermal signals can be described by a combination of several non-stationary characteristic signal components. These components originate from physiological sources as well as from noise. To approach this challenge subsequent signal analysis and classification is done in time-frequency domain to decompose the signal into its characteristic time-dependent components. As the physiological influences of cortical and subcortical perfusion as well as tissue composition are unknown and difficult to estimate there is no parametric model about cortical heat emission. For what reason we propose a machine learning framework that extracts empirical knowledge from learning data using linear feature representation schemes for pixelwise latent state prediction. The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, all intraoperative procedures in this work were approved by the Human Ethics Committee of the Technische Universität Dresden (no. EK 323122008). Informed consent has been obtained from all individuals included in this study.
2.1 Preliminaries
The global thermographic signal
The approximation of background noise is done by exploiting its rather smooth nature by a two dimensional B-Spline model[1] to estimate and remove the background signal. Suppose we are given two one-dimensional B-Spline bases
In order to catch dynamic thermographic effects that correlate with the actual imaged object subsequent analysis is done in time-frequency domain by applying the Discrete Wavelet transform to the data. Given a frequency band (scale) j, index k and wavelet ψ, the background corrected wavelet coefficients ci( j, k) of pixel i read
The transformation is done in linear time using the fast wavelet transform based on pyramid algorithm. In the following we will drop ( j, k) to simplify the notation if both are clear from context.
2.2 Representation learning
Now we can proceed to finding a high-level representation of the thermal characteristics of our physiological signals. The representation should be robust to noise while preserving as much information as possible. In the following we will discuss two common approaches to learn a representation f(ci) of the wavelet transformed thermal data ci.
2.2.1 Bag of frequency words
A common approach to unveil representative features is the bag of words model. In terms of wavelet coefficients
and get representative frequency words
with length p column vector of ones
2.2.2 PCA
In the past, PCA was applied to learn a representation of neuroimaging data [6]. Suppose we measure n correlated m-dimensional wavelet transformed time-series stored as n rows in matrix
with
2.3 Learning unary potentials
Suppose each pixel i has latent state
The probability distribution is estimated by training each of the t decision trees on a bootstrap sample of our training data. This strategy effectively prevents overfitting as discussed by [7].
2.4 Structured classification of thermal neuroimaging data
In order to incorporate structural information into the classification process we propose a conditional random field (CRF) model. In this framework, the posterior distribution
with V being the set of pixels and E the set of edges between adjacent pixels i and i′. By fulfilling the local Markov property, this function can be factored so that
with indicator function
3 Results and discussion
The results were achieved by evaluating five intraoperative thermal measurements of five different cases of length 1024 frames (20 s). For this purpose thermal neuroimaging data was acquired just after exposure of the cerebral cortex during neurosurgical tumour resections. Three out of these datasets were used for training plus testing and two for validation. The training sample consists of 30% equally sampled data points of all three test datasets equally representing cortical and background pixels. All computations were done on a workstation with dual Intel Xeon E5-2630, 128 GB Ram and Nvidia Geforce GTX Titan Black graphics card.
We evaluated both PCA and the Bag of Words model for feature representation learning. Baseline performance was quantified by training a RF on the average temperature distribution μT. The accuracy was computed by
given multiple labeled datasets. TC denotes the number of true cortical-, TB the number of true background- and C as well as B represent the number of cortex and background pixels. These labellings were acquired by a medical expert. The results of both approaches are shown in Tables 1 and 2.
RF accuracy (%) | μT | BoW | PCA |
---|---|---|---|
Test | 86.4 | 93.1 | 96.5 |
Validation | 81.3 | 86.6 | 88.1 |
CRF accuracy (%) | μT | BoW | PCA |
---|---|---|---|
Test | 88.7 | 96 | 98.5 |
Validation | 87.7 | 89.1 | 89.2 |
By introducing high-level features into the classification process a significant improvement in accuracy between the baseline classifier and the extended version can be observed. The introduction of learnt high-level thermal features provides additional discriminative information resulting from characteristic thermal signatures. Structural information shrink the difference in accuracy between our baseline classifier and the extended ones. This is caused by temperature inhomogeneities correlating with tissue composition and perfusion which are compensated by the Potts model. In the present cortex classification task this behaviour seems favourable. Yet, in case of smaller objects like tumours or vessels further attention has to be paid to this behaviour in order to achieve reasonable true-positive rates.
4 Conclusion
Intraoperative thermal neuroimaging is a novel technique to image time-dependent cortical temperature variations during neurosurgical interventions. The main cause of temperature changes is cortical perfusion which is influenced by cell metabolism and tissue composition. The thermal processes of the exposed brain are not well understood, yet they provide valuable information to characterize tissue. In this work thermal process signatures are employed to improve the differentiation of pixels of the cerebral cortex to background pixels. For this purpose we propose a novel machine learning framework for analysis of intraoperative thermal neuroimaging data. The learning goal is to recognize dynamic temperature behaviour of the imaged human cortex. These high-level features are then incorporated into a subsequent tissue classification stage based on conditional random fields improving overall classification accuracy. In the future, this framework might enable a more fine-grain characterization of tissue composition based on its dynamic thermal behaviour.
Acknowledgement
The authors would like to thank all organizations and individuals that supported this research project.
Author’s Statement
Research funding: This work was supported by the European Social Fund (grant no. 100087783) and the Saxonian Ministry of Science and Art. Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use complies with all the relevant national regulations, institutional policies and was performed in accordance with the tenets of the Helsinki Declaration, all intraoperative procedures in this work were approved by the Human Ethics Committee of the Technische Universität Dresden (no. EK 323122008).
References
[1] Gorbach AM, Heiss JD, Kopylev L, Oldfield EH. Intraoperative infrared imaging of brain tumors. J Neurosurg. 2004;101;960–9.10.3171/jns.2004.101.6.0960Search in Google Scholar
[2] Shevelev IA, Tsicalov EN, Gorbach AM, Budko KP, Sharaev GA. Thermoimaging of the brain. J Neurosci Methods. 1993;46:49–57.10.1016/0165-0270(93)90140-MSearch in Google Scholar
[3] Steiner G, Sobottka SB, Koch E, Schackert G, Kirsch M. Intraoperative imaging of cortical cerebral perfusion by time-resolved thermography and multivariate data analysis. J Biomed Opt. 2011;16:016001–016001–6.10.1117/1.3528011Search in Google Scholar
[4] Eilers PH, Currie ID, Durban M. Fast and compact smoothing on large multidimensional grids. Comput Stat Data An. 2006;50:61–76.10.1016/j.csda.2004.07.008Search in Google Scholar
[5] Durban M, Currie I, Eilers P. Multidimensional P-spline mixed models : a unified approach to smoothing on large grids. 2006;1:2–5.Search in Google Scholar
[6] Lu H, Plataniotis KN, Venetsanopoulos AN. MPCA: multilinear principal component analysis of tensor objects. IEEE Trans Neural Netw. 2008;19:18–39.10.1109/TNN.2007.901277Search in Google Scholar
[7] Breiman L. Random forests. Machine Learning. 2001;45:5–32.10.1023/A:1010933404324Search in Google Scholar
©2016 Nico Hoffmann et al., licensee De Gruyter.
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.