Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Current Directions in Biomedical Engineering

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Buzug, Thorsten M. / Haueisen, Jens / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Kraft, Marc / Lenarz, Thomas / Leonhardt, Steffen / Malberg, Hagen / Penzel, Thomas / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Urban, Gerald A.


CiteScore 2018: 0.47

Source Normalized Impact per Paper (SNIP) 2018: 0.377

Open Access
Online
ISSN
2364-5504
See all formats and pricing
More options …

Dimensionality reduction of medical image descriptors for multimodal image registration

Johanna Degen / Jan Modersitzki / Mattias P. Heinrich
Published Online: 2015-09-12 | DOI: https://doi.org/10.1515/cdbme-2015-0051

Abstract

Defining similarity forms a challenging and relevant research topic in multimodal image registration. The frequently used mutual information disregards contextual information, which is shared across modalities. A recent popular approach, called modality independent neigh-bourhood descriptor, is based on local self-similarities of image patches and is therefore able to capture spatial information. This image descriptor generates vectorial representations, i.e. it is multidimensional, which results in a disadvantage in terms of computation time. In this work, we present a problem-adapted solution for dimensionality reduction, by using principal component analysis and Horn’s parallel analysis. Furthermore, the influence of dimensionality reduction in global rigid image registration is investigated. It is shown that the registration results obtained from the reduced descriptor have the same high quality in comparison to those found for the original descriptor.

Keywords: multimodal image registration; similarity measure; MIND; PCA; parallel analysis

1 Introduction

Physicians and surgeons have great interest in combining different imaging techniques to improve the diagnosis or image guided interventions. Therefore multimodal image registration is needed, whose aim is to find a model for the best possible correspondence of anatomical and functional structures between images obtained from different modalities. An important part of image registration is a distance measure, which evaluates the similarity between the registered images. The definition of a suitable distance measure is a difficult task in multimodal image registration because there are amongst others intensity variations caused by the different physical principals, which are employed by the different imaging devices. In order to overcome these problems, a new image descriptor for defining the similarity between two images has been introduced – the modality independent neigh-bourhood descriptor (MIND) [1]. The idea is to reduce a multimodal registration problem to a monomodal one by mapping both images to a common comparable space. To capture as many features the image descriptor is multidimensional. This increases the computation time that is necessary for distance calculations.

The purpose of this paper is to study whether the dimensionality of the descriptor can be reduced without causing significant loss of quality in the registration results. The benefits due to the dimensionality reduction are noise removal and reduced computing times when calculating the similarity between descriptors as well as memory requirements. The principal component analysis (PCA) is used as a dimensionality reduction method [2]. With a subsequent Horn’s parallel Analysis (PA), the problem specific number of principal components is selected [3]. Finally, the registration results based on the reduced and the complete descriptor are compared.

2 Material and methods

This section presents an overview of the background to this work with a focus on the introduction of the image descriptor MIND. Furthermore, Horn’s PA is briefly explained. For a detailed description see [3].

2.1 Modality independent neighbourhood descriptor

The basic idea of MIND is the concept of local self-similarity, which has been previously used in image denoising [4]. Multimodal images have a non corresponding intensity distribution, but they still represent the same anatomical structures. We take advantage of the assumption that the particular structure within a local neighbour-hood can be described by the similarities of small image patches and is preserved across modalities. This principle is illustrated in Fig. 1 where it is exemplarily shown that an edge can be extracted from both images using self-similarities. The formulation of an image descriptor based on this concept has several benefits. MIND is independent of the modality and the related contrast, while enabling a simple intensity-based registration. Due to the usage of patches, it is sensitive to a variety of features, such as corners, edges and textures.

On the right and on the left histological images of the head, obtained from different color channels are shown. The MIND descriptors of the elements at the center of the marked rectangles are presented in the middle. This example illustrates that the concept of local self-similarities is suitable for extracting image features and the descriptors are comparable using monomodal distance measurements, e.g. sum of absolute/ squared differences.
Figure 1

On the right and on the left histological images of the head, obtained from different color channels are shown. The MIND descriptors of the elements at the center of the marked rectangles are presented in the middle. This example illustrates that the concept of local self-similarities is suitable for extracting image features and the descriptors are comparable using monomodal distance measurements, e.g. sum of absolute/ squared differences.

The real-valued measurements are stored in an array I with corresponding spatial dimensionality d. The positions of the elements of I are denoted by xj=[xj1,,xjd]d. MIND is defined in form of a Gaussian-function with a patch-based distance DP and a variance estimate V as the essential components.

2.1.1 Patch-based distance

The patch-based distance is defined as the sum of squared differences (SSD) of all elements I(xj) within the two patches centred at two different positions xj. As this calculation has to be performed for each element of I, a convolution can be used instead of an element-wise calculation of the SSD. A convolution with a d-dimensional kernel CP of the same size as the size of the patch P is a very efficient method to generate the patch-based distances, yielding:

DP(I,xj,CP,r)=CP*(IIr)2rR(1)withIr(xj)=I(xj+r)xj,(2)

were r is an element of the set of search positions R within the search region. Ir is the array I translated by r in terms of (2). The patch-based distances are calculated for each search position according to (1) and stored in an (d + 1)-dimensional array DP. We have chosen a Gaussian filter kernel for CP.

2.1.2 Variance measure

The local variance measure V is calculated directly from the patch distances.V is defined as the average of the directly neighbouring patch distances for each element xj.

The formula can be written as:

V(I(xj),CP)=1|N|nNDP(I,xj,CP,n).(3)

N defines the four-neighbourhood in two dimensions or the six-neighbourhood in three dimensions.

2.1.3 Spatial search region

It is important to define the search region since it has a large influence on the descriptor. The size and spatial sampling configuration of the search region have to be chosen (for more details see [1]). We used a dense sampling. In this case, the search positions r are elements of the set

R={k,,0,,k}d\{0}dk.(4)

All neighbours of the current element I(xj) within a fixed neighbourhood, defined by k, are included for the calculation of MIND. When selecting the size and the configuration of the search region, it is important to note that the computation time is directly proportional to the number of search positions. In general, the search region as well as the patches are symmetrical, but they do not have to be square.

Using the previously described components, the general formula for MIND can be specified as follows:

MIND(I(xj),CP,r)=1zexp(DP(I,xj,CP,rV(I(xj),CP))(5)

with z as an normalization constant, so that all entries of MIND are in a range of 0 to 1. This formulation of a Gaussian-function results in high entries for MIND, if two patches are similar, which leads to small distances. Otherwise low entries occur, if the patches are dissimilar and therefore have large patch-based distances. The registration process using MIND is divided into two main parts: Initially the descriptors are determined for each modality independently. In the subsequent registration procedure the optimal transformation parameters are searched by comparing the descriptors using monomodal similarity metrics, such as the sum of absolute differences (SAD).

2.2 Horn’s parallel analysis

A crucial issue while performing a PCA as dimensionality reduction method is the number of significant principal components, which provides a sufficiently good approximation of the data. An visual method for determining the number of relevant components is the scree plot, which demonstrates the curve shape of the eigenvalues that resembles the letter L or an elbow [2]. One can determine the vertex of this L-curve from which all components to the right should be discarded. Because this approach requires a subjective decision, Horn’s parallel analysis, which has been introduced as a method to determine the number of factors to retain from the factor analysis is preferred [3]. He suggested to compare the empirically obtained eigenvalues with those of un-correlated, normally distributed random variables. With increasing sample size, the distribution of the eigenvalues of random variables approximates a straight line parallel to the x-axis. Only factors with eigenvalues above this line are considered significant. Similarly, the PA can be used to determine the relevant components of a PCA. In this contribution the MATLAB® file pa_test (Shteingart, Hanan (2014). Parallel Analysis (PA) (www.mathworks.com/matlabcentral/fileexchange/44996), MATLAB Central File Exchange. Retrieved Dec 10, 2014) was used to generate the random eigenvalues. Thereby, the random data sets are generated by permutation of the original data set and the eigenvalues are determined by means of a PCA.

3 Results and discussion

The following experiments were carried out on data sets of the Visible Human Project® [5]. Transverse histological images of the head, which were recovered with different color channels and T1- as well as T2-weighted coronal MRI-images of the thorax were used.

3.1 Dimensionality reduction of MIND

In the following, we explain our proposed approach to reduce the dimensionality of MIND. First, the descriptors of the given data sets were determined by using a dense sampling search region with k = 2 and a Gaussian filter kernel with the size of five elements in each dimension. Subsequently, a PCA with both descriptors is performed at the same time, because otherwise, the order of the calculated components be different. For this purpose, the descriptors have been concatenated in such a way that the number of observations equals double the number of elements in the data set and the number of variables corresponds to the cardinality of R. The resulting coefficient matrix is used to reduce the descriptors in its dimension. The reduced MIND arises from the multiplication of the entire MIND and a certain part of the coefficient matrix. In order to decide which part of the coefficient matrix is to be used, it has to be decided which principal components are regarded as significant. To answer this question, a PA is performed and the result is displayed in a scree plot together with the eigenvalues obtained from the PCA. Figure 2 illustrates the scree plots for both data sets. As seen from the left scree plot, the comparison with the eigenvalues obtained from the PA suggests that the first three principal components approximate the histological data set satisfactorily. For the MRI data set, however, five PCs have to be taken into account, since the corresponding eigenvalues are bigger than those of the random variables. The magnitudes of the eigenvalues are very different for the considered data sets. While in the histological data set, the high eigenvalue of the first PC is followed by very small eigenvalues of the other PCs. In contrast, the eigenvalues of the MRI data set are more evenly distributed. This can also be observed in Table 1, where the percentages of total variance explained by each of the first five PCs are listed. As seen from Table 1, 83.04 % of the total variance are explained by the selected three PCs. In the MRI data set, the first five PCs explain 78.22 % of the total variance. For a visual example of our proposed dimensionality reduced MIND both descriptors of the histological data set are depicted as RGB-images and slice by slice in Fig. 3. It can be seen that at some places the representations of the reduced MIND appears to be similar to the results of an edge detection. Besides edges other image details are also included in the reduced MIND. Furthermore, it becomes clear that the descriptors are comparable with each other.

On the left side the eigenvalues of the histological data set are shown together with the eigenvalues obtained from the PA. On the right side the scree plot of the MRI data set is visualised.
Figure 2

On the left side the eigenvalues of the histological data set are shown together with the eigenvalues obtained from the PA. On the right side the scree plot of the MRI data set is visualised.

Visualisation of the reduced MIND for the histological data set. The rows represents the two different color channels from which the histological images were taken. Column by column, the reduced descriptors are shown once as RGB-images in the first column and in the other columns slice by slice.
Figure 3

Visualisation of the reduced MIND for the histological data set. The rows represents the two different color channels from which the histological images were taken. Column by column, the reduced descriptors are shown once as RGB-images in the first column and in the other columns slice by slice.

Table 1

Percentages of the explained variance

3.2 Global rigid registration using MIND

A comparison of the registration results corresponding to the entire and reduced MIND is made, using the proposed automatic dimension reduction. One of the histological measurements is set as reference while the other one is defined as template. The reference is rotated clockwise by 20 degrees compared to the template. Successively the template is rotated by angles ranging from -180 to 180 degrees (with a step size of 1°). For each angle, the PCA and the resulting descriptors are recalculated. Then, the SAD of the descriptors corresponding to the particular elements of I are determined and summed over all descriptor elements. The results are shown in Fig. 4. As can be seen, the optima of both registration functions are located by -20 degrees, where it was to be expected since a counterclockwise rotation-direction was used. Due to the difference of a rotation about 20 degrees and the fact the the brain looks similar up-side down there is an other local minimum at 160 degrees. However, the optimum can be uniquely determined in a range of 60 degrees, which is acceptable for medical image registration, since rotations of smaller angles are common.

The registration results of the global rigid registration with the entire as well as the reduced MIND. The correct alignment of both images is found equally with the reduced MIND and the quality of the registration result is preserved.
Figure 4

The registration results of the global rigid registration with the entire as well as the reduced MIND. The correct alignment of both images is found equally with the reduced MIND and the quality of the registration result is preserved.

4 Conclusion

In summary it can be said that the dimensionality reduction of MIND preserves the quality in the registration result. Our method could also be valuable for other vectorial images (such as diffusion MRI) and offers a new way of visualisation of high-dimensional data. In future, the presented method should be applied to more complex registration problems. In particular, variational deformable image registration is an interesting application area due to the numerous existing regularisers. Furthermore, it could be sought of an efficient possibility for not having to recalculate MIND in every step of the registration process.

References

  • [1]

    M. P. Heinrich, M. Jenkinson, M. Bhushan, T. Matin, F. V. Glee-son, S. M. Brady, and J. A. Schnabel, “Mind: Modality independent neighbourhood descriptor for multi-modal deformable registration,” Medical Image Analysis, vol. 16, no. 7, pp. 1423– 1435, 2012. Google Scholar

  • [2]

    I. Jolliffe, Principal Component Analysis . Springer, 2002. Google Scholar

  • [3]

    J. Horn, “A rationale and test for the number of factors in factor analysis,” Psychometrika, vol. 30, no. 2, pp. 179–185, 1965. Google Scholar

  • [4]

    A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” in IEEE CVPR, vol. 2, pp. 60–65, 2005. Google Scholar

  • [5]

    M. J. Ackerman, “The visible human project,” Proceedings of the IEEE, vol. 86, no. 3, pp. 504–511, 1998. Google Scholar

About the article

Published Online: 2015-09-12

Published in Print: 2015-09-01


Author’s Statement

Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animals use.


Citation Information: Current Directions in Biomedical Engineering, Volume 1, Issue 1, Pages 201–205, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2015-0051.

Export Citation

© 2015 by Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in