Physicians and surgeons have great interest in combining different imaging techniques to improve the diagnosis or image guided interventions. Therefore multimodal image registration is needed, whose aim is to find a model for the best possible correspondence of anatomical and functional structures between images obtained from different modalities. An important part of image registration is a distance measure, which evaluates the similarity between the registered images. The definition of a suitable distance measure is a difficult task in multimodal image registration because there are amongst others intensity variations caused by the different physical principals, which are employed by the different imaging devices. In order to overcome these problems, a new image descriptor for defining the similarity between two images has been introduced – the modality independent neigh-bourhood descriptor (MIND) . The idea is to reduce a multimodal registration problem to a monomodal one by mapping both images to a common comparable space. To capture as many features the image descriptor is multidimensional. This increases the computation time that is necessary for distance calculations.
The purpose of this paper is to study whether the dimensionality of the descriptor can be reduced without causing significant loss of quality in the registration results. The benefits due to the dimensionality reduction are noise removal and reduced computing times when calculating the similarity between descriptors as well as memory requirements. The principal component analysis (PCA) is used as a dimensionality reduction method . With a subsequent Horn’s parallel Analysis (PA), the problem specific number of principal components is selected . Finally, the registration results based on the reduced and the complete descriptor are compared.
2 Material and methods
This section presents an overview of the background to this work with a focus on the introduction of the image descriptor MIND. Furthermore, Horn’s PA is briefly explained. For a detailed description see .
2.1 Modality independent neighbourhood descriptor
The basic idea of MIND is the concept of local self-similarity, which has been previously used in image denoising . Multimodal images have a non corresponding intensity distribution, but they still represent the same anatomical structures. We take advantage of the assumption that the particular structure within a local neighbour-hood can be described by the similarities of small image patches and is preserved across modalities. This principle is illustrated in Fig. 1 where it is exemplarily shown that an edge can be extracted from both images using self-similarities. The formulation of an image descriptor based on this concept has several benefits. MIND is independent of the modality and the related contrast, while enabling a simple intensity-based registration. Due to the usage of patches, it is sensitive to a variety of features, such as corners, edges and textures.
The real-valued measurements are stored in an array I with corresponding spatial dimensionality d. The positions of the elements of I are denoted by . MIND is defined in form of a Gaussian-function with a patch-based distance DP and a variance estimate V as the essential components.
2.1.1 Patch-based distance
The patch-based distance is defined as the sum of squared differences (SSD) of all elements I(xj) within the two patches centred at two different positions xj. As this calculation has to be performed for each element of I, a convolution can be used instead of an element-wise calculation of the SSD. A convolution with a d-dimensional kernel CP of the same size as the size of the patch P is a very efficient method to generate the patch-based distances, yielding:(1)(2)
were r is an element of the set of search positions R within the search region. Ir is the array I translated by r in terms of (2). The patch-based distances are calculated for each search position according to (1) and stored in an (d + 1)-dimensional array DP. We have chosen a Gaussian filter kernel for CP.
2.1.2 Variance measure
The local variance measure V is calculated directly from the patch distances.V is defined as the average of the directly neighbouring patch distances for each element xj.
The formula can be written as:(3)
N defines the four-neighbourhood in two dimensions or the six-neighbourhood in three dimensions.
2.1.3 Spatial search region
It is important to define the search region since it has a large influence on the descriptor. The size and spatial sampling configuration of the search region have to be chosen (for more details see ). We used a dense sampling. In this case, the search positions r are elements of the set(4)
All neighbours of the current element I(xj) within a fixed neighbourhood, defined by k, are included for the calculation of MIND. When selecting the size and the configuration of the search region, it is important to note that the computation time is directly proportional to the number of search positions. In general, the search region as well as the patches are symmetrical, but they do not have to be square.
Using the previously described components, the general formula for MIND can be specified as follows:(5)
with z as an normalization constant, so that all entries of MIND are in a range of 0 to 1. This formulation of a Gaussian-function results in high entries for MIND, if two patches are similar, which leads to small distances. Otherwise low entries occur, if the patches are dissimilar and therefore have large patch-based distances. The registration process using MIND is divided into two main parts: Initially the descriptors are determined for each modality independently. In the subsequent registration procedure the optimal transformation parameters are searched by comparing the descriptors using monomodal similarity metrics, such as the sum of absolute differences (SAD).
2.2 Horn’s parallel analysis
A crucial issue while performing a PCA as dimensionality reduction method is the number of significant principal components, which provides a sufficiently good approximation of the data. An visual method for determining the number of relevant components is the scree plot, which demonstrates the curve shape of the eigenvalues that resembles the letter L or an elbow . One can determine the vertex of this L-curve from which all components to the right should be discarded. Because this approach requires a subjective decision, Horn’s parallel analysis, which has been introduced as a method to determine the number of factors to retain from the factor analysis is preferred . He suggested to compare the empirically obtained eigenvalues with those of un-correlated, normally distributed random variables. With increasing sample size, the distribution of the eigenvalues of random variables approximates a straight line parallel to the x-axis. Only factors with eigenvalues above this line are considered significant. Similarly, the PA can be used to determine the relevant components of a PCA. In this contribution the MATLAB® file pa_test (Shteingart, Hanan (2014). Parallel Analysis (PA) (www.mathworks.com/matlabcentral/fileexchange/44996), MATLAB Central File Exchange. Retrieved Dec 10, 2014) was used to generate the random eigenvalues. Thereby, the random data sets are generated by permutation of the original data set and the eigenvalues are determined by means of a PCA.
3 Results and discussion
The following experiments were carried out on data sets of the Visible Human Project® . Transverse histological images of the head, which were recovered with different color channels and T1- as well as T2-weighted coronal MRI-images of the thorax were used.
3.1 Dimensionality reduction of MIND
In the following, we explain our proposed approach to reduce the dimensionality of MIND. First, the descriptors of the given data sets were determined by using a dense sampling search region with k = 2 and a Gaussian filter kernel with the size of five elements in each dimension. Subsequently, a PCA with both descriptors is performed at the same time, because otherwise, the order of the calculated components be different. For this purpose, the descriptors have been concatenated in such a way that the number of observations equals double the number of elements in the data set and the number of variables corresponds to the cardinality of R. The resulting coefficient matrix is used to reduce the descriptors in its dimension. The reduced MIND arises from the multiplication of the entire MIND and a certain part of the coefficient matrix. In order to decide which part of the coefficient matrix is to be used, it has to be decided which principal components are regarded as significant. To answer this question, a PA is performed and the result is displayed in a scree plot together with the eigenvalues obtained from the PCA. Figure 2 illustrates the scree plots for both data sets. As seen from the left scree plot, the comparison with the eigenvalues obtained from the PA suggests that the first three principal components approximate the histological data set satisfactorily. For the MRI data set, however, five PCs have to be taken into account, since the corresponding eigenvalues are bigger than those of the random variables. The magnitudes of the eigenvalues are very different for the considered data sets. While in the histological data set, the high eigenvalue of the first PC is followed by very small eigenvalues of the other PCs. In contrast, the eigenvalues of the MRI data set are more evenly distributed. This can also be observed in Table 1, where the percentages of total variance explained by each of the first five PCs are listed. As seen from Table 1, 83.04 % of the total variance are explained by the selected three PCs. In the MRI data set, the first five PCs explain 78.22 % of the total variance. For a visual example of our proposed dimensionality reduced MIND both descriptors of the histological data set are depicted as RGB-images and slice by slice in Fig. 3. It can be seen that at some places the representations of the reduced MIND appears to be similar to the results of an edge detection. Besides edges other image details are also included in the reduced MIND. Furthermore, it becomes clear that the descriptors are comparable with each other.
3.2 Global rigid registration using MIND
A comparison of the registration results corresponding to the entire and reduced MIND is made, using the proposed automatic dimension reduction. One of the histological measurements is set as reference while the other one is defined as template. The reference is rotated clockwise by 20 degrees compared to the template. Successively the template is rotated by angles ranging from -180 to 180 degrees (with a step size of 1°). For each angle, the PCA and the resulting descriptors are recalculated. Then, the SAD of the descriptors corresponding to the particular elements of I are determined and summed over all descriptor elements. The results are shown in Fig. 4. As can be seen, the optima of both registration functions are located by -20 degrees, where it was to be expected since a counterclockwise rotation-direction was used. Due to the difference of a rotation about 20 degrees and the fact the the brain looks similar up-side down there is an other local minimum at 160 degrees. However, the optimum can be uniquely determined in a range of 60 degrees, which is acceptable for medical image registration, since rotations of smaller angles are common.
In summary it can be said that the dimensionality reduction of MIND preserves the quality in the registration result. Our method could also be valuable for other vectorial images (such as diffusion MRI) and offers a new way of visualisation of high-dimensional data. In future, the presented method should be applied to more complex registration problems. In particular, variational deformable image registration is an interesting application area due to the numerous existing regularisers. Furthermore, it could be sought of an efficient possibility for not having to recalculate MIND in every step of the registration process.
M. P. Heinrich, M. Jenkinson, M. Bhushan, T. Matin, F. V. Glee-son, S. M. Brady, and J. A. Schnabel, “Mind: Modality independent neighbourhood descriptor for multi-modal deformable registration,” Medical Image Analysis, vol. 16, no. 7, pp. 1423– 1435, 2012. Google Scholar
I. Jolliffe, Principal Component Analysis . Springer, 2002. Google Scholar
J. Horn, “A rationale and test for the number of factors in factor analysis,” Psychometrika, vol. 30, no. 2, pp. 179–185, 1965. Google Scholar
A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” in IEEE CVPR, vol. 2, pp. 60–65, 2005. Google Scholar
M. J. Ackerman, “The visible human project,” Proceedings of the IEEE, vol. 86, no. 3, pp. 504–511, 1998. Google Scholar
About the article
Published Online: 2015-09-12
Published in Print: 2015-09-01
Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animals use.