An Overview of Segmentation Algorithms for the Analysis of Anomalies on Medical Images

Abstract Human disease identification from the scanned body parts helps medical practitioners make the right decision in lesser time. Image segmentation plays a vital role in automated diagnosis for the delineation of anatomical organs and anomalies. There are many variants of segmentation algorithms used by current researchers, whereas there is no universal algorithm for all medical images. This paper classifies some of the widely used medical image segmentation algorithms based on their evolution, and the features of each generation are also discussed. The comparative analysis of segmentation algorithms is done based on characteristics like spatial consideration, region continuity, computation complexity, selection of parameters, noise immunity, accuracy, and computation time. Finally, in this work, some of the typical segmentation algorithms are implemented on real-time datasets using Matlab 2010 software, and the outcome of this work will be an aid for the researchers in medical image processing.


Introduction
In the computer-aided diagnosis of human diseases from medical images, the detection of a region of interest decides the path of diagnosis. More precisely, segmentation algorithms for medical images can be grouped into three generations based on their characteristics and evolution and are depicted in Figure 1. Prior to segmentation, the pre-processing of the medical image was done for the removal of noise and artifacts [28,79]. In Ref. [21], selective average filter was proposed for the filtering of speckle noise. Satisfactory results were produced for synthetic ultrasound (US) images corrupted by speckle noise and real US images of female pelvic cavity. The inspiration from the behavior of herbivore organism paves the way for new artificial life model for the enhancement of synthetic and real images [20].
The variational filtering algorithm was found to be efficient in the removal of multiplicative noise, and the parallelism was utilized in graphical processor unit through the CUDA architecture [30]. In Refs. [18,66,98], a detailed study on medical image segmentation algorithms and their characteristics were analyzed. A detailed study on clustering and supervised classification segmentation models was carried out in Ref. [36] for identification and quantification of atherosclerotic plaques in computed tomography (CT), magnetic resonance (MR), and US images. Roberta B. Oliveira et al. performed an exhaustive survey on segmentation of skin lesions on dermoscopy images [62]. Mercedes Filho et al. also analyzed the segmentation, classification, and quantification models for the skin lesions on dermoscopy images [47]. In Ref. [84], the automated segmentation models for the three-dimensional (3D) pulmonary nodule detection in CT images of lungs were discussed. The biomechanical modeling techniques and segmentation models for ear images were described in Ref. [26]. The Materials and Methods section describes the acquisition protocol and works related to segmentation algorithms. An exhaustive search has been made on the works related to segmentation algorithms from Web of Science, Scopus database, IEEE Explorer, and Google Scholar. The classical segmentation algorithms and their modifications are discussed in first-generation algorithms. The second-generation algorithms comprise of deformable, clustering, watershed, and Markov random field (MRF) techniques and their variants. The thirdgeneration algorithms highlight classifier, graph-based, atlas-guided, and hybrid approaches. The Results and Discussion section depicts the comparative analysis of segmentation algorithms and the simulation results of some of the widely used segmentation algorithms tested on real-time medical images.

Acquisition Protocol
The medical images corresponding to four data sets were taken in this work for analysis. The brain and abdomen data sets have been acquired on an Optima CT machine (General Electric, USA). The CT abdomen data sets comprise of malignant liver tumor subject and renal cell carcinoma subject. The CT brain data set corresponds to a malignant lesion. The mammogram images have been acquired on a Metrotonica mammography machine (General Electric, USA), and it is a case of a malignant lesion. The segmentation algorithms result for the typical slice corresponding to each data set is depicted here. The Digital Imaging and Communications in Medicine (DICOM) images of size 512 × 512 with slice thickness of 3.6 mm are used in this work. The ethical committee for biomedical activities of Mar Ephraem Centre for Medical Image Processing and Metro Scans, Trivandrum, approved the study of medical images for research work.

First-Generation Algorithms
Thresholding is the basic segmentation algorithm, and it is a non-contextual technique [23]. The improved Otsu thresholding produces robust results with low computational complexity when compared with classical thresholding [90]. Multilevel thresholding based on entropy (Shannon, Renyi, and Tsallis entropies) are more efficient than single-level thresholding [11]. In addition, multilevel thresholding based on particle swarm optimization (PSO) produces superior results compared with Otsu and Kapur's thresholding [40]. The comparative analysis of global thresholding, iterative thresholding, and local thresholding techniques (Niblack, Sauvola) was done on medical images; Niblack technique generates efficient result [33]. The authors of Ref. [78] formulated that the liver parenchyma was extracted from abdomen CT images by adaptive thresholding along with morphological operators to preserve the anatomical structure.
Edge detection algorithms are used to trace the boundary of the ROI, and edges are determined by thresholding the gradient of image. The wavelet transform based edge detection was found to be efficient in the case of images with noise [7,65]. The edge relaxation approach relies on the principle of building up of chains from individual candidate pixels based on the local neighborhood values [95]. Graph searching technique is also employed for extending the edge pixels into chains [77]. Pixel aggregation is the simplest region growing procedure in which growing of region takes place iteratively from a seed pixel that meets some specific homogeneity criteria [81]. The thresholding with region growing improves the accuracy of white matter and gray matter segmentation in the MR images of the brain [1]. The region growing algorithm based on automated seed point selection was used for the segmentation of brain lesions in diffusion-weighted MR images [56]. Also, the automated seed region growing was used for the segmentation of X-ray angiogram and US heart images [46]. The authors of Ref. [89] described the automatic seed point selection based region growing algorithm for the extraction of ROI from cardiac MR images. Muhammad et al. proposed a region growing technique with automatic seed point selection by Harris corner detector, and the result was refined by canny edge detector technique. The proposed algorithm was tested on X-ray, CT, and MR images; superior results were produced when compared with conventional edge detection techniques [58].

Second-Generation Algorithms
The Fuzzy C means (FCM) is a soft clustering algorithm and is better than K-means, which is a hard clustering algorithm. In FCM, the pixel can be grouped into more than one class based on fuzzy membership function value [49,100]. The conventional FCM is sensitive to noise, and its performance can be improved by incorporating the spatial information [38,49]. Gupta S. et al. formulated that FCM has its application in image registration when used along with Speeded-Up Robust Feature (SURF) on MR images of the brain [31]. Ozturk C. et al. proposed that FCM with artificial bee colony (ABC) algorithm generates superior results when compared with genetic algorithm (GA) and particle swarm optimization (PSO) based FCM [63]. Alsmadi [4] verbalized that FCM with firefly optimization algorithm was used for the segmentation of multiple sclerosis lesions in MRI images, and better results were produced when compared with conventional FCM algorithm. The iterative Fuzzy C means (IFCM) was found to be effective when compared with FCM and type II Fuzzy C-means (T2FCM) in the case of noisy images [6,54].
Expectation maximization (EM) is a clustering technique based on the assumption that data follow a Gaussian mixture model. The EM algorithm with local adaptivity (global statistical, local statistical, and geometric features) produces satisfactory results for benchmark and MR images of the brain [45]. In Ref. [72], the different types of EM algorithm for medical image segmentation were analyzed. The ISODATA clustering algorithm is similar to FCM, while the number of clusters is determined by the threshold defined in the splitting and merging procedure [24]. MRF is often associated with clustering algorithms to produce a better result, as its emphasis is on the spatial information of pixels [22]. Although the computational time is high and parameter selection is crucial, it is widely used in MR brain image segmentation with intensity inhomogeneity [59,91]. The fuzzy logic was incorporated in MRF, and the proposed adaptive fuzzy inference model produces robust segmentation for MR brain images [59]. The modified EM algorithm was found to be effective for the accurate segmentation of multimodal images [25].
Active contour is a deformable model in which a set of finite points is placed such that it will evolve into a curve. Kass et al. proposed the classical snake model, and it comprises of internal force from the curve or surface and external force from the image data [39]. In the parametric model, the boundary is traced from the initial set of contour points called active contour model, and non-parametric model is based on the curve evolution called level set or geometric active contour model [5]. The balloon snake model comprises an inflating force that causes the closed curve to move from the initial position, and it can be termed as an additional external force [17]. The snake model based on gradient vector flow (GVF) replaces the external energy term, and it has the ability to trace boundaries of ROI [64]. The T snake is an improved version of regular snake model [55], which performs implicit representation and change parameters after each deformation step. The level set active contour model is an implicit representation of some functions without any restriction, unlike snake model, which does not need to change parameters and is independent of dimensions [93]. Geodesic active contour (GAC) based on prior knowledge locally adapts to enforce smoothness in some parts of the image, while the curve evolution follows image details in other parts [13].
The deformable models were initially used for the extraction of object contour in images and played a prominent role in computer vision [27]. Pedro Morais performed a comparative analysis of deformable models for atrial and aortic tract segmentation on CT images [57]. The deformable models also gain importance in the segmentation of lesions from dermoscopy images [48]. The geometric deformable models produce efficient results for the segmentation of female pelvic organs in MR images [52,53]. The modified GAC, along with shape-guided Chan-Vese (C-V) model, was used for the segmentation of inner and outer boundaries of the bladder wall in MR images [51]. The shape-guided C-V model yields efficient results for the segmentation of levator ani muscles in MR images [50]. The C-V model was also found to be proficient for lumen segmentation of the carotid artery in US images [76]. The 3D adaptive crisp active contour method was proposed for the segmentation of CT lung images; efficient results were produced when compared with 3D region growing, the level-set algorithm based on coherent propagation, and 3D OsiriX toolbox [73].
In the watershed algorithm, gray-level images are considered as a topographic surface, pixel values are considered as heights, and watershed lines mark the boundaries of catchment basins; the image is segmented into desired regions [16]. The deformable models gain their role in speech processing for the shape extraction of vocal tract from MR images [86]. The active shape and active appearance models were found to be efficient for segmentation of hand palms and faces in images for biometrics application [85]. The conventional watershed algorithm suffers from oversegmentation, and it can be controlled by suitable preprocessing algorithms and marker-controlled approach [69,75].

Third-Generation Algorithms
Atlas-based segmentation is used when the standard anatomical template of an organ is there, and it will be used as a reference frame for further segmentation of other CT/MR slices [82]. In atlas-based methods, initially, a rigid registration is performed to register an atlas with each patient volume and a global database is framed [14]. Manual intervention is needed initially to draw the contour on the atlas, and all successive segmentations are fully automatic [44]. The fully automatic brain and cerebellum segmentations are used to determine the brain volume [44].
In Ref. [19], the atlas-based segmentation was performed on MR brain images for the analysis of abnormal lesions using the model of lesion growth. Automated segmentation was proposed for the extraction of incus and malleus ear ossicles in conventional tri-dimensional X-ray computed tomography images; the results were found to be efficient when compared with the manual segmentation result by expert radiologist [60]. The merits of automatic ROI extraction were highlighted in Ref. [61] for the quantification of specific to nonspecific uptake ratio on single photoemission computed tomography (SPECT) images. A detailed survey was done in Ref. [35], which comprises of generation of atlases, o ine learning, atlas selection, label propagation, online learning, label fusion, and post-processing techniques. The atlas-based segmentation with contour model produces an efficient result for the extraction of parotid glands in head and neck CT scans and left atrium in cardiac MR angiography images [87]. The main objective of graph cut algorithm is to perform an optimal cut that separates the object and background. The prior shape model was incorporated in the graph cut model for the segmentation of kidney from abdomen MR images [2]. The graph cut model based on statistical significance with regularization weight generates a notable result for segmentation in liver CT, knee MRI, and lung CT images [12].
A wide number of supervised and unsupervised classifiers are there in image processing for pattern classification and segmentation. The back propagation neural network (BPNN) with feature extraction by gray-level co-occurrence matrix (GLCM) and feature selection by principal component analysis (PCA) were employed for the classification of prostate cancer on MR images [99]. In Ref. [68], the radial basis function neural network (RBFNN) yields good classification accuracy for mammogram images compared with BPNN. The classification accuracy of the neural network was improved by the inclusion of optimization techniques; BPNN with bee colony optimization improves the accuracy in MR brain images [80]. BPNN with multiple feature extraction techniques (first-order statistics, GLCM, Law's texture features, echogenicity) improves the accuracy of liver lesions in US images. The Kth nearest neighbor (KNN) algorithm produces good classification accuracy for Alzheimer disease, when compared with support vector machine (SVM) and naive Bayes (NB) approach [71]. The deep learning neural network (DLNN) was found to produce efficient results when compared with the classical neural networks [29,94]. Rudra et al. in Ref. [74] insisted on improved graph cut algorithm based on the probabilistic information for the modification of edge weights, which produces an efficient result compared with conventional graph cut algorithm. The series ellipses shape prior incorporated accurately delineates the cerebral white matter segmentation in brain MR images [74]. The authors of Ref. [83] proposed an optimal surface graph cut algorithm for the segmentation of carotid artery wall with less user interaction, and the results closely match with the manual delineation. The variants of atlas based segmentation algorithms are discussed in Refs. [3,8,32,88]. Many hybrid segmentation models are there that incorporate multiple segmentation techniques to yield fruitful results, and some of the related works are as follows. Koch et al. [41] were prudent that MRF was visualized in the graph cut model on atlases and target images to generate partially annotated atlas. In Ref. [70], the authors proposed a hybrid model for MR brain images segmentation comprising of Kohonen self-organizing map learning procedure for the Gaussian mixture model and shape model derived from the probabilistic atlas. In addition, the fused information was regularized by max-flow segmentation model. In Ref. [34], the authors made evident the fact that oversegmentation in conventional watershed algorithm was minimized by a partial differential equation based watershed algorithm with the level set model. The regularization of watershed contours takes place, and it does not require any specific markers. It was evident from Ref. [92] that the 3D liver volume was constructed from CT images of the abdomen by the super voxel-based graph cut algorithm. Yongqian and Liang [96] projected that the classifier based region growing approach with seed point selection by SVM produces good segmentation result for CT lung images.
The authors of Ref. [15] proposed entropy-based adaptive multi-thresholding for the segmentation of liver tumor from abdomen CT images; the initial liver contour extraction was done by a watershed algorithm. The hybrid segmentation model comprising of BPNN and Lankton active contour model produces efficient results for segmentation of liver and kidney and its anomalies like cyst and tumor on abdomen CT images [42]. A hybrid approach comprising of K-means clustering and active contour was used for lumen region segmentation in intravascular US images of the coronary artery [37]. The FCM was coupled with active contour model for bladder segmentation in CT images [67]. The intensity, spatial information, and shape feature are utilized for the segmentation of vertebral bodies in CT images [9]. The shape prior along with intensity features was used in the level set segmentation model for the refinement of results [10].

Results and Discussions
The Matlab 2010 software was used for the analysis of typical segmentation algorithms, and the results of typical slices from medical datasets are depicted here. The algorithms are tested on the laptop with 2.10-Ghz Intel Pentium processor with 2-GB RAM. The following are the inferences that can be made from the literature survey of segmentation algorithms for medical images.
The thresholding technique gives rise to serious error when an image comprises of various segments with different gray-level intensity. In the image acquisition process, noise is induced, which in turn causes intensity variation in an image, and thresholding techniques are affected by noise and images of poor contrast. In spite of the issues, the simplicity and less computation time in the thresholding algorithm make it easy for hardware implementation. The boundary leakage problem arises in the case of ROI with weak boundaries. The undesirable effects of simple region growing techniques were overcome by region growing with multiple seed points, and a satisfactory result was obtained for noisy images. The region growing approach exploits the spatial features much, and hence, better results are produced than thresholding, especially for ROI with well-structured boundaries (e.g. lungs, bone structures). The edge detectors produce false edges and discontinuity in edge boundaries due to the variation in gradient strength. The Canny edge detector is robust to noise, as it incorporates the Gaussian smoothing prior to edge pixels detection. The thresholding algorithm response is improved by choosing multiple thresholds based on optimization algorithms and by the inclusion of spatial information. The automatic seed point selection by optimization techniques, homotopic region growing for topology preservation, and artificial intelligence based seed point selection alleviate the issues of conventional region growing algorithm.
Unlike the supervised neural networks, the clustering techniques do not require any training. The conventional FCM is sensitive to noise; however, spatial information is incorporated in improved clustering techniques for better results. Many parameters like number of clusters, iteration, and error threshold have to be predetermined, and computation time is also high. The deformable snake models and its variants are sensitive to noise and spurious edges, and computation time is high because of iterative optimization. The level set models have good spatial continuity, are less sensitive to noise, and are efficient in the extraction of complex ROI shape; however, appropriate parameters selection and manual intervention are essential. Oversegmentation is a problem in the conventional watershed algorithm, and it was overcome by appropriate pre-processing and post-processing approaches. The MRF well considers the spatial relationship and is less sensitive to intensity inhomogeneity and noise. Parameter selection is crucial in MRF, and computation complexity is solved by iterative MRF.
The accuracy of atlas-based segmentation is good; however, expert opinion and prior anatomical knowledge are needed. The high computation time and difficulty for the segmentation of ROI with complex and variable shape are the discrepancies. Noise immunity is good and is generally preferred when ROI to be segmented is stable in the population of study. Spatial information is much considered in graph cut algorithm; hence, region continuity is efficient, and noise immunity is fair. Although graph cut algorithm has many good features, parameter selection is crucial. The computation time is high for large-size images and especially when multiple regions have to be segmented. The neural network plays a vital role in solving complex and nonlinear problems. Depending upon the system memory architecture, it can handle large data sets; however, the spatial modeling is poor. The noise immunity is high; however, the training process consumes time. The response depends on the data involved in the training, and a wide range of data sets is required in realtime scenario for the anatomical organs and anomalies delineation. In graph cut algorithms, the seed point selection and weight assignment are crucial, but it maintains good spatial relationship and region continuity is good. The computation time is high when the size of the image is large as well as for the segmentation of multiple ROI.
Thresholding is a basic segmentation algorithm, and the variants of thresholding algorithm are analyzed in this paper. Wellner's adaptive thresholding results corresponding to the inputs in Figure 2A-D are depicted in Figure 2E-H, and the parameters of this algorithms are filter size, threshold value, filter type, and threshold mode. The filter type for all the inputs chosen was Gaussian, with a kernel size of 15. The maximum entropy thresholding results are depicted in Figure 2I-L. The local statistics thresholding results are depicted in Figure 2M-P.
The results of region growing algorithm with manual seed point selection are depicted in Figure 3, and (x, y) represents seed point for the region growing technique. The threshold value of T = 0.2 is chosen for all the inputs, and the seed point coordinates will be a changing parameter, and it depends on the region of interest. The region growing process terminates when the intensity difference between the region mean and new pixel becomes greater than the threshold. Out of the conventional edge detectors, Canny algorithm generates efficient results. The Canny edge detection algorithm results are depicted in Figure 4. The Gauss gradient edge detector results are depicted here, and the parameter sigma plays a crucial role. For all the input images, sigma = 1.5 was chosen [43]. Boundary extraction is better in Gauss gradient edge detector technique. The gradient-based watershed algorithm results are depicted in Figure 5. Here, prior to segmentation, pre-processing was done by the median filter of kernel size 3 × 3.
The parameters of the FCM algorithm are cluster fuzziness value (f = 2), a number of clusters (c = 3), and stopping criterion (ε = 0.001). Cost function minimization takes place by comparing the changes in membership function or the cluster center at two consecutive iterations. Cost function minimization takes place when the change in consecutive values of the fuzzy membership function is less than the stopping criterion (ε = 0.001). The parameters of the Fuzzy local information C-means (FLICM) algorithm are number of clusters (cnum = 4), weighting fuzzy exponent membership (m = 2), window size (w = 3), maximum iteration   (i max = 500), and threshold value (th = 0.001). In the FLICM algorithm, the fuzzy factor has to be determined in each iteration and the computational complexity is high. The FCM and FLICM segmentation results are depicted in Figure 6. The Lankton algorithm evolves the initial contour based on the local neighborhood statistics and can segment the image into two homogeneous regions [42], and the results are depicted in Figure 7. In graph cut algorithm, the objective is to extract the object from the background by performing an optimum cut, and the results are depicted in Figure 8. Table 1 depicts the segmentation algorithms ranking for the characteristics spatial information, region continuity noise immunity, selection of parameters, the complexity of algorithms, computation time, and accuracy.     1  2  1  3  2  2  2  3  3  3  Region continuity  2  2  1  3  3  2  3  3  3  3  Noise immunity  1  2  2  2  2  1  2  2  1  2  Parameters selection  2  1  3  1  2  2  1  1  1  1  Complexity  3  2  3  1  2  2  1  2  1  1  Computation time  3  2  3  2  2  2  1  1  1  2  Accuracy  1  2  1  3  3  2  3  3  3  3  Overall score  13  13  14  15  16  13  13 15 13 15 The comparative analyses of various segmentation algorithms (thresholding -I, region based -II, edge based -III, active contour -IV, clustering -V, watershed -VI, neural network -VII, graph cut -VIII, atlas-based -XI, MRF -X) are made based on the characteristics as follows.
Characteristics are ranked by giving numerical values for ease of representation. The overall score of the segmentation algorithms is depicted below. The plot in Figure 9 depicts the percentage in terms of overall score based on the characteristics made from qualitative and quantitative analysis.
a. Spatial information: less considered -1, moderately considered -2, highly considered -3 b. Region continuity: poor -1, fair -2, good -3 c.  The plot reveals that researchers are having wide research areas in segmentation to improve the overall efficiency of algorithms. The entropy-based segmentation metric was proposed by Zhang et al. [97], and it works well when the ground truth image is difficult to generate. The idea behind this metric is to maximize the uniformity of pixels within the segmented region and maximize uniformity across regions. The entropy of each region (S j ) is determined as follows: where G j represents the possible gray scale values in the region j and N j (n) represents the count of pixels belonging to region j with gray scale n.
The entropy measure of the image I is expressed as follows: The entropy values for thresholding techniques, region growing, clustering, and watershed algorithms are depicted in Figure 10. Data sets ID1 to ID4 represent the input images corresponding to row 1 images in Figure 2. The lower the entropy, the better is the quality of segmentation algorithm. The plot reveals that based on the medical imaging modality such as CT, MRI, and mammogram, the performance of the algorithm changes. The choice of segmentation technique depends upon the medical imaging modality. The maximum entropy thresholding technique was found to have the lowest entropy in three data sets corresponding to ID1, ID2, and ID3. The Lankton active contour model was evaluated by Dice coefficient (DC).
The DC values corresponding to the results in Figure 7 are 0.92, 0.90, 0.94, and 0.95. For graph cut segmentation, the DC values corresponding to the results in Figure 8 are 0.94 and 0.93. The following inferences are also made from the review of segmentation models: the noise immunity can be improved by the proper choice of filtering approach based on medical imaging modality, parameter selection is vital in segmentation, and optimization algorithm can be employed for improving the results. The computation complexity can be relaxed by using graphical processing unit (GPU) utilizing parallel processing. This work emphasizes the need for hybrid segmentation model for improving the accuracy and incorporation of optimization technique for tuning of parameters.

Conclusion
This work addresses some of the typical segmentation algorithms for medical images. Segmentation plays a vital role in telemedicine applications for the analysis of the region of interest. The algorithms are grouped into three generations, and the rank is assigned based on the comparative analysis of their features. The algorithms are tested on real-time data sets, and this work will be a guidance for the researchers in medical image segmentation. The hybrid segmentation algorithms are gaining much importance, as they produce superior results by overwhelming the limitations of conventional algorithms. A segmentation technique will not yield efficient result for all medical imaging modalities; based on the image modality and region of interest, appropriate algorithm has to be chosen. The outcome of this work will be an aid for the identification of diseases like cyst and tumor in medical images. Our future work will be a detailed analysis of hybrid segmentation models for the delineation of anatomical organs and anomalies. The lagging characteristics in one generation algorithm will be overcome by another generation algorithm, and hence, a proper coupling of segmentation approaches will lead to an efficient extraction of ROI.