Abstract
To find an optimal balance between accuracy and model transparency in fuzzy systems modeling, a novel approach based on the non-dominated sorting fireworks optimization algorithm (NSFOA) is proposed in this paper. The FOA is a new swarm intelligence algorithm mimicking the explosion of fireworks, which could be used to tune the parameters and optimize the structure of fuzzy systems with high effectiveness and correctness. At each iteration of the NSFOA algorithm, all individuals are sorted based on fast non-dominated sorting and crowding-distance evaluation, and the multiobjective fitness functions consider the aspects of both interpretability and accuracy contemporaneously. Subsequently, the individuals with better transparency and accuracy are selected for the next generation. The proposed NSFOA algorithm is applied to the second-order non-linear system and the Iris benchmark problem, and the experimental results demonstrate that our method leads to comprehensible fuzzy models with remarkable accuracy.
1 Introduction
Fuzzy systems describe real-world systems or natural phenomena by using intelligible language, which make human beings learn the systems easily [2]. In various realms, such as data mining and pattern recognition, people prefer to adopt fuzzy systems because of their capability of solving uncertain or incomplete problems with expertise knowledge. In the design of fuzzy rule-based models, accuracy is another vital index. To represent a real problem faithfully, the system has to be accurate enough. However, interpretability and accuracy are two contradictory goals. High accuracy is always obtained at the cost of the detriment of system comprehensibility and vice versa.
In recent years, some researchers have already focused on the construction of interpretable and accurate fuzzy systems [3, 13]. Some studies have tried to transform the multiple objectives to a single fitness function with different weighting factors [10]. Cpalka et al. [7] proposed the method of neuro-fuzzy systems structure combined with evolutionary algorithms to obtain an accurate and interpretable system. Zhang et al. [17] presented the Pareto multiobjective cooperative co-evolutionary algorithm to generate fuzzy systems from data. Multiobjective evolutionary algorithm has been considered as a promising approach to solve multiobjective problems, thanks to its ability to find a set of compromise solutions in a single run.
Fireworks algorithm (FA) is a relatively developed global optimization method mimicking the process of fireworks explosion [18, 20]. In order to ensure the diversity and balance the local and global search, the explosion amplitude and the population size of each firework are different. Fireworks with better fitness have a smaller explosion amplitude and a larger number of explosion sparks than the fireworks with lower fitness, so the convergence speed is accelerated. On the contrary, fireworks with lower fitness generate less sparks in a larger range to avoid the local optimum. Imran et al. [9] used the FA to reconfigure and allocate optimally distributed generation placement in power distribution networks. In Ref. [19], Zheng et al. applied the fireworks optimization algorithm (FOA) to multiobjective oil crop production, and the result demonstrated that the advantage of FOA was remarkable when dealing with multiobjective problems.
To find a high degree of transparency and accuracy in fuzzy modeling, a novel method based on non-dominated sorting FOA (NSFOA) is proposed in this paper. Different from the classical weighted sum method, the proposed NSFOA technique is combined with an estimation operation to derive a diverse set of solutions that are converged to be the neighborhood of true Pareto-optimal set in a single run. The fuzzy C-means clustering method (FCM) is adopted to establish the initial system, and the proposed NSFOA is applied to optimize the fuzzy system and the fitness functions consider the accuracy index and interpretability index simultaneously. At each iteration of the algorithm, a simplification method [13] using similarity measure is applied to reduce the redundancy and complexity of fuzzy rules and fuzzy sets; thus, the transparency of the fuzzy system is improved.
The paper is structured as follows. Section 2 briefly introduces the preliminaries including the fuzzy models, accuracy, and interpretability issues in fuzzy systems. In Section 3, the generation of fuzzy systems by the method of NSFOA is shown in detail. Section 4 describes the experimental results, and conclusions are given in Section 5.
2 Preliminaries
2.1 Fuzzy Models
Fuzzy models describe the real problems by human-comprehensible rules in the form of if–then. Depending on the consequent of fuzzy rules, the rule base could be divided into two types:
Takagi–Sugeno (T–S) fuzzy model
Assume that X= (x1, x2, …, xn) is a generic input vector of the system P(X, Y); the typical if–then rule of the T–S fuzzy system [5] is expressed as
where Ri indicates the ith fuzzy rule, i= 1, 2, …, m;
Fuzzy rule-based classifiers
Considering an n-dimensional classification problem with N patterns x ∈ X ⊆ Rn, x= (x1, x2, …, xn) from M classes {g1, g2, …, gM}, the typical if–then rule of the fuzzy classification system [17] is indicated as
where
The strategy of winer-take-all [17] is applied to calculate the outputs of the fuzzy classification system. Moreover, the label of class for a certain pattern is determined by the consequent of the rule that is maximally activated.
2.2 Accuracy and Interpretability Issues in Fuzzy Systems
An ideal fuzzy system should not most possibly approximate real problems only, but also should be easily understandable to the users. Some measures have been widely accepted to assess the degree of accuracy. The mean square error (MSE) between the actual system outputs and the predicted values is generally used to calculate the accuracy of a certain TS fuzzy system [1]. Moreover, for a fuzzy classification system, accuracy is usually assessed by the number of training patterns that are mistakenly classified.
Unlike the property of accuracy, there is no formal definition of interpretability in the scientific papers [6]. In general, the characteristics of linguistic variables, fuzzy rules, and fuzzy sets are believed to be essential to system complexity. They are summarized as follows [4, 13, 17].
The number of features or variables. The reduction of variables set can improve the comprehensibility of the knowledge base. In the fuzzy systems with high dimensions, the size of rules could be exponentially increased as more inputs are added.
The number of rules. A fuzzy model with complex rule base is difficult to be interpreted and analyzed. To improve the model simplicity, the number of fuzzy rules should be as small as possible under the condition that the model accuracy is kept at a high level.
The number of membership functions. The number of membership functions defined by each variable has great importance to the system complexity. Actually, the number of membership functions per variable should be less than nine because of human information processing capability [1].
3 NSFOA for Generating Fuzzy Systems
In this section, we introduce the generation of highly accurate and highly interpretable fuzzy systems by the method of the NSFOA algorithm in detail. The FCM algorithm is adopted to establish the initial system, and the proposed NSFOA algorithm is applied to optimize fuzzy system accuracy and complexity. At each iteration, a simplification method is adopted to reduce the redundancy and complexity of fuzzy rules and fuzzy sets, and thus the transparency of the fuzzy system is increased. First, we briefly introduce the base FOA.
3.1 The Basic FOA
Considering p fireworks with D dimensions, the number of sparks si and the amplitude of explosion Ai for each firework xi are calculated as follows [18]:
where m and
To avoid overwhelming effects at the good locations, bounds are defined for si.
On the basis of the explosion amplitude Ai and the spark size si of firework xi, the location of each spark xj is generated by Eq. (6). In addition, for a few specific sparks, the Gaussian mutation operator is applied by Eq. (7) to keep the diversity of the population.
If the new sparks generated by both ways fall out of the bounds, they should be mapped to the search space as follows:
where % denotes the modulo operator for floating-point numbers.
After the explosion, the best location among all the newborn sparks and fireworks is kept for the next generation, and the other p − 1 fireworks for the next generation are selected based on the probabilities that are proportional to their distance to other locations [18].
3.2 Initial Fuzzy System Generation
The FCM [12] is used to form the rule antecedents of the original fuzzy system, and the rule consequents of the initial system are obtained by the least square method for the type of T–S fuzzy system. To identify the rule consequents of the fuzzy classification system, the following function is defined:
where uik denotes the belongingness of the data k to cluster i, and fj(xk) is formulated as
Then, the consequent of rule i can be determined by
where Nc denotes the number of classes of the certain data set.
3.3 Coding and Initial Population Generation
As explained previously, the antecedents of the initial fuzzy system obtained by FCM algorithm could be decomposed into the first individual of the initial population. The first individual is coded by a sequence of genes denoting the parameters of the initial rule antecedent:
Given the search space [Hmin, Hmax]:
where
3.4 Multiobjective Functions
Two main requirements should be considered in the process of fuzzy modeling: the maximization of accuracy and the minimization of complexity. According to the analysis in Section 2, the accuracy can be accessed by the MSE between the real system outputs and the predicted values for the type of fuzzy T–S system. For the fuzzy classification system, the accuracy is accessed by the size of training patterns that are mistakenly classified. For both type of fuzzy systems, the numbers of fuzzy rules and fuzzy sets are the objectives can be used to quantify system interpretability.
Accuracy: For the fuzzy T–S system, the accuracy is quantified by the method of well-known MSE:
where N is the data set size, yi is the predicted output, and
For the fuzzy classification system, the accuracy is quantified by the number of patterns that are mistakenly classified:
where ek is defined by
Interpretability: Interpretability of the fuzzy system is quantified by
where R is the number of rules and FS is the number of fuzzy sets.
To design interpretability–accuracy trade-off systems, three objectives should satisfy the following formulations:
3.5 Individual Selection for the Next Generation
After the generation of sparks, the better individuals are selected as fireworks for the next generation. On the basis of the method of fast and elitist non-dominated sorting genetic algorithm (NSGA-II) [8], non-dominated sorting is applied to divide the population into different non-domination levels and crowding-distance evaluation is performed to estimate the density of solutions in the population. Based on the two properties – non-domination rank and crowding distance – the crowded-comparison operator is performed to sort two arbitrary individuals in the population. That is, between two individuals with two different non-domination ranks, we prefer the one with the lower (better) rank. Otherwise, we prefer the individual with the higher crowding distance. After sorting all the current individuals, the first solution is selected as the firework for the next generation, and other p − 1 solutions are randomly selected from the top 2p individuals.
3.6 NSFOA Algorithm for Generating Fuzzy Systems
Now we present the algorithmic framework of fuzzy model construction based on NSFOA as follows:
Initialization.
The FCM algorithm is performed to obtain the antecedents of the initial fuzzy system, which are decomposed into the first individual of the initial population P with p solutions. Other p − 1 individuals are generated randomly around the first individual within the search space.
The consequence of the initial fuzzy system is calculated by the least square method (T–S fuzzy system) or the strategy of winner-take-all (fuzzy classification system).
Iterative improvement.
Simplify the fuzzy sets and rules based on the simplification method and update the population P.
For each individual xi in P, do:
Calculate the consequence of the fuzzy models by the least square method (T–S fuzzy system) or the strategy of winner-take-all (fuzzy classification system).
Calculate si for xi according to Eqs. (3) and (5).
Calculate Ai for xi according to Eq. (4).
Generate si sparks of xi according to Eqs. (6) and (8).
Generate three specific sparks of xi according to Eqs. (7) and (8).
Sort the obtained solutions (all the current sparks and fireworks) with the defined objective functions based on the two properties: non-domination rank and crowding distance.
Select the first individual as the firework for next generation, and other p − 1 individuals are selected randomly from the top 2p individuals.
If the stop criterion is not met, go to step 2.1; otherwise, end.
The entire algorithm flow of the fuzzy model construction is described in Figure 1.
4 Experiments
In this section, the experiments on the second-order non-linear system and the Iris benchmark classification problem are conducted to investigate the performance of the proposed method: NSFOA. Table 1 gives the control parameters of NSFOA.
Parameters | Values |
---|---|
Maximum iterations | 200 |
Population size | 40 |
Threshold of merging neighboring fuzzy sets | 0.4 |
Threshold of merging fuzzy sets similar to the universal set | 0.9 |
Threshold of merging fuzzy rules | 1 |
4.1 The Second-Order Non-linear System
The second-order non-linear plant is described as
where
The construction of a T–S fuzzy model is aimed to approximate the non-linear component g(y(k − 1), y(k − 2)) of the plant. Here, 400 simulated data points are generated by Eqs. (21) and (22). Two hundred training data are obtained with a random input signal u(k) uniformly distributed in the interval [–1.5, 1.5]. The rest of the 200 testing data are obtained by using a sinusoidal input signal u(k)= sin(2πk/25).
The fuzzy systems are generated from the 200 training data, and the standard MSE performance index is used to evaluate their property. First, the initial fuzzy systems are constructed by the FCM algorithm combined with the least square method, and the number of initial fuzzy clusters is five. To improve the accuracy and interpretability of the initial systems, the NSFOA algorithm is used to learn and optimize their structure and parameters, and the simplification method based on the similarity measure is applied to decrease the complexity of the fuzzy model. The final model obtained by the NSFOA algorithm consists of four rules and four fuzzy sets, and the model accuracy keeps at a high level, MSE= 2.7965e-4 and 5.4447e-5 for training and testing data, respectively. The structure and parameters of the optimized fuzzy system are listed in Table 2. The membership functions of the initial model and the optimized model, and the comparison between optimized model output and real output are shown in Figure 2.
Rule base of fuzzy model |
R1: If y(k − 1) is small and y(k − 2) is small, then g(k)= 0.3504y(k − 1) + 0.0961y(k − 2) − 0.2606 |
R2: If y(k − 1) is small and y(k − 2) is big, then g(k)= –0.3215y(k − 1) + 0.1187y(k − 2) + 0.2245 |
R3: If y(k − 1) is big and y(k − 2) is small, then g(k)= –0.6443y(k − 1) − 0.0493y(k − 2) + 0.3748 |
R4: If y(k − 1) is big and y(k − 2) is big, then g(k)= 0.5928y(k − 1) − 0.0381y(k − 2) − 3430 |
Antecedent parameters |
y(k − 1): small= [−0.9241 1.1681], big= [0.8264 0.8491] |
y(k − 2): small= [−0.8673 0.7554], big= [0.9465 0.8729] |
The performance of the proposed fuzzy model is compared with those that have already been existing, and the comparative results are stated in Table 3. According to the comparative results, it can be seen that the fuzzy model in this paper is better than the models in Refs. [15] and [16] in the aspects of both accuracy and interpretability. In Ref. [13], the fuzzy models performed a little better in the aspect of system complexity, while the accuracy is worse than the optimized model in this paper. Overall, the proposed algorithm gains appropriate balance between precision and interpretability in fuzzy modeling.
Ref. | No. of rules | No. of sets | MES(Tr) | MES(Te) |
---|---|---|---|---|
[13] | 4 | 3 | 5.4611e-4 | 5.4360e-4 |
[15] | 28 | 28 | 3.3299e-4 | 5.9595e-4 |
[16] | 20 | 20 | 6.8341e-4 | 2.3836e-4 |
This paper | 4 | 4 | 5.4447e-5 | 7.9650e-5 |
4.2 Iris
The Iris database contains 150 samples from three species, and each data point consists of four features. First, feature selection is performed to decrease the dimension of the system, and the selected features are petal length and petal width, which are defined as x3 and x4, respectively. Then, the FCM algorithm is applied to identify the initial fuzzy classification system with 9 fuzzy rules and 18 fuzzy sets, and the number of training patterns that are mistakenly classified is 10. To improve the performance of the initial fuzzy system, the NSFOA algorithm is used to optimize it. The final system consists of five fuzzy sets and three rules, and the number of training patterns that are mistakenly classified is four. The structure and parameters of the optimized fuzzy model are stated in Table 4, and the distribution of membership functions is shown in Figure 3.
Rule base of the fuzzy model |
R1: If x3 is small, then x belongs to class 1 |
R2: If x3 is medium and x4 is small, then x belongs to class 2 |
R3: If x3 is big and x4 is large, then x belongs to class 3 |
Parameters of antecedents |
x3: small= [1.7619 0.8714], medium= [4.1145 1.0721], big= [5.5795 0.8714] |
x4: small= [1.0777 0.4467], big= [1.8796 0.3631] |
According to the result of comparison with other existing models in Table 5, the fuzzy model in Ref. [11] owns the best accuracy; however, the system is too complex. In Ref. [17], the fuzzy model is a little simpler than the model obtained by the proposed NSFOA algorithm; however, its objective property of accuracy is a little worse. The model in Ref. [14] has almost the same accuracy index, but the complexity of the fuzzy model is much higher compared with the model obtained in this paper. From the comparative results stated in Table 5, it is clear that the proposed algorithm gains a good trade-off between accuracy and model simplicity in fuzzy modeling compared with the existing fuzzy models.
Ref. | No. of fuzzy rules | No. of fuzzy sets | Accuracy |
---|---|---|---|
[17] | 3 | 4 | 96.67% |
[11] | 5 | 18 | 100% |
[14] | 3 | 11 | 97.5% |
This paper | 3 | 5 | 97.3% |
5 Conclusion
The FOA is a promising global optimization method that could be effective in multiobjective optimization. In this paper, an efficient NSFOA algorithm is proposed to construct an accurate and compact fuzzy system. This algorithm uses the FCM clustering method for generating the initial model. The FOA is used to learn and tune the model structure. On the basis of the properties of non-domination rank and crowding distance, the individuals are evaluated and selected for the next generation. In addition, this research is a novel application of FOA in fuzzy modeling. Experimental results show that the NSFOA algorithm could be applied to the construction of accurate and parsimonious fuzzy models with fast convergence rate. It is mainly due to the capability to balance between exploration and exploitation. The fireworks at good locations generate more sparks in smaller areas in order to find the optimal solution quickly. The fireworks at bad locations generate fewer sparks in larger areas to search promising good solutions. Furthermore, the Gaussian mutation operation increases the diversity of population to escape from local optimization. Future research will concern further improvement of the performance and reducing the burdens of computation of the proposed algorithm.
Acknowledgments
This work was funded by the National Natural Science Foundations of China (61473266).
Bibliography
[1] M. Antonelli, P. Ducange, B. Lazzerini and F. Marcelloni, Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity, Soft Comput.15 (2011), 2335–2354.10.1007/s00500-010-0665-0Search in Google Scholar
[2] R. Babuška, Construction of fuzzy systems – interplay between precision and transparency, Proc. ESIT2000 (2000), 445–452.Search in Google Scholar
[3] J. Casillas, ed., Accuracy Improvements in Linguistic Fuzzy Modeling, vol. 129, Springer, Berlin, 2003.10.1007/978-3-540-37058-1Search in Google Scholar
[4] M. Y. Chen and D. A. Linkens, Rule-base self-generation and simplification for data-driven fuzzy models, in: The 10th IEEE International Conference on Fuzzy Systems, 2001, pp. 424–427, vol. 1, IEEE, 2001.Search in Google Scholar
[5] M. Cococcioni, B. Lazzerini and F. Marcelloni, On reducing computational overhead in multi-objective genetic Takagi–Sugeno fuzzy systems, Appl. Soft Comput.11 (2011), 675–688.10.1016/j.asoc.2009.12.028Search in Google Scholar
[6] O. Cordón, A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: designing interpretable genetic fuzzy systems, Int. J. Approx. Reason.52 (2011), 894–913.10.1016/j.ijar.2011.03.004Search in Google Scholar
[7] K. Cpalka, K. Lapa, A. Przybyl and M. Zalasinski, A new method for designing neuro-fuzzy systems for nonlinear modelling with interpretability aspects, Neurocomputing135 (2014), 203–217.10.1016/j.neucom.2013.12.031Search in Google Scholar
[8] K. Deb, A. Pratap, S. Agarwal and T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Trans. Evol. Comput.6 (2002), 182–197.10.1109/4235.996017Search in Google Scholar
[9] A. M. Imran, M. Kowsalya and D. P. Kothari, A novel integration technique for optimal network reconfiguration and distributed generation placement in power distribution networks, Int. J. Elect. Power Energy Syst.63 (2014), 461–472.10.1016/j.ijepes.2014.06.011Search in Google Scholar
[10] L. S. Oliveira, M. Morita, R. Sabourin and F. Bortolozzi, Multi-objective genetic algorithms to create ensemble of classifiers, in: Evolutionary Multi-criterion Optimization, pp. 592–606, Springer, Berlin, 2005.10.1007/978-3-540-31880-4_41Search in Google Scholar
[11] M. Russo, Genetic fuzzy learning, IEEE Trans. Evol. Comput.4 (2000), 259–273.10.1109/4235.873236Search in Google Scholar
[12] M. Taherdangkoo and M. H. Bagheri, A powerful hybrid clustering method based on modified stem cells and fuzzy C-means algorithms, Eng. Appl. Artif. Intell.26 (2013), 1493–1502.10.1016/j.engappai.2013.03.002Search in Google Scholar
[13] H. L. Wang, S. Kwong, Y. C. Jin, W. Wei and K. F. Man, Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction, Fuzzy Sets Syst.149 (2005), 149–186.10.1016/j.fss.2004.07.013Search in Google Scholar
[14] J. S. Wang and C. S. G. Lee, Self-adaptive neuro-fuzzy inference systems for classification applications, IEEE Trans. Fuzzy Syst.10 (2002), 790–802.10.1109/TFUZZ.2002.805880Search in Google Scholar
[15] L. Wang and J. Yen, Extracting fuzzy rules for system modeling using a hybrid of genetic algorithms and Kalman filter, Fuzzy Sets Syst.101 (1999), 353–362.10.1016/S0165-0114(97)00098-5Search in Google Scholar
[16] J. Yen and L. Wang, Simplifying fuzzy rule-based models using orthogonal transformation methods, IEEE Trans. Syst. Man Cybernet. Pt B Cybernet.29 (1999), 13–24.10.1109/3477.740162Search in Google Scholar PubMed
[17] Y. Zhang, X. B. Wu, Z. Y. Xing and W. L. Hu, On generating interpretable and precise fuzzy systems based on Pareto multi-objective cooperative co-evolutionary algorithm, Appl. Soft Comput.11 (2011), 1284–1294.10.1016/j.asoc.2010.03.005Search in Google Scholar
[18] S. Q. Zheng, A. Janecek and Y. Tan, Enhanced fireworks algorithm, in: 2013 IEEE Congress on Evolutionary Computation, pp. 2069–2077, Cancun, 2013.Search in Google Scholar
[19] Y. J. Zheng, Q. Song and S. Y. Chen, Multiobjective fireworks optimization for variable-rate fertilization in oil crop production, Appl. Soft Comput.13 (2013), 4253–4263.10.1016/j.asoc.2013.07.004Search in Google Scholar
[20] Y. J. Zheng, X. L. Xu, H. F. Ling and S. Y. Chen, A hybrid fireworks optimization method with differential evolution operators, Neurocomputing148 (2015), 75–82.10.1016/j.neucom.2012.08.075Search in Google Scholar
©2016 by De Gruyter
This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.