Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Intelligent Systems

Editor-in-Chief: Fleyeh, Hasan


CiteScore 2018: 1.03

SCImago Journal Rank (SJR) 2018: 0.188
Source Normalized Impact per Paper (SNIP) 2018: 0.533

Online
ISSN
2191-026X
See all formats and pricing
More options …
Volume 25, Issue 2

Issues

Neuro-Fuzzy Modeling for Multi-Objective Test Suite Optimization

Zeeshan Anwar
  • Corresponding author
  • Department of Electrical and Computer Engineering, Center for Advanced Studies in Engineering, 44000 Islamabad, Pakistan
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Ali Ahsan
  • Department of Engineering Management, Center for Advanced Studies in Engineering, 44000 Islamabad, Pakistan
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Cagatay Catal
Published Online: 2015-05-05 | DOI: https://doi.org/10.1515/jisys-2014-0152

Abstract

Regression testing is a type of testing activity, which ensures that source code changes do not affect the unmodified portions of the software adversely. This testing activity may be very expensive in, some cases, due to the required time to execute the test suite. In order to execute the regression tests in a cost-effective manner, the optimization of regression test suite is crucial. This optimization can be achieved by applying test suite reduction (TSR), regression test selection (RTS), or test case prioritization (TCP) techniques. In this paper, we designed and implemented an expert system for TSR problem by using neuro-fuzzy modeling-based approaches known as “adaptive neuro-fuzzy inference system with grid partitioning” (ANFIS-GP) and “adaptive neuro-fuzzy inference system with subtractive clustering” (ANFIS-SC). Two case studies were performed to validate the model and fuzzy logic, multi-objective genetic algorithms (MOGAs), non-dominated sorting genetic algorithm (NSGA-II) and multi-objective particle swarm optimization (MOPSO) algorithms were used for benchmarking. The performance of the models were evaluated in terms of reduction of test suite size, reduction in fault detection rate, reduction in test suite execution time, and reduction in requirement coverage. The experimental results showed that our ANFIS-based optimization system is very effective to optimize the regression test suite and provides better performance than the other approaches evaluated in this study. Size and execution time of the test suite is reduced up to 50%, whereas loss in fault detection rate is between 0% and 25%.

Keywords: Regression testing; test suite optimization; neuro-fuzzy modeling; computational intelligence

Bibliography

  • [1]

    K. K. Aggarwal and Y. Singh, A book on software engineering, New Age International (P) Ltd.; Publishers, 4835/24, Ansari Road, Daryaganj, New Delhi, 2001.Google Scholar

  • [2]

    A. Ajith, Neuro fuzzy systems: state-of-the-art modeling techniques. Connectionist models of neurons, learning processes, and artificial intelligence, pp. 269–276, Springer, Berlin Heidelberg, 2001.Google Scholar

  • [3]

    A. M. Alakeel, A fuzzy test cases prioritization technique for regression testing programs with assertions, in: ADVCOMP 2012: The Sixth International Conference on Advanced Engineering Computing and Applications in Sciences, pp. 78–82.Google Scholar

  • [4]

    A. Al-Hmouz, J. S., R. Al-Hmouz and J. Yan, Modeling and simulation of an adaptive neuro-fuzzy inference system (ANFIS) for mobile learning, IEEE Trans. Learn. Technol. 5 (2012), 226–237.Google Scholar

  • [5]

    Z. Anwar, Neuro-fuzzy modeling based regression test suite optimization, MSc. Dissertation, Center for Advanced Studies in Engineering (CASE), 2013.Google Scholar

  • [6]

    Z. Anwar and A. Ahsan, Multi-objective regression test suite optimization with fuzzy logic, in: 16th International Multi-Topic Conference (INMIC-2013), pp. 95–100, December 2013.Google Scholar

  • [7]

    Z. Anwar and A. Ahsan, Comparative analysis of MOGA, NSGA-II and MOPSO for regression test suite optimization, Int. J. Softw. Eng. (IJSE) 7 (2014), 41–56.Google Scholar

  • [8]

    Z. Anwar and A. Ahsan, Exploration and analysis of regression test suite optimization, ACM SIGSOFT Softw. Eng. Notes 39 (2014).Google Scholar

  • [9]

    E. Ashraf, A. Rauf and K. Mahmood, Value based regression test case prioritization, in: Proceedings of the World Congress on Engineering and Computer Science, Vol. 1, 2012.Google Scholar

  • [10]

    W. K. G. Assunção, T. E. Colanzi, S. R. Vergilio and A. Pozo, A multi-objective optimization approach for the integration and test order problem, Inf. Sci. 267 (2014), 119–139.Google Scholar

  • [11]

    A. T. Azar, Adaptive neuro-fuzzy systems, fuzzy systems, ISBN 978-953-7619-92-3, pp. 216, INTECH, Croatia, February 2010.Google Scholar

  • [12]

    R. Carlson, H. Do and A. Denton, A clustering approach to improving test case prioritization: an industrial case study, in: Proceedings of IEEE International Conference on Software Maintenance (ICSM), September 2011.Google Scholar

  • [13]

    C. A. Coello, G. T. Pulido and M. Salazar Lechuga, Handling multiple objectives with particle swarm optimization, IEEE T. Evol. Comput. 8 (2004), 256–279.Google Scholar

  • [14]

    A. da Silva Simão and L. J. Senger, A technique to reduce the test case suites for regression testing based on a self-organizing neural network architecture, in: Proceedings of the 30th Annual International Computer Software and Application Conference (COMPSAC’06), 2006.Google Scholar

  • [15]

    K. Deb, S. Agrawal, A. Pratab and T. Meyarivan, A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II, in: Proceedings of the Parallel Problem Solving from Nature VI Conference, Lecture Notes in Computer Science No. 1917, pp. 849–858, Springer, Paris, France, 2000.Google Scholar

  • [16]

    H. Do, S. Elbaum and G. Rothermel, Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir. Softw. Eng. 10 (2005), 405–435.Google Scholar

  • [17]

    H. Do, S. Mirarab, L. Tahvildari and G. Rothermel, The effects of time constraints on test case prioritization: a series of controlled experiments, IEEE Trans. Softw. Eng. 36 (2010) 593–617.Google Scholar

  • [18]

    S. G. Elbaum, P. Kallakuri, A. G. Malishevsky and G. Rothermel and S. Kanduri, Understanding the effects of changes on the cost-effectiveness of regression testing techniques, Journal of Software Testing, Verification, and Reliability, 13 (2001), 65–83.Google Scholar

  • [19]

    Guide to the software engineering body of knowledge – SWEBOK, A. Abran, P. Bourque, R. Dupuis and J. W. Moore (Eds.), IEEE Press, Piscataway, NJ, USA, 2004.Google Scholar

  • [20]

    A. A. Haider, S. Rafique and A. Nadeem, Test suite optimization using fuzzy logic, in: 8th International Conference of Emerging Techniques (ICET), 8th October 2012.Google Scholar

  • [21]

    M. Harman, Making the case for MORTO: multi objective regression test optimization, in: Software Testing, Verification and Validation Workshops (ICSTW), 2011 IEEE Fourth International Conference on. IEEE, 2011.Google Scholar

  • [22]

    H.-Y. Hsu and A. Orso, MINTS: a general framework and tool for supporting test-suite minimization, in: Proceedings of the 31st IEEE and ACM SIGSOFT International Conference on Software Engineering (ICSE 2009), Vol 1, pp. 419–429, 2009.Google Scholar

  • [23]

    M. Hutchins, et al. Experiments of the effectiveness of dataflow-and control flow-based test adequacy criteria, in: Proceedings of the 16th international conference on Software engineering, IEEE Computer Society Press, 1994.Google Scholar

  • [24]

    J. Jablonský, Software support for multiple criteria decision making problems, Management 4 (2009), 029–034.Google Scholar

  • [25]

    J.-S. R. Jang, ANFIS: adaptive-network-based fuzzy inference system, IEEE Trans. Syst. Man Cybern. 23, (1993), 665–685.Google Scholar

  • [26]

    J.-S. R. Jang, Input selection for ANFIS learning. Fuzzy Systems, 1996, in: Proceedings of the Fifth IEEE International Conference on., Vol. 2, IEEE, 1996.Google Scholar

  • [27]

    J.-S. R. Jang, C.-T. Sun and E. Mizutani, Neuro-fuzzy and soft computing: a computational approach to learning and machine learning, Prentice Hall Upper Saddle River, NJ 07458, ISBN: 0-13-261066-3, 1997.Google Scholar

  • [28]

    A. Kaur, A bee colony optimization algorithm for code coverage test suite prioritization, Int. J. Eng. Sci. Technol. (IJEST), 3 (2011), 2786–2795.Google Scholar

  • [29]

    A. Kaur and S. Goyal, A genetic algorithm for regression test case prioritization using code coverage, Int. J. Comput. Sci. Eng. (IJCSE), ISSN: 0975-3397 3 (2011), 1839–1847.Google Scholar

  • [30]

    A. Kaur and S. Goyal, A genetic algorithm for fault based regression test case prioritization, Int. J. Comput. Appl. (0975 – 8887) 32 (2011), 30–37.Google Scholar

  • [31]

    A. Konak, D. W. Coit and A. E. Smith, Multi-objective optimization using genetic algorithms: a tutorial, Reliab. Eng. Syst. Saf. 91 (2006), 992–1007.Google Scholar

  • [32]

    Z. Li, M. Harman and R. M. Hierons, Search algorithms for regression test case prioritization, IEEE T. Softw. Eng. 33 (2007), 225–237.Google Scholar

  • [33]

    L. Magdalena, What is soft computing? Revisiting possible answers, Int. J. Comput. Intell. Syst. 3 (2010), 148–159.Google Scholar

  • [34]

    R. Malhotra, A. Kaur and Y. Singh, A regression test selection and prioritization technique, J. Inf. Process. Syst. 6 (2010), 235–252.Google Scholar

  • [35]

    E. H. Mamdani and S. Assilian, An experiment in linguistic synthesis with a fuzzy logic controller, Int. J. Man Mach. Stud. 7 (1975), 1–13.Google Scholar

  • [36]

    MATLAB: Fuzzy Logic toolbox user’s guide version 2.2.16 The MathWorks, 2012.Google Scholar

  • [37]

    S. Nachiyappan, A. Vimaladevi and C. B. Selva Lakshmi, An evolutionary algorithm for regression test suite reduction, in: Proceedings of the International Conference on Communication and Computational Intelligence – 2010, pp 503–508, 2010.Google Scholar

  • [38]

    A. Nanda, S. Mani, S. Sinha, M. J. Harrold and A. Orso, Regression testing in the presence of non-code changes, in: Software Testing, Verification and Validation (ICST), 2011 IEEE Fourth International Conference on, pp. 21–30. IEEE, 2011.Google Scholar

  • [39]

    M. Nashat and W. Statieh, Regression test selection for C# programs, Adv. Softw. Eng. 2009 (2009), 1.Google Scholar

  • [40]

    S. Parsa and A. Khalilian, On the optimization approach towards test suite minimization, Int. J. Softw. Eng. Appl. 4 (2010), 15–28.Google Scholar

  • [41]

    S. Prasad, M. Jain and S. Singh, Regression optimizer a multi coverage criteria test suite minimization technique, Int. J. Appl. Inf. Syst. (IJAIS) – ISSN: 2249-0868, Foundation of Computer Science FCS, New York, USA Volume 1– No.8, April 2012 – www.ijais.org, pp. 5–11.

  • [42]

    S. Raju and G. V. Uma, Factors oriented test case prioritization technique in regression testing using genetic algorithm. Eur. J. Sci. Res. 74 (2012), 389–402.Google Scholar

  • [43]

    S. Sumathi and P. Surekha, Computational intelligence paradigms theory and applications using MATLAB, CRC Press Taylor & Francis Group, Boca Raton, London, New York, ISBN: 978-1-4398-0902-0, 2010.Google Scholar

  • [44]

    C.-A. Sun, A constraint-based test suite reduction method for conservative regression testing, J. Softw. 6 (2011), 314–321.Google Scholar

  • [45]

    L. H. Tahat, A. Bader, B. Vaysburg and B. Korel. Requirement-based automated black-box test generation, in: Proceedings of the 25th International Computer Software and Applications Conference on Invigorating Software Development (COMPSAC ’01), pp 489–495, IEEE Computer Society, Washington, DC, USA, 2001.Google Scholar

  • [46]

    T. Takagi and M. Sugeno. Fuzzy identification of systems and its applications to modeling and control, IEEE T. Syst. Man Cybern. 1 (1985), 116–132.Google Scholar

  • [47]

    J. Vieira, F. Morgado Dias and A. Mota. Neuro-fuzzy systems: a survey, in: 5th WSEAS NNA International Conference on Neural Networks and Applications, Udine, Italia, 2004.Google Scholar

  • [48]

    G. Whyte and D. L. Mulder, Mitigating the impact of software test constraints on software testing effectiveness, Electron. J. Inf. Syst. Eval. 14 (2011), 254–270.Google Scholar

  • [49]

    W. E. Wong, J. R. Horgan, S. London and H. Agrawal, A study of effective regression testing in practice, in: 8th IEEE International Symposium on Software Reliability Engineering (ISSRE’97), pp. 264–274, Albuquerque, NM, November 1997.Google Scholar

  • [50]

    Z. Xu, K. Gao, T. M. Khoshgoftaar and N. Seliya, System regression test planning with a fuzzy expert system. Inf. Sci. 259 (2014), 532–543.Google Scholar

  • [51]

    S. Yoo and M. Harman, Regression testing minimisation, selection and prioritisation: a survey, Softw. Test. Verif. Reliab. 00:1–7 (2007), 1–60.Google Scholar

  • [52]

    S. Yoo and M. Harman, Pareto efficient multi-objective test case selection, in: ISSTA, pp. 140–150, 2007.Google Scholar

  • [53]

    X. Zhiwei, K. Gao and T. M. Khoshgoftaar. Application of fuzzy expert system in test case selection for system regression test, in: IRI-2005 IEEE International Conference on Information Reuse and Integration, Conf, 2005., IEEE, 2005.Google Scholar

  • [54]

    Q. Zhongsheng, Test case generation and optimization for user session-based web application testing, J. Comput. 5 (2010), 1655–1662.Google Scholar

  • [55]

    A. Zhou, B.-Y. Qu, H. Li, S.-Z. Zhao, P. N. Suganthan and Q. Zhang, Multiobjective evolutionary algorithms: a survey of the state of the art, Swarm Evol. Comput. 1 (2011), 32–49.Google Scholar

About the article

Received: 2014-10-13

Published Online: 2015-05-05

Published in Print: 2016-04-01


Citation Information: Journal of Intelligent Systems, Volume 25, Issue 2, Pages 123–146, ISSN (Online) 2191-026X, ISSN (Print) 0334-1860, DOI: https://doi.org/10.1515/jisys-2014-0152.

Export Citation

©2016 by De Gruyter.Get Permission

Comments (0)

Please log in or register to comment.
Log in