Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

4 Issues per year

Open Access
See all formats and pricing
More options …

Learning Structures of Conceptual Models from Observed Dynamics Using Evolutionary Echo State Networks

Hassan Abdelbari
  • School of Engineering and Information Technology, University of New South Wales at Canberra, Northcott Drive, Campbell, ACT, 2600, Australia
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Kamran Shafi
  • School of Engineering and Information Technology, University of New South Wales at Canberra, Northcott Drive, Campbell, ACT, 2600, Australia
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2017-11-01 | DOI: https://doi.org/10.1515/jaiscr-2018-0010


Conceptual or explanatory models are a key element in the process of complex system modelling. They not only provide an intuitive way for modellers to comprehend and scope the complex phenomena under investigation through an abstract representation but also pave the way for the later development of detailed and higher-resolution simulation models. An evolutionary echo state network-based method for supporting the development of such models, which can help to expedite the generation of alternative models for explaining the underlying phenomena and potentially reduce the manual effort required, is proposed. It relies on a customised echo state neural network for learning sparse conceptual model representations from the observed data. In this paper, three evolutionary algorithms, a genetic algorithm, differential evolution and particle swarm optimisation are applied to optimize the network design in order to improve model learning. The proposed methodology is tested on four examples of problems that represent complex system models in the economic, ecological and physical domains. The empirical analysis shows that the proposed technique can learn models which are both sparse and effective for generating the output that matches the observed behaviour.

Keywords: Complex systems modelling; Conceptual models; Causal loop diagrams; Computational intelligence; Echo state networks; Evolutionary algorithms


  • [1] J. D. Sterman, Business Dynamics: Systems Thinking and Modeling for a Complex World, vol. 19. Irwin/McGraw-Hill Boston, 2000.Google Scholar

  • [2] F. C. Billari, Agent-based computational modelling: applications in demography, social, economic and environmental sciences. Taylor & Francis, 2006.Web of ScienceGoogle Scholar

  • [3] R. A. Howard and J. E. Matheson, Influence diagrams, Decis. Anal., vol. 2, no. 3, pp. 127–143, 2005.Google Scholar

  • [4] F.-R. Lin, M.-C. Yang, and Y.-H. Pai, A generic structure for business process modeling, Bus. Process Manag. J., vol. 8, no. 1, pp. 19–41, 2002.Google Scholar

  • [5] L. Schruben, Simulation modeling with event graphs, Commun. ACM, vol. 26, no. 11, pp. 957–963, 1983.CrossrefGoogle Scholar

  • [6] S. Robinson, Simulation: the practice of model development and use. Palgrave Macmillan, 2014.Google Scholar

  • [7] J. Ryan and C. Heavey, Requirements gathering for simulation, in Proceedings of the 3rd Operational Research Society Simulation Workshop. The Operational Research Society, Birmingham, UK, 175-184, 2006.Google Scholar

  • [8] A. Medina-Borja and K. S. Pasupathy, Uncovering complex relationships in system dynamics modeling: Exploring the use of CART, CHAID and SEM, in Proceedings of the 25th International Conference of the System Dynamics Society, (Boston, USA), pp. 1–24, 2007.Google Scholar

  • [9] V. Quiñones-Avila and A. Medina-Borja, Universal healthcare: key behavioural factors affecting providers and recipients value propositions: a structural causal model of the puerto rico experience, Int. J. of Behav. and Hlthc. Res., vol. 3, no. 1, pp. 25–45, 2012.Google Scholar

  • [10] M. Drobek, W. Gilani, T. Molka, and D. Soban, Automated equation formulation for causal loop diagrams, Lecture Notes in Business Information Processing, vol. 208, pp. 38–49, 2015.Google Scholar

  • [11] E. Pruyt, S. Cunningham, J. Kwakkel, and J. De Bruijn, From data-poor to data-rich: system dynamics in the era of big data, in Proceedings of the 32nd International Conference of the System Dynamics Society, Delft, The Netherlands, 20-24 July 2014.Google Scholar

  • [12] H. Jaeger, The ’echo state’ approach to analysing and training recurrent neural networks-with an erratum note, Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, vol. 148, p. 34, 2001.Google Scholar

  • [13] H. Abdelbari and K. Shafi, Learning causal loop diagram-like structures for system dynamics modeling using echo state networks, Syst. Dynam. Rev. - In Press, 2017.Google Scholar

  • [14] D. E. Goldberg, Genetic algorithms. Pearson Education India, 2006.Google Scholar

  • [15] R. Storn and K. Price, Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces, J. Global. Optim., vol. 11, no. 4, pp. 341–359, 1997.CrossrefGoogle Scholar

  • [16] J. Kennedy, Particle swarm optimization, in Encyclopedia of machine learning, pp. 760–766, Springer, 2011.Google Scholar

  • [17] Z. Wang, J. Zhang, J. Ren, and M. N. Aslam, A geometric singular perturbation approach for planar stationary shock waves, Physica D, vol. 310, pp. 19–36, 2015.Web of ScienceGoogle Scholar

  • [18] C. K. Jones, R. Marangell, P. D. Miller, and R. G. Plaza, On the stability analysis of periodic sine–gordon traveling waves, Physica D, vol. 251, pp. 63–74, 2013.Web of ScienceGoogle Scholar

  • [19] V. V. Gursky, J. Reinitz, and A. M. Samsonov, How gap genes make their domains: An analytical study based on data driven approximations, Chaos, vol. 11, no. 1, pp. 132–141, 2001.Google Scholar

  • [20] P. Young, Data-based mechanistic modelling of environmental, ecological, economic and engineering systems, Environ. Modell. Softw., vol. 13, no. 2, pp. 105–122, 1998.Google Scholar

  • [21] Y. Zhao, T. Weng, and M. Small, Response of the parameters of a neural network to pseudoperiodic time series, Physica D, vol. 268, pp. 79–90, 2014.Google Scholar

  • [22] Y. Feng, Y. Liu, X. Tong, M. Liu, and S. Deng, Modeling dynamic urban growth using cellular automata and particle swarm optimization rules, Landscape Urban Plan., vol. 102, no. 3, pp. 188–196, 2011.CrossrefGoogle Scholar

  • [23] N. Petrov and A. Gegov, Model optimization for complex systems using fuzzy networks theory, in Proceedings of the 8th WSEAS international conference on Artificial intelligence, knowledge engineering and data bases, pp. 116–121, World Scientific and Engineering Academy and Society (WSEAS), 2009.Google Scholar

  • [24] I. M. Greca and M. A. Moreira, Mental models, conceptual models, and modelling, Int. J. Sci. Educ, vol. 22, no. 1, pp. 1–11, 2000.Google Scholar

  • [25] J. D. Sterman, Systems dynamics modeling: tools for learning in a complex world, IEEE Eng. Manag. Rev., vol. 30, no. 1, pp. 42–42, 2002.Google Scholar

  • [26] G. Desthieux, F. Joerin, and M. Lebreton, Ulysse: a qualitative tool for eliciting mental models of complex systems, Syst. Dynam. Rev., vol. 26, no. 2, pp. 163–192, 2010.CrossrefGoogle Scholar

  • [27] K.-i. Funahashi and Y. Nakamura, Approximation of dynamical systems by continuous time recurrent neural networks, Neural networks, vol. 6, no. 6, pp. 801–806, 1993.Google Scholar

  • [28] H. Jaeger, Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the” echo state network” approach, Tech. Rep. 159, Fraunhofer Institute for Autonomous Intelligent Systems (AIS), 2002b.Google Scholar

  • [29] D. Koryakin, J. Lohmann, and M. V. Butz, Balanced echo state networks, Neural Networks, vol. 36, pp. 35–45, 2012.CrossrefWeb of ScienceGoogle Scholar

  • [30] I. B. Yildiz, H. Jaeger, and S. J. Kiebel, Re-visiting the echo state property, Neural networks, vol. 35, pp. 1–9, 2012.Google Scholar

  • [31] M. Lukoševišius, A practical guide to applying echo state networks, in Neural Networks: Tricks of the Trade, pp. 659–686, Springer, 2012.CrossrefGoogle Scholar

  • [32] C. E. Martin and J. A. Reggia, Fusing swarm intelligence and self-assembly for optimizing echo state networks, Comput. Intell. Neurosci., vol. 2015, p. 9, 2015.Web of ScienceGoogle Scholar

  • [33] A. A. Ferreira and T. B. Ludermir, Comparing evolutionary methods for reservoir computing pretraining, in Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, California, USA, pp. 283–290, July 31 - August 5 2011.Google Scholar

  • [34] A. Deihimi and A. Solat, optimised echo state networks using a big bang–big crunch algorithm for distance protection of series-compensated transmission lines, Int. J. Elec. Power., vol. 54, pp. 408–424, 2014.CrossrefGoogle Scholar

  • [35] A. A. Ferreira, T. B. Ludermir, and R. R. De Aquino, An approach to reservoir computing design and training, Expert. Syst. Appl., vol. 40, no. 10, pp. 4172–4182, 2013.Google Scholar

  • [36] D. Liu, J. Wang, and H. Wang, Short-term wind speed forecasting based on spectral clustering and optimised echo state networks, Renew. Energ., vol. 78, pp. 599–608, 2015.Web of ScienceGoogle Scholar

  • [37] J. L. Gross and J. Yellen, Handbook of graph theory. CRC press, 2004.Google Scholar

  • [38] R. Tarjan, Depth-first search and linear graph algorithms, SIAM J. Comput., vol. 1, no. 2, pp. 146–160, 1972.CrossrefGoogle Scholar

  • [39] V. Petridis, S. Kazarlis, and A. Bakirtzis, Varying fitness functions in genetic algorithm constrained optimization: the cutting stock and unit commitment problems, IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 28, no. 5, pp. 629–640, 1998.Google Scholar

  • [40] A. E. Smith and D. M. Tate, Genetic optimization using a penalty function, in Proceedings of the 5th international conference on genetic algorithms, pp. 499–505, Morgan Kaufmann Publishers Inc., 1993.Google Scholar

  • [41] K. Langfield-Smith and A. Wirth, Measuring differences between cognitive maps, J. Oper. Res. Soc., pp. 1135–1150, 1992.CrossrefGoogle Scholar

  • [42] Y.-C. Chuang, C.-T. Chen, and C. Hwang, A simple and efficient real-coded genetic algorithm for constrained optimization, Appl. Soft. Comput., vol. 38, pp. 87–105, 2016.CrossrefWeb of ScienceGoogle Scholar

  • [43] J. Lane, A. Engelbrecht, and J. Gain, Particle swarm optimization with spatially meaningful neighbours, in Swarm Intelligence Symposium, 2008. SIS 2008. IEEE, pp. 1–8, IEEE, 2008.Google Scholar

  • [44] R. C. Eberhart and Y. Shi, Comparing inertia weights and constriction factors in particle swarm optimization, in Proceedings of the 2000 Congress on Evolutionary Computation, vol. 1, pp. 84–88, IEEE, 2000.Google Scholar

  • [45] S. N. Grösser and M. Schaffernicht, Mental models of dynamic systems: taking stock and looking ahead, Syst. Dynam. Rev., vol. 28, no. 1, pp. 46–68, 2012.Google Scholar

  • [46] E. M. Aylward, P. A. Parrilo, and J.-J. E. Slotine, Stability and robustness analysis of nonlinear systems via contraction metrics and sos programming, Automatica, vol. 44, no. 8, pp. 2163–2170, 2008.CrossrefGoogle Scholar

  • [47] M. Rafferty, Butterflies and buffers, in Proceedings of the 27th International Conference of the System Dynamics Society, Albuquerque, Mexico, USA, July 26-30 2009.Google Scholar

  • [48] E. Theodorsson-Norheim, Friedman and quade tests: Basic computer program to perform nonparametric two-way analysis of variance and multiple comparisons on ranks of several related samples, Comput. Biol. Med., vol. 17, no. 2, pp. 85–99, 1987.CrossrefGoogle Scholar

  • [49] M. R. Stoline, The status of multiple comparisons: simultaneous estimation of all pairwise comparisons in one-way anova designs, Am. Stat., vol. 35, no. 3, pp. 134–141, 1981.Google Scholar

  • [50] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Trans. Evol. Comput., vol. 6, no. 2, pp. 182–197, 2002.CrossrefGoogle Scholar

About the article

Received: 2017-03-04

Accepted: 2017-03-29

Published Online: 2017-11-01

Published in Print: 2018-04-01

Citation Information: Journal of Artificial Intelligence and Soft Computing Research, Volume 8, Issue 2, Pages 133–154, ISSN (Online) 2083-2567, DOI: https://doi.org/10.1515/jaiscr-2018-0010.

Export Citation

© 2018 Hassan Abdelbari et al., published by De Gruyter Open. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in