Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

4 Issues per year

Open Access
Online
ISSN
2083-2567
See all formats and pricing
More options …

Enhancing Constructive Neural Network Performance Using Functionally Expanded Input Data

João Roberto Bertini Junior
  • Department of Computer Science, UFSCar, Rodovia Washington Luís, km 235, São Carlos-SP, Brazil
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Maria do Carmo Nicoletti
  • Department of Computer Science, UFSCar & FACCAMP, São Carlos & Campo Limpo Paulista - SP, Brazil
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2016-03-10 | DOI: https://doi.org/10.1515/jaiscr-2016-0010

Abstract

Constructive learning algorithms are an efficient way to train feedforward neural networks. Some of their features, such as the automatic definition of the neural network (NN) architecture and its fast training, promote their high adaptive capacity, as well as allow for skipping the usual pre-training phase, known as model selection. However, such advantages usually come with the price of lower accuracy rates, when compared to those obtained with conventional NN learning approaches. This is, perhaps, the reason for conventional NN training algorithms being preferred over constructive NN (CoNN) algorithms. Aiming at enhancing CoNN accuracy performance and, as a result, making them a competitive choice for machine learning based applications, this paper proposes the use of functionally expanded input data. The investigation described in this paper considered six two-class CoNN algorithms, ten data domains and seven polynomial expansions. Results from experiments, followed by a comparative analysis, show that performance rates can be improved when CoNN algorithms learn from functionally expanded input data.

Keywords: Constructive neural networks; Functional link artificial neural networks; Functionally expanded input data

References

  • [1] E. Amaldi and B. Guenin, Two constructive methods for designing compact feedfoward networks of threshold units, International Journal of Neural System, vol. 8, no. 5-6, 1997, pp. 629-645.Google Scholar

  • [2] J. K. Anlauf, M. Bieh, The AdaTron: an adaptive perceptron algorithm, Europhysics Letters, vol. 10,1989, pp. 687-692.CrossrefGoogle Scholar

  • [3] J. R. Bertini Jr. and M. C. Nicoletti, Refining constructive neural networks using functionally expanded input data, Proc. Int. Joint Conference on Neural Networks, 2015, pp. 1-8.Google Scholar

  • [4] J. R. Bertini Jr. and M. C. Nicoletti, A constructive neural network algorithm based on the geometric concept of barycenter of convex hull, In: Computational Intelligence: Methods and Applications, IEEE Comp. Intelligence Society, Poland, 2008, pp. 1-12.Google Scholar

  • [5] N. Burgess, A constructive algorithm that converges for real-valued input patterns, International Journal of Neural Systems, vol. 5, no. 1, 1994, pp. 59-66.Google Scholar

  • [6] S. Fahlman and C. Lebiere, The cascade correlation architecture, in Advances in Neural Information Processing Systems, vol. 2, 1990, pp. 524-532.Google Scholar

  • [7] S. E. Fahlman, Faster-learning variations on backpropagation: an empirical study, In: Proc. of the 1988 Connectionist Models Summer School, D. S. Touretzky, G. E. Hinton and T. J. Sejnowski (Eds.), Morgan Kaufmann, San Mateo, CA, 1988, pp. 38-51.Google Scholar

  • [8] S. Fahlman and C. Lebiere, The cascade correlation architecture, in Advances in Neural Information Processing Systems, vol. 2, 1990, pp. 524-532.Google Scholar

  • [9] L. Franco, D. A. Elizondo and J. M. Jerez, Constructive Neural Networks, Studies in Comp. Intelligence Series, v. 258, Springer, 2010.Google Scholar

  • [10] M. Frean, A thermal perceptron learning rule, Neural Computation, vol. 4, 1992, pp. 946-957.CrossrefGoogle Scholar

  • [11] M. Frean, The upstart algorithm: a method for constructing and training feedforward neural networks, Neural Computation, vol. 2, pp. 198-209, 1990.CrossrefGoogle Scholar

  • [12] S. I. Gallant, Perceptron-based learning algorithms, IEEE Transactions on Neural Networks, vol. 1, no. 2, 1990, pp. 179-191.Google Scholar

  • [13] S. I. Gallant, Neural Network Learning and Expert Systems, The MIT Press, London, England, 1994.Google Scholar

  • [14] X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, In: Proc. AISTATS, 2010, pp. 249-256.Google Scholar

  • [15] T. Hrycej, Modular Learning in Neural Networks, A. Wiley, N. York, 1992.Google Scholar

  • [16] Y-C. Hu, Functional-link nets with geneticalgorithm- based learning for robust nonlinear interval regression analysis, Neurocomputing, vol. 72, 2009, pp. 1808-1816.Web of ScienceGoogle Scholar

  • [17] W. Krauth and M. M´ezard, Learning algorithms with optimal stability in neural networks, Journal of Physics A, vol. 20, pp. 745-752, 1987.Google Scholar

  • [18] M. Lichman, UCI Machine Learning Repository. [Online]. Available: http://archive.ics.uci.edu/ml. Irvine, CA: University of California, School of Information and Computer Science, 2013.Google Scholar

  • [19] D. Martinez and D. Est`eve, The offset algorithm: building and learning method for multilayer neural networks, Europhysics Letters, vol. 18, no. 2, 1992, pp. 95-100.CrossrefGoogle Scholar

  • [20] M.M´ezard and J.-P. Nadal, Learning in feedfoward networks: the tiling algorithm, Journal of Physics A: Mathematical and General, vol. 22, 1989, pp. 2191-2203.Google Scholar

  • [21] M. Muselli, Sequential constructive techniques, Neural Network Systems Techniques and Applications, C. Leondes (Ed.), San Diego, CA: Academic, vol. 2, 1998, pp. 81-144.Google Scholar

  • [22] M. C. Nicoletti and J. R. Bertini Jr., An empirical evaluation of constructive neural network algorithms in classification tasks, International Journal of Innovative Computing and Applications, vol. 1, 2007, pp. 2-13.Google Scholar

  • [23] M. C. Nicoletti, J. R. Bertini Jr., D. Elizondo, L. Franco and J. M. Jerez, Constructive neural network algorithms for feedforward architectures suitable for classification tasks, In: Constructive Neural Networks, Studies in Comp. Intelligence, D. Elizondo, L. Franco and J.M. Jerez, Springer, 2010, pp. 1-23.Google Scholar

  • [24] Y. H. Pao, Adaptive pattern recognition and neural networks, Addison-Wesley, Reading, MA, 1989.Google Scholar

  • [25] Y. H. Pao and Y. Takefuji, Functional-link net computing: theory, system architecture, and functionalities, Computer, vol. 25, no. 5, 1992, pp. 76-79.Google Scholar

  • [26] R. G. Parekh, J. Yang and V. Honavar, Constructive neural-network learning algorithms for pattern classification, IEEE Transactions on Neural Networks, vol. 11, no. 2, 2000, pp. 436-451.Google Scholar

  • [27] H. Poulard, Barycentric correction procedure - fast method of learning threshold unit, In: Proc. of WCNN 95, vol. 1, 1995, pp. 710-713.Google Scholar

  • [28] D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning representations by back-propagating errors, Nature, vol. 323, no. 6088, 1986, pp. 533-536.Google Scholar

  • [29] M. R. Spiegel, Mathematical Handbook of Formulas and Tables, Schaum’s outline series, McGraw- Hill Inc., USA, 1968.Google Scholar

  • [30] L. Toth and T. Grosz, A comparison of deep neural network training methods for large vocabulary speech recognition, In: TSD 2013, I. Habernal and V. Matousek (Eds.), LNAI 8082, Springer, 2013, pp. 36-43.Google Scholar

  • [31] A. Wendmuth, Learning the unlearnable, Journal of Physics A: Mathematical and General, vol. 28, 1995, pp. 5423-5436.Google Scholar

  • [32] J. Yosinski, J. Clune, Y. Bengio and H. Lipson, How transferable are features in deep neural networks?, Advances in Neural Information Processing Systems 27, NIPS Foundation, 2014, pp. 3320-3328 Google Scholar

About the article

Published Online: 2016-03-10

Published in Print: 2016-04-01


Citation Information: Journal of Artificial Intelligence and Soft Computing Research, Volume 6, Issue 2, Pages 119–131, ISSN (Online) 2083-2567, DOI: https://doi.org/10.1515/jaiscr-2016-0010.

Export Citation

© 2016. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Beatriz Pérez-Sánchez, Oscar Fontenla-Romero, and Bertha Guijarro-Berdiñas
Artificial Intelligence Review, 2016

Comments (0)

Please log in or register to comment.
Log in