Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

4 Issues per year

Open Access
Online
ISSN
2083-2567
See all formats and pricing
More options …

Recursive-Rule Extraction Algorithm With J48graft And Applications To Generating Credit Scores

Yoichi Hayashi / Yuki Tanaka / Tomohiro Takagi / Takamichi Saito / Hideaki Iiduka / Hiroaki Kikuchi / Guido Bologna
  • Department of Information Technology, University of Applied Sciences of Western Switzerland Rue de la prairie 4, 1204 Geneva, Switzerland
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Sushmita Mitra
  • Sushmita Mitra Machine Intelligence Unit, Indian Statistical Institute 203 B.T. Road, Kolkata 700 108, India
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2016-01-13 | DOI: https://doi.org/10.1515/jaiscr-2016-0004

Abstract

The purpose of this study was to generate more concise rule extraction from the Recursive-Rule Extraction (Re-RX) algorithm by replacing the C4.5 program currently employed in Re-RX with the J48graft algorithm. Experiments were subsequently conducted to determine rules for six different two-class mixed datasets having discrete and continuous attributes and to compare the resulting accuracy, comprehensibility and conciseness. When working with the CARD1, CARD2, CARD3, German, Bene1 and Bene2 datasets, Re-RX with J48graft provided more concise rules than the original Re-RX algorithm. The use of Re-RX with J48graft resulted in 43.2%, 37% and 21% reductions in rules in the case of the German, Bene1 and Bene2 datasets compared to Re-RX. Furthermore, the Re-RX with J48graft showed 8.87% better accuracy than the Re-RX algorithm for the German dataset. These results confirm that the application of Re-RX in conjunction with J48graft has the capacity to facilitate migration from existing data systems toward new concise analytic systems and Big Data.

Keywords: Rule Extraction; Credit Scoring; Re-RX algorithm; J48graft

References

  • [1] Marqus, A.I., Garca, V., & Snchez, J.S. On the suitability of resampling techniques for the class imbalance problem in credit scoring. Journal of the Operational Research Society 64, pp. 1060–1070, 2013.CrossrefWeb of ScienceGoogle Scholar

  • [2] Zhao Z., Xu, S., Kang, B. H., Kabir, M. M. J., & Liu, Y. Investigation and improvement of multilayer perceptron neural networks for credit scoring Expert Systems with Applications 42, pp. 3508–3516, 2015Web of ScienceGoogle Scholar

  • [3] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011.Web of ScienceGoogle Scholar

  • [4] Quinlan, J.R. Programs for Machine Learning. Morgan Kaufmann Series in Machine Learning, San Mateo, CA, 1993, Morgan Kaufman.Google Scholar

  • [5] Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. Comprehensible credit scoring models using support vector machines. European Journal of Operational Research 183, pp. 1497–1488, 2007.Web of ScienceGoogle Scholar

  • [6] Baesens, B., et al. Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science 49, No. 3, pp. 312–329, 2004.CrossrefGoogle Scholar

  • [7] Abellan, J., and Mantas, C. Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications, pp. 3825–3830, 2014.Web of ScienceGoogle Scholar

  • [8] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011.Web of ScienceGoogle Scholar

  • [9] Braket, N., and Bradely, A.P. Rule extraction from support vector machine: a review. Neurocomputing 74, pp. 178–190, 2010.CrossrefGoogle Scholar

  • [10] Setiono, R., and Liu, H. Neurolinear: From neural networks to oblique decision rules. Neurocomputing 17, No. 1, pp. 1–24, 1997.CrossrefGoogle Scholar

  • [11] Setiono, R. and Liu, H. A connectionist approach to generating oblique decision trees. IEEE Trans. Syst., Man, Cybern. B, Cybern. Vol. 29, No. 3, pp. 440–444, Jun. 1999.Google Scholar

  • [12] Setiono, R., Baesens, B. & Mues, C. A note on knowledge discovery using neural Setiono networks and its application to credit card screening. European Journal of Operational Research 192, pp.326-332, 2009.Web of ScienceGoogle Scholar

  • [13] Setiono, R., et al. Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19, No. 2, pp. 299–307, 2008.Web of ScienceGoogle Scholar

  • [14] Setiono, R., and Liu, H. Symbolic representation of neural networks. IEEE Computer 29, No. 3, pp. 71–77, 1996.CrossrefGoogle Scholar

  • [15] Gupta, A, Park, S. and Lam, S.M. Generalized analytic rule extraction for feedforward neural networks, IEEE Trans. Knowledge and Data Engineering, 11, pp.985-991, 1999.Google Scholar

  • [16] Etchell, T.A. and Lisboa, J.P.G., Orthogonal search-based rule extraction (OSRE) for trained neural-networks: A practical and efficient approach, IEEE Trans. Neural Networks 17, pp.374-384, 2006.CrossrefGoogle Scholar

  • [17] Hansen, L.K., and Salamon, P., Neural network ensembles. IEEE Trans. Patter Analysis and Machine Intelligence 12, pp. 993–1001, 1990.CrossrefGoogle Scholar

  • [18] Igelnik, S., Pao, Y.-H., LeClair, S. R., and Shen, C. Y. The ensemble approach to neural-network learning and generalization. IEEE Trans. Neural Networks 10, pp. 19–30, 1999.CrossrefGoogle Scholar

  • [19] Liao, J.-J., Shih, C.-H., Chen, T.-F., and Hsu, M.-F. An example-based model for two-class imbalanced financial problem. Economic Modelling 37, pp. 175–183, 2014.Web of ScienceCrossrefGoogle Scholar

  • [20] Setiono R. et al., Rule extraction from minimal neural networks for credit card screening, Inter. J. of Neural Systems., Vol. 21, No. 4, pp. 265–276, 2011.Google Scholar

  • [21] Setiono R. et al., A note on knowledge discovery using neural networks and its application to credit card screening, European J. Operational Research, Vol. 192, pp. 326–332, 2009.Google Scholar

  • [22] Hayashi Y. et al., Understanding consumer heterogeneity: A business intelligence application of neural networks, Knowledge-Based Systems, Vol. 23, No. 8, pp. 856–863, 2010.Google Scholar

  • [23] Bologna G., Is it worth generating rules from neural network ensemble?, J. of Applied Logic, Vol. 2, pp. 325–348, 2004.Google Scholar

  • [24] Zhou, Z.-H. Extracting symbolic rules from trained neural network ensembles. AI Communications 16, pp. 3–15, 2003.Google Scholar

  • [25] http://fiji.sc/javadoc/weka/classifiers/trees/J48graft.html

  • [26] Setiono R. et al., A penalty-function approach for pruning feedforward neural networks, Neural Comp., Vol. 9, No. 1, pp. 185–204, 1997.Google Scholar

  • [27] Witten, I.H. and Frank, E., Data Ming: Practical Machine Learning Tools With Java Implementations. San Francisco, CA: Morgan Kaufmann, 1999.Google Scholar

  • [28] Quinlan J.R., Induction of decision trees, Machine Learning, Vol.1, pp.81-106, 1986.Google Scholar

  • [29] Webb. G.I., Decision Tree Grafting from the All-Tests-But-One Partition, in Proc. 16th International Joint Conference on Artificial Intelligence (IJCAI), Vol. 2, pp. 702–707, 1999.Google Scholar

  • [30] Webb, G. I., Decision Tree Grafting, Learining, IJCAI’97 Proceedings of the 15th International Conference on Artificial Intelligence, Vol.2, pp. 846-885, 1997.Google Scholar

  • [31] Frank, A. & Asuncion, A. University of California. Irvine Machine Learning Repository. http://archive.ics.uci.edu/ml/, 2010

  • [32] Prechelt, L. Proben1 — A set of benchmarks and benchmarking rules for neural network training algorithms, Technical Report 21/94, Fakultt fr Informatik, Universitt Karlsruhe, Germany. Anonymous ftp available from ftp://pub/papers/techreport/1994/1994-21.ps.gz on ftp.ira.uka.de, 1994.Google Scholar

  • [33] Smith, M. Neural Networks for Statistical Modeling, New York: Van Nostrand Reinhold, 1993.Google Scholar

About the article

Published Online: 2016-01-13

Published in Print: 2016-01-01


Citation Information: Journal of Artificial Intelligence and Soft Computing Research, ISSN (Online) 2083-2567, DOI: https://doi.org/10.1515/jaiscr-2016-0004.

Export Citation

© 2016 Academy of Management (SWSPiZ), Lodz. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Ying Xie, Yaohua Chen, and Lingxi Peng
Cluster Computing, 2017, Volume 20, Number 2, Page 1379

Comments (0)

Please log in or register to comment.
Log in