Jump to ContentJump to Main Navigation
Show Summary Details
In This Section

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

4 Issues per year

Open Access
See all formats and pricing
In This Section

Recursive-Rule Extraction Algorithm With J48graft And Applications To Generating Credit Scores

Yoichi Hayashi
  • Department of Computer Science, Meiji University Kawasaki 214-8571, Japan
/ Yuki Tanaka
  • Department of Computer Science, Meiji University Kawasaki 214-8571, Japan
/ Tomohiro Takagi
  • Department of Computer Science, Meiji University Kawasaki 214-8571, Japan
/ Takamichi Saito
  • Department of Computer Science, Meiji University Kawasaki 214-8571, Japan
/ Hideaki Iiduka
  • Department of Computer Science, Meiji University Kawasaki 214-8571, Japan
/ Hiroaki Kikuchi
  • Department of Frontier Media Science,, Meiji University Nakano-ku, Tokyo 164-8525, Japan
/ Guido Bologna
  • Department of Information Technology, University of Applied Sciences of Western Switzerland Rue de la prairie 4, 1204 Geneva, Switzerland
/ Sushmita Mitra
  • Sushmita Mitra Machine Intelligence Unit, Indian Statistical Institute 203 B.T. Road, Kolkata 700 108, India
Published Online: 2016-01-13 | DOI: https://doi.org/10.1515/jaiscr-2016-0004


The purpose of this study was to generate more concise rule extraction from the Recursive-Rule Extraction (Re-RX) algorithm by replacing the C4.5 program currently employed in Re-RX with the J48graft algorithm. Experiments were subsequently conducted to determine rules for six different two-class mixed datasets having discrete and continuous attributes and to compare the resulting accuracy, comprehensibility and conciseness. When working with the CARD1, CARD2, CARD3, German, Bene1 and Bene2 datasets, Re-RX with J48graft provided more concise rules than the original Re-RX algorithm. The use of Re-RX with J48graft resulted in 43.2%, 37% and 21% reductions in rules in the case of the German, Bene1 and Bene2 datasets compared to Re-RX. Furthermore, the Re-RX with J48graft showed 8.87% better accuracy than the Re-RX algorithm for the German dataset. These results confirm that the application of Re-RX in conjunction with J48graft has the capacity to facilitate migration from existing data systems toward new concise analytic systems and Big Data.

Keywords: Rule Extraction; Credit Scoring; Re-RX algorithm; J48graft


  • [1] Marqus, A.I., Garca, V., & Snchez, J.S. On the suitability of resampling techniques for the class imbalance problem in credit scoring. Journal of the Operational Research Society 64, pp. 1060–1070, 2013. [Crossref] [Web of Science]

  • [2] Zhao Z., Xu, S., Kang, B. H., Kabir, M. M. J., & Liu, Y. Investigation and improvement of multilayer perceptron neural networks for credit scoring Expert Systems with Applications 42, pp. 3508–3516, 2015 [Web of Science]

  • [3] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011. [Web of Science]

  • [4] Quinlan, J.R. Programs for Machine Learning. Morgan Kaufmann Series in Machine Learning, San Mateo, CA, 1993, Morgan Kaufman.

  • [5] Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. Comprehensible credit scoring models using support vector machines. European Journal of Operational Research 183, pp. 1497–1488, 2007. [Web of Science]

  • [6] Baesens, B., et al. Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science 49, No. 3, pp. 312–329, 2004. [Crossref]

  • [7] Abellan, J., and Mantas, C. Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications, pp. 3825–3830, 2014. [Web of Science]

  • [8] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011. [Web of Science]

  • [9] Braket, N., and Bradely, A.P. Rule extraction from support vector machine: a review. Neurocomputing 74, pp. 178–190, 2010. [Crossref]

  • [10] Setiono, R., and Liu, H. Neurolinear: From neural networks to oblique decision rules. Neurocomputing 17, No. 1, pp. 1–24, 1997. [Crossref]

  • [11] Setiono, R. and Liu, H. A connectionist approach to generating oblique decision trees. IEEE Trans. Syst., Man, Cybern. B, Cybern. Vol. 29, No. 3, pp. 440–444, Jun. 1999.

  • [12] Setiono, R., Baesens, B. & Mues, C. A note on knowledge discovery using neural Setiono networks and its application to credit card screening. European Journal of Operational Research 192, pp.326-332, 2009. [Web of Science]

  • [13] Setiono, R., et al. Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19, No. 2, pp. 299–307, 2008. [Web of Science]

  • [14] Setiono, R., and Liu, H. Symbolic representation of neural networks. IEEE Computer 29, No. 3, pp. 71–77, 1996. [Crossref]

  • [15] Gupta, A, Park, S. and Lam, S.M. Generalized analytic rule extraction for feedforward neural networks, IEEE Trans. Knowledge and Data Engineering, 11, pp.985-991, 1999.

  • [16] Etchell, T.A. and Lisboa, J.P.G., Orthogonal search-based rule extraction (OSRE) for trained neural-networks: A practical and efficient approach, IEEE Trans. Neural Networks 17, pp.374-384, 2006. [Crossref]

  • [17] Hansen, L.K., and Salamon, P., Neural network ensembles. IEEE Trans. Patter Analysis and Machine Intelligence 12, pp. 993–1001, 1990. [Crossref]

  • [18] Igelnik, S., Pao, Y.-H., LeClair, S. R., and Shen, C. Y. The ensemble approach to neural-network learning and generalization. IEEE Trans. Neural Networks 10, pp. 19–30, 1999. [Crossref]

  • [19] Liao, J.-J., Shih, C.-H., Chen, T.-F., and Hsu, M.-F. An example-based model for two-class imbalanced financial problem. Economic Modelling 37, pp. 175–183, 2014. [Web of Science] [Crossref]

  • [20] Setiono R. et al., Rule extraction from minimal neural networks for credit card screening, Inter. J. of Neural Systems., Vol. 21, No. 4, pp. 265–276, 2011.

  • [21] Setiono R. et al., A note on knowledge discovery using neural networks and its application to credit card screening, European J. Operational Research, Vol. 192, pp. 326–332, 2009.

  • [22] Hayashi Y. et al., Understanding consumer heterogeneity: A business intelligence application of neural networks, Knowledge-Based Systems, Vol. 23, No. 8, pp. 856–863, 2010.

  • [23] Bologna G., Is it worth generating rules from neural network ensemble?, J. of Applied Logic, Vol. 2, pp. 325–348, 2004.

  • [24] Zhou, Z.-H. Extracting symbolic rules from trained neural network ensembles. AI Communications 16, pp. 3–15, 2003.

  • [25] http://fiji.sc/javadoc/weka/classifiers/trees/J48graft.html

  • [26] Setiono R. et al., A penalty-function approach for pruning feedforward neural networks, Neural Comp., Vol. 9, No. 1, pp. 185–204, 1997.

  • [27] Witten, I.H. and Frank, E., Data Ming: Practical Machine Learning Tools With Java Implementations. San Francisco, CA: Morgan Kaufmann, 1999.

  • [28] Quinlan J.R., Induction of decision trees, Machine Learning, Vol.1, pp.81-106, 1986.

  • [29] Webb. G.I., Decision Tree Grafting from the All-Tests-But-One Partition, in Proc. 16th International Joint Conference on Artificial Intelligence (IJCAI), Vol. 2, pp. 702–707, 1999.

  • [30] Webb, G. I., Decision Tree Grafting, Learining, IJCAI’97 Proceedings of the 15th International Conference on Artificial Intelligence, Vol.2, pp. 846-885, 1997.

  • [31] Frank, A. & Asuncion, A. University of California. Irvine Machine Learning Repository. http://archive.ics.uci.edu/ml/, 2010

  • [32] Prechelt, L. Proben1 — A set of benchmarks and benchmarking rules for neural network training algorithms, Technical Report 21/94, Fakultt fr Informatik, Universitt Karlsruhe, Germany. Anonymous ftp available from ftp://pub/papers/techreport/1994/1994-21.ps.gz on ftp.ira.uka.de, 1994.

  • [33] Smith, M. Neural Networks for Statistical Modeling, New York: Van Nostrand Reinhold, 1993.

About the article

Published Online: 2016-01-13

Published in Print: 2016-01-01

Citation Information: Journal of Artificial Intelligence and Soft Computing Research, ISSN (Online) 2083-2567, DOI: https://doi.org/10.1515/jaiscr-2016-0004. Export Citation

© 2016 Academy of Management (SWSPiZ), Lodz. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. (CC BY-NC-ND 3.0)

Comments (0)

Please log in or register to comment.
Log in