Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Measurement Science Review

The Journal of Institute of Measurement Science of Slovak Academy of Sciences

6 Issues per year

IMPACT FACTOR 2016: 1.344

CiteScore 2016: 1.88

SCImago Journal Rank (SJR) 2016: 0.495
Source Normalized Impact per Paper (SNIP) 2016: 1.419

Open Access
See all formats and pricing
More options …
Volume 15, Issue 4


Comparison of the Effects of Cross-validation Methods on Determining Performances of Classifiers Used in Diagnosing Congestive Heart Failure

Yalcin Isler
  • Department of Biomedical Engineering, Izmir Katip Celebi Univertsity, Balatcik Campus, 35620, Izmir, Turkey
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Ali Narin
  • Department of Electrical and Electronics Engineering, Bulent Ecevit University, Incivez, 67100, Zonguldak, Turkey
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Mahmut Ozer
  • Department of Electrical and Electronics Engineering, Bulent Ecevit University, Incivez, 67100, Zonguldak, Turkey
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2015-08-27 | DOI: https://doi.org/10.1515/msr-2015-0027


Congestive heart failure (CHF) occurs when the heart is unable to provide sufficient pump action to maintain blood flow to meet the needs of the body. Early diagnosis is important since the mortality rate of the patients with CHF is very high. There are different validation methods to measure performances of classifier algorithms designed for this purpose. In this study, k-fold and leave-one-out cross-validation methods were tested for performance measures of five distinct classifiers in the diagnosis of the patients with CHF. Each algorithm was run 100 times and the average and the standard deviation of classifier performances were recorded. As a result, it was observed that average performance was enhanced and the variability of performances was decreased when the number of data sections used in the cross-validation method was increased.

Keywords: Heart rate variability; heart failure; cross-validation; classification.


  • [1] U.S. National Library of Medicine. Heart failure (Medical Encyclopedia). http://www.nlm.nih.gov.Google Scholar

  • [2] Flavell, C., Stevenson, L.W. (2001). Take Heart with Heart Failure. Circulation, 104, 89.Google Scholar

  • [3] Wilbur, J., James, P. (2005). Diagnosis and management of heart failure in the outpatient setting. Primary Care, 32, 1115-1129.Web of ScienceGoogle Scholar

  • [4] American Heart Association (2006). Heart Disease and Stroke Statistics-2006 Update: A Report From the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation, 113, 85-151.Google Scholar

  • [5] Isler, Y., Kuntalp, M. (2007). Combining Classical HRV Indices with Wavelet Entropy Measures Improves to Performance in Diagnosing Congestive Heart Failure. Computers in Biology and Medicine, 37(10), 1502-1510.CrossrefWeb of ScienceGoogle Scholar

  • [6] Isler, Y., Kuntalp, M. (2010). Heart Rate Normalization in the Analysis of Heart Rate Variability in Congestive Heart Failure. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 224(3), 453-463.Web of ScienceGoogle Scholar

  • [7] Yu, S.N., Lee, M.Y. (2012). Bispectral analysis and genetic algorithm for congestive heart failure recognition based on heart rate variability. Computers in Biology and Medicine, 42, 816-825.Web of ScienceGoogle Scholar

  • [8] Yu, S.N., Lee, M.Y. (2012). Conditional mutual information-based feature selection for congestive heart failure recognition using heart rate variability. Computer Methods and Programs in Biomedicine, 108, 299-309.Web of ScienceGoogle Scholar

  • [9] Jovic, A., Bogunovic, N. (2011). Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features. Artificial Intelligence in Medicine, 51, 175-186.Web of ScienceGoogle Scholar

  • [10] Pecchia, L., Melillo, P., Sansone, M., Bracale, M. (2011). Discrimination power of short-term heart rate variability measures for CHF assessment. IEEE Transactions on Information Technology in Biomedicine, 15(1), 40-46.Google Scholar

  • [11] Narin, A., Isler, Y., Ozer, M. (2014). Investigating the performance improvement of HRV Indices in CHF using feature selection methods based on backward elimination and statistical significance. Computers in Biology and Medicine, 45, 72-79.Web of ScienceGoogle Scholar

  • [12] Narin, A., Isler, Y. (2012). Effect of Principal Component Analysis on Diagnosing Congestive Heart Failure Patients using Heart Rate Records. In IEEE 20th Signal Processing and Communications Applications Conference (SIU2012), 18-20 April 2012, Fethiye / Mugla.Google Scholar

  • [13] Isler, Y., Selver, M.A., Kuntalp, M. (2005). Effects of Detrending in Heart Rate Variability Analysis. In II. Muhendislik Bilimleri Genc Arastirmacilar Kongresi MBGAK’2005, 17-19 October 2005, Istanbul, 213-219.Google Scholar

  • [14] Duda, R.O., Hart, P.E., Stork, D.G. (2000). Pattern Classification, 2nd edition, New York. Wiley.Google Scholar

  • [15] Goldberger, A.L., et.al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals. Circulation, 101(23), e215-e220.CrossrefGoogle Scholar

  • [16] Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology. (1996). Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. Circulation, 93, 1043-1065.Google Scholar

  • [17] Lomb, N.R. (1976). Least-squares frequency analysis of unequally spaced data. Astrophysics and Space Science, 39, 447-462.Google Scholar

  • [18] Quiraga, R.Q., Rosso, O.A., Basar, E., Schurmann, M. (2001). Wavelet entropy in event-related potentials: A new method shows ordering of EEG oscillations. Biological Cybernetics, 84(4), 291-299.CrossrefGoogle Scholar

  • [19] Woo, M.A., Stevenson, W.G., Moser, D.K., Trelease, R.B., Harper, R.H. (1992). Patterns of beat-to-beat heart rate variability in advanced heart failure. American Heart Journal, 123, 704-710.Google Scholar

  • [20] Kamen, P.W., Krum, H., Tonkin, A.M. (1996). Poincare plot of heart rate variability allows quantitative display of parasympathetic nervous activity. Clinical Science, 92, 201-208.Google Scholar

  • [21] Kamen, P.W. (1996). Heart rate variability. Australian Family Physician, 25, 1087-1094.Google Scholar

  • [22] Brennan, M., Palaniswami, M., Kamen, P. (2001). Do existing measures of poincare plot geometry reflect nonlinear features of heart rate variability? IEEE Transactions on Biomedical Engineering, 48(11), 1342-1347.CrossrefGoogle Scholar

  • [23] Huikuri, H.V., Makikallio, T.H., Peng, C.K., Goldberger, A.L., Hintze, U., Moller, M. (2000). Fractal correlation properties of R-R interval dynamics and mortality in patients with depressed left ventricular function after an acute myocardial infarction. Circulation, 101, 47-53.Google Scholar

  • [24] Acharya, U.R., Kannathal, N., Seng, O.W., Ping, L.Y., Chua, T. (2004). Heart rate analysis in normal subjects of various age groups. BioMedical Engineering OnLine, 3(24).CrossrefGoogle Scholar

  • [25] Caminal, P., Vallverdu, M., Giraldo, B., Benito, S., Vazquez, G., Voss, A. (2005). Optimized symbolic dynamics approach for the analysis of the respiratory pattern. IEEE Transactions on Biomedical Engineering, 52(11), 1832-1839.CrossrefGoogle Scholar

  • [26] Xu, J.H., Liu, Z.R., Liu, R. (1994). The measures of sequence complexity for EEG studies. Chaos, 4(11), 2111-2119.Google Scholar

  • [27] Richman, J.S., Randall, M.J. (2000). Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology - Heart and Circulatory Physiology, 278, H2039-H2049.Google Scholar

  • [28] Akgul, A. (2003). Tibbi Arastirmalarda Istatistiksel Analiz Teknikleri: SPSS Uygulamalari (Statistical Analysis Techniques in Medical Researches: SPSS Experiments). 2nd edition, Ankara, Turkey, Seckin Yayincilik.Google Scholar

  • [29] Kohavi, R. (1995). A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In 14th International Joint Conference on Artificial Intelligence (IJCAI), 20-25 August 1995, Montreal, Quebec, Canada. Google Scholar

About the article

Received: 2014-08-18

Accepted: 2015-08-05

Published Online: 2015-08-27

Published in Print: 2015-08-01

Citation Information: Measurement Science Review, Volume 15, Issue 4, Pages 196–201, ISSN (Online) 1335-8871, DOI: https://doi.org/10.1515/msr-2015-0027.

Export Citation

© by Yalcin Isler. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Okan Erkaymaz, Mahmut Ozer, and Matjaž Perc
Applied Mathematics and Computation, 2017, Volume 311, Page 22
A. Iduseri and J. E. Osemwenkhae
Annals of Data Science, 2015, Volume 2, Number 4, Page 489

Comments (0)

Please log in or register to comment.
Log in