A comprehensive evaluation for the prediction of mortality in intensive care units with LSTM networks: patients with cardiovascular disease

Saumil Maheshwari 1 , Aman Agarwal 1 , Anupam Shukla 1  and Ritu Tiwari 1
  • 1 Soft Computing and Expert System Laboratory, ABV-Indian Institute of Information Technology and Management, Gwalior 474010, Madhya Pradesh, India
Saumil Maheshwari
  • Corresponding author
  • Soft Computing and Expert System Laboratory, ABV-Indian Institute of Information Technology and Management, Gwalior 474010, Madhya Pradesh, India
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Aman Agarwal
  • Soft Computing and Expert System Laboratory, ABV-Indian Institute of Information Technology and Management, Gwalior 474010, Madhya Pradesh, India
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Anupam Shukla
  • Soft Computing and Expert System Laboratory, ABV-Indian Institute of Information Technology and Management, Gwalior 474010, Madhya Pradesh, India
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Ritu Tiwari
  • Soft Computing and Expert System Laboratory, ABV-Indian Institute of Information Technology and Management, Gwalior 474010, Madhya Pradesh, India
  • Search for other articles:
  • degruyter.comGoogle Scholar

Abstract

Intensive care units (ICUs) are responsible for generating a wealth of useful data in the form of electronic health records. We aimed to build a mortality prediction model on a Medical Information Mart for Intensive Care (MIMIC-III) database and to assess whether the use of deep learning techniques like long short-term memory (LSTM) can effectively utilize the temporal relations among clinical variables. The models were built on clinical variable dynamics of the first 48 h of ICU admission of 12,550 records from the MIMIC-III database. A total of 36 variables including 33 time series variables and three static variables were used for the prediction. We present the application of LSTM and LSTM attention (LSTM-AT) model for mortality prediction with such a large number of clinical variables dataset. For training and validation purpose, we have used International Classification of Diseases, 9th edition (ICD-9) codes for extracting the patients with cardiovascular disease, and infections and parasitic disease, respectively. The effectiveness of the LSTM model is achieved over non-recurrent baseline models like naïve Bayes, logistic regression (LR), support vector machine and multilayer perceptron (MLP) by generating state of the art results (area under the curve [AUC], 0.852). Next, by providing attention at each time stamp, we developed a model, LSTM-AT, which exhibits even better performance (AUC, 0.876).

  • [1]

    Global Health Estimates 2015: Deaths by Cause, Age, Sex, by Country and by Region, 2000–2015. Geneva: World Health Organization; 2016 [cited 2018 August 25]. Available from: https://www.who.int/healthinfo/global_burden_disease/estimates/en/.

  • [2]

    Heidenreich PA, Trogdon JG, Khavjou OA, Butler J, Dracup K, Ezekowitz MD, et al. Forecasting the future of cardiovascular disease in the United States: a policy statement from the American Heart Association. Circulation 2011;123:933–44.

    • Crossref
    • PubMed
    • Export Citation
  • [3]

    Le JG, Loirat P, Alperovitch A, Glaser P, Granthil C, Mathieu D, et al. A simplified acute physiology score for ICU patients. Crit Care Med 1984;12:975–7.

    • Crossref
    • PubMed
    • Export Citation
  • [4]

    Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981;9:591–7.

    • Crossref
    • PubMed
    • Export Citation
  • [5]

    Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. J Am Med Assoc 1993;270:2957–63.

    • Crossref
    • Export Citation
  • [6]

    Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818–29.

    • Crossref
    • PubMed
    • Export Citation
  • [7]

    Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, et al. The APACHE III prognostic system: risk prediction of hospital mortality for critically III hospitalized adults. Chest 1991;100:1619–36.

    • Crossref
    • PubMed
    • Export Citation
  • [8]

    Poole D, Rossi C, Anghileri A, Giardino M, Latronico N, Radrizzani D, et al. External validation of the Simplified Acute Physiology Score (SAPS) 3 in a cohort of 28,357 patients from 147 Italian intensive care units. Intens Care Med 2009;35:1916–24.

    • Crossref
    • Export Citation
  • [9]

    Katsaragakis S, Papadimitropoulos K, Antonakis P, Strergiopoulos S, Konstadoulakis MM, Androulakis G. Comparison of Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score II (SAPS II) scoring systems in a single Greek intensive care unit. Crit Care Med 2000;28:426–32.

    • Crossref
    • Export Citation
  • [10]

    Beck DH, Smith GB, Pappachan JV, Millar B. External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intens Care Med 2003;29:249–56.

    • Crossref
    • Export Citation
  • [11]

    Nassar Jr AP, Mocelin AO, Nunes AL, Giannini FP, Brauer L, Andrade FM, et al. Caution when using prognostic models: a prospective comparison of 3 recent prognostic models. J Crit Care 2012;27:423.e1–7.

    • Crossref
    • Export Citation
  • [12]

    Clermont G, Angus DC, DiRusso SM, Griffin M, Linde-Zwirble WT. Predicting hospital mortality for patients in the intensive care unit: a comparison of artificial neural networks with logistic regression models. Crit Care Med 2001;29:291–6.

    • Crossref
    • PubMed
    • Export Citation
  • [13]

    Nguile-Makao M, Zahar JR, Français A, Tabah A, Garrouste-Orgeas M, Allaouchiche B, et al. Attributable mortality of ventilator-associated pneumonia: respective impact of main characteristics at ICU admission and VAP onset using conditional logistic regression and multi-state models. Intens Care Med 2010;36:781–9.

    • Crossref
    • Export Citation
  • [14]

    Rosenberg AL. Recent innovations in intensive care unit risk-prediction models. Curr Opin Crit Care 2002;8:321–30.

    • Crossref
    • PubMed
    • Export Citation
  • [15]

    Dybowski R, Gant V, Weller P, Chang R. Prediction of outcome in critically ill patients using artificial neural network synthesised by genetic algorithm. Lancet 1996;347:1146–50.

    • Crossref
    • PubMed
    • Export Citation
  • [16]

    Delen D, Walker G, Kadam A. Predicting breast cancer survivability: a comparison of three data mining methods. Artif Intell Med 2005;34:113–27.

    • Crossref
    • PubMed
    • Export Citation
  • [17]

    Sierra B, Serrano N, LarrañAga P, Plasencia EJ, Inza I, JiméNez JJ, et al. Using bayesian networks in the construction of a bi-level multi-classifier. A case study using intensive care unit patients data. Artif Intell Med 2001;22:233–48.

    • Crossref
    • PubMed
    • Export Citation
  • [18]

    Vieira SM, Mendonça LF, Farinha GJ, Sousa JM. Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients. Appl Soft Comput 2013;13:3494–504.

    • Crossref
    • Export Citation
  • [19]

    Liu J, Chen XX, Fang L, Li JX, Yang T, Zhan Q, et al. Mortality prediction based on imbalanced high-dimensional ICU big data. Comput Indust 2018;8:218–25.

  • [20]

    Bengio Y. Learning deep architectures for AI. Found Trends Mach Learn. 2009;2:1–127.

    • Crossref
    • Export Citation
  • [21]

    Wu P, Hoi SC, Xia H, Zhao P, Wang D, Miao C. Online multimodal deep similarity learning with application to image retrieval. In: Proceedings of the 21st ACM International Conference on Multimedia, Barcelona, Spain — October 21–25, 2013. ACM; 2013:153–62.

  • [22]

    Humphrey EJ, Bello JP, LeCun Y. Feature learning and deep architectures: new directions for music informatics. J Intell Inform Syst 2013;41:461–81.

    • Crossref
    • Export Citation
  • [23]

    Huang PS, He X, Gao J, Deng L, Acero A, Heck L. Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, San Francisco, CA, USA. ACM; 2013:2333–38.

  • [24]

    Socher R, Bengio Y, Manning CD. Deep learning for NLP (without magic). In: Tutorial Abstracts of ACL. Jeju Island, Korea: Association for Computational Linguistics; 2012, pp. 5.

  • [25]

    Hinton G, Deng L, Yu D, Dahl G, Mohamed AR, Jaitly N, et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process Mag 2012;29:82–97.

    • Crossref
    • Export Citation
  • [26]

    Choi E, Schuetz A, Stewart WF, Sun J. Using recurrent neural network models for early detection of heart failure onset. J Am Med Informat Assoc 2016;24:361–70.

  • [27]

    Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542:115–8.

    • Crossref
    • PubMed
    • Export Citation
  • [28]

    Che Z, Kale D, Li W, Bahadori MT, Liu Y. Deep computational phenotyping. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia. ACM; 2015:507–16.

  • [29]

    Mokeddem SA. A fuzzy classification model for myocardial infarction risk assessment. Appl Intell 2018;48:1233–50.

  • [30]

    Lipton ZC, Kale DC, Elkan C, Wetzel R. Learning to diagnose with LSTM recurrent neural networks. arXiv preprint arXiv:1511.03677v7. 2015 Nov 11.

  • [31]

    Lipton ZC, Kale DC, Wetzel R. Modeling missing data in clinical time series with RNNs. Mach Learn Healthcare 2016 Jun 13.

  • [32]

    Johnson AE, Pollard TJ, Shen L, Li-wei HL, Feng M, Ghassemi M, et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016;3:160035.

    • Crossref
    • PubMed
    • Export Citation
Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

Biomedical Engineering / Biomedizinische Technik (BMT) is a high-quality forum for the exchange of knowledge in the fields of biomedical engineering, medical information technology and biotechnology/bioengineering. BMT addresses engineers, natural scientists, and clinicians working in research, industry, or clinical practice. All articles are peer-reviewed.

 

Search