Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter February 25, 2021

Antecedent treat-and-release diagnoses prior to sepsis hospitalization among adult emergency department patients: a look-back analysis employing insurance claims data using Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) methodology

  • Najlla Nassery ORCID logo , Michael A. Horberg ORCID logo EMAIL logo , Kevin B. Rubenstein ORCID logo , Julia M. Certa ORCID logo , Eric Watson , Brinda Somasundaram , Ejaz Shamim , Jennifer L. Townsend , Panagis Galiatsatos , Samantha I. Pitts , Ahmed Hassoon and David E. Newman-Toker
From the journal Diagnosis



The aim of this study was to identify delays in early pre-sepsis diagnosis in emergency departments (ED) using the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) approach.


SPADE methodology was employed using electronic health record and claims data from Kaiser Permanente Mid-Atlantic States (KPMAS). Study cohort included KPMAS members ≥18 years with ≥1 sepsis hospitalization 1/1/2013–12/31/2018. A look-back analysis identified treat-and-release ED visits in the month prior to sepsis hospitalizations. Top 20 diagnoses associated with these ED visits were identified; two diagnosis categories were distinguished as being linked to downstream sepsis hospitalizations. Observed-to-expected (O:E) and temporal analyses were performed to validate the symptom selection; results were contrasted to a comparison group. Demographics of patients that did and did not experience sepsis misdiagnosis were compared.


There were 3,468 sepsis hospitalizations during the study period and 766 treat-and-release ED visits in the month prior to hospitalization. Patients discharged from the ED with fluid and electrolyte disorders (FED) and altered mental status (AMS) were most likely to have downstream sepsis hospitalizations (O:E ratios of 2.66 and 2.82, respectively). Temporal analyses revealed that these symptoms were overrepresented and temporally clustered close to the hospitalization date. Approximately 2% of sepsis hospitalizations were associated with prior FED or AMS ED visits.


Treat-and-release ED encounters for FED and AMS may represent harbingers for downstream sepsis hospitalizations. The SPADE approach can be used to develop performance measures that identify pre-sepsis.

Corresponding author: Michael A. Horberg, MD, MAS, Kaiser Permanente Mid-Atlantic States, Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, 2101 East Jefferson St., 3-East, Rockville, MD 20852, USA; and Kaiser Permanente Mid-Atlantic States, Mid-Atlantic Permanente Medical Group, Department of Infectious Diseases, Rockville, MD, USA, Phone: +1 301 852 9307, E-mail:
Najlla Nassery and Michael A. Horberg contributed equally to this work and wish to be listed as co-first authors.

Award Identifier / Grant number: GBMF5756


We would like to thank Carla V. Rodriguez-Watson, PhD (formerly of MAPRI); Richard Rothman, MD PhD (Johns Hopkins); Zheyu Wang, PhD (Johns Hopkins); and Yuxin Zhu, PhD (Johns Hopkins) for their contributions to this project, and the patients of Kaiser Permanente for helping us improve care using information collected via the electronic health record system.

  1. Research funding: This journal article was supported by a sub-agreement from the Johns Hopkins University with funds provided by Grant No. #GBMF5756 from the Gordon and Betty Moore Foundation. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Gordon and Betty Moore Foundation or the Johns Hopkins University. Dr. Newman-Toker’s effort was supported partly by the Armstrong Institute Center for Diagnostic Excellence at the Johns Hopkins University School of Medicine.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Dr. Newman-Toker conducts research related to diagnostic error, including serving as the principal investigator for grants on this topic. He serves as an unpaid member of the Board of Directors of the Society to Improve Diagnosis in Medicine and as its current President. He serves as a medico-legal consultant for both plaintiff and defense in cases related to diagnostic error. There are no other conflicts of interest. None of the authors have any financial or personal relationships with other people or organizations that could inappropriately influence (bias) their work.

  4. Informed consent and ethical approval: The KPMAS Institutional Review Board deemed this study exempt from review and waived the informed consent requirement.


1. Newman-Toker, DE, Schaffer, AC, Yu-Moe, CW, Nassery, N, Saber Tehrani, AS, Clemens, GD, et al.. Serious misdiagnosis-related harms in malpractice claims: the “Big Three” – vascular events, infections, and cancers. Diagnosis (Berl) 2019;6:227–40. in Google Scholar PubMed

2. Singh, H, Graber, ML, Kissam, SM, Sorensen, AV, Lenfestey, NF, Tant, EM, et al.. System-related interventions to reduce diagnostic errors: a narrative review. BMJ Qual Saf 2012;21:160–70. in Google Scholar PubMed PubMed Central

3. National Academies of Sciences Engineering and Medicine. Improving diagnosis in health care. In: Balogh, EP, Miller, BT, Ball, JR, editors. Washington DC: The National Academies Press; 2015.Search in Google Scholar

4. Graber, ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013;22(Suppl 2):ii21–ii7. in Google Scholar PubMed PubMed Central

5. Newman-Toker, DE. A unified conceptual model for diagnostic errors: underdiagnosis, overdiagnosis, and misdiagnosis. Diagnosis (Berl) 2014;1:43–8. in Google Scholar PubMed PubMed Central

6. Newman-Toker, DE, Wang, Z, Zhu, Y, Nassery, N, Saber Tehrani, AS, Schaffer, AC, et al.. Rate of diagnostic errors and serious misdiagnosis-related harms for major vascular events, infections, and cancers: toward a national incidence estimate using the “Big Three”. Diagnosis (Berl) 2020;8:67–84. in Google Scholar PubMed

7. Newman-Toker, DE, Moy, E, Valente, E, Coffey, R, Hines, AL. Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample. Diagnosis (Berl) 2014;1:155–66. in Google Scholar PubMed PubMed Central

8. Liberman, AL, Newman-Toker, DE. Symptom-Disease Pair Analysis of Diagnostic Error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data. BMJ Qual Saf 2018;27:557–66. in Google Scholar PubMed PubMed Central

9. Mane, KK, Rubenstein, KB, Nassery, N, Sharp, AL, Shamim, EA, Sangha, NS, et al.. Diagnostic performance dashboards: tracking diagnostic errors using big data. BMJ Qual Saf 2018;27:567–70. in Google Scholar PubMed

10. Sharp, AL, Baecker, A, Nassery, N, Park, S, Hassoon, A, Lee, MS, et al.. Missed acute myocardial infarction in the emergency department-standardizing measurement of misdiagnosis-related harms using the SPADE method. Diagnosis (Berl) 2020. in Google Scholar PubMed

11. Singer, M, Deutschman, CS, Seymour, CW, Shankar-Hari, M, Annane, D, Bauer, M, et al.. The third international consensus definitions for sepsis and septic shock (Sepsis-3). J Am Med Assoc 2016;315:801–10. in Google Scholar PubMed PubMed Central

12. Rhee, C, Dantes, R, Epstein, L, Murphy, DJ, Seymour, CW, Iwashyna, TJ, et al.. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014. J Am Med Assoc 2017;318:1241–9. in Google Scholar PubMed PubMed Central

13. Liu, V, Escobar, GJ, Greene, JD, Soule, J, Whippy, A, Angus, DC, et al.. Hospital deaths in patients with sepsis from 2 independent cohorts. J Am Med Assoc 2014;312:90–2. in Google Scholar PubMed

14. Martin, GS, Mannino, DM, Eaton, S, Moss, M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003;348:1546–54. in Google Scholar

15. Gaieski, DF, Edwards, JM, Kallan, MJ, Carr, BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med 2013;41:1167–74. in Google Scholar

16. Rhee, C, Kadri, S, Huang, SS, Murphy, MV, Li, L, Platt, R, et al.. Objective sepsis surveillance using electronic clinical data. Infect Control Hosp Epidemiol 2016;37:163–71. in Google Scholar PubMed PubMed Central

17. Paoli, CJ, Reynolds, MA, Sinha, M, Gitlin, M, Crouser, E. Epidemiology and costs of sepsis in the United States-an analysis based on timing of diagnosis and severity level. Crit Care Med 2018;46:1889–97. in Google Scholar

18. Angus, DC, Linde-Zwirble, WT, Lidicker, J, Clermont, G, Carcillo, J, Pinsky, MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001;29:1303–10. in Google Scholar PubMed

19. Dombrovskiy, VY, Martin, AA, Sunderram, J, Paz, HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med 2007;35:1244–50. in Google Scholar PubMed

20. Wang, HE, Shapiro, NI, Angus, DC, Yealy, DM. National estimates of severe sepsis in United States emergency departments. Crit Care Med 2007;35:1928–36. in Google Scholar PubMed

21. Iwashyna, TJ, Odden, A, Rohde, J, Bonham, C, Kuhn, L, Malani, P, et al.. Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care 2014;52:e39–43. in Google Scholar

22. Klompas, M, Osborn, TM, Rhee, C. Who owns sepsis?. Ann Intern Med 2020;172:210–1. in Google Scholar PubMed

23. Kumar, A, Roberts, D, Wood, KE, Light, B, Parrillo, JE, Sharma, S, et al.. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34:1589–96. in Google Scholar

24. Peltan, ID, Brown, SM, Bledsoe, JR, Sorensen, J, Samore, MH, Allen, TL, et al.. ED door-to-antibiotic time and long-term mortality in sepsis. Chest 2019;155:938–46.10.1016/j.chest.2019.02.008Search in Google Scholar PubMed PubMed Central

25. Jolley, RJ, Quan, H, Jette, N, Sawka, KJ, Diep, L, Goliath, J, et al.. Validation and optimisation of an ICD-10-coded case definition for sepsis using administrative health data. BMJ Open 2015;5:e009487. in Google Scholar PubMed PubMed Central

26. Vaillancourt, S, Guttmann, A, Li, Q, Chan, IY, Vermeulen, MJ, Schull, MJ. Repeated emergency department visits among children admitted with meningitis or septicemia: a population-based study. Ann Emerg Med 2015;65:625–32 e3. in Google Scholar PubMed

27. Cifra, CL, Westlund, E, Eyck, PT, Ward, MM, Mohr, NM, Katz, DA. An estimate of missed pediatric sepsis in the emergency department. Diagnosis 2020:20200023. in Google Scholar PubMed PubMed Central

28. Newman-Toker, DE, Pronovost, PJ. Diagnostic errors--the next frontier for patient safety. JAMA 2009;301:1060–2. in Google Scholar PubMed

29. Kaiser Permanente. About us: Mid-Atlantic States fast facts 2019. [updated September 30, 2019. Available from: in Google Scholar

30. Kaiser Foundation Health Plan of Mid-Atlantic States Inc. KPMAS equity, inclusion, & diversity annual report with language analysis supplements: October 1, 2017 - September 31, 2018; 2019.Search in Google Scholar

31. Bartolome, RE, Chen, A, Handler, J, Platt, ST, Gould, B. Population care management and team-based approach to reduce racial disparities among African Americans/Blacks with hypertension. Perm J 2016;20:53–9. in Google Scholar PubMed PubMed Central

32. Rhee, C, Murphy, MV, Li, L, Platt, R, Klompas, M. Comparison of trends in sepsis incidence and coding using administrative claims versus objective clinical data. Clin Infect Dis 2015;60:88–95. in Google Scholar PubMed PubMed Central

33. Prescott, HC, Cope, TM, Gesten, FC, Ledneva, TA, Friedrich, ME, Iwashyna, TJ, et al.. Reporting of sepsis cases for performance measurement versus for reimbursement in New York State. Crit Care Med 2018;46:666–73. in Google Scholar PubMed PubMed Central

34. Rhee, C, Jentzsch, MS, Kadri, SS, Seymour, CW, Angus, DC, Murphy, DJ, et al.. Variation in identifying sepsis and organ dysfunction using administrative versus electronic clinical data and impact on hospital outcome comparisons. Crit Care Med 2019;47:493–500. in Google Scholar PubMed PubMed Central

35. Shahraz, S, Lagu, T, Ritter, GA, Liu, X, Tompkins, C. Use of systematic methods to improve disease identification in administrative data: the case of severe sepsis. Med Care 2017;55:e16–24. in Google Scholar PubMed PubMed Central

36. Gardlund, B, Dmitrieva, NO, Pieper, CF, Finfer, S, Marshall, JC, Taylor Thompson, B. Six subphenotypes in septic shock: latent class analysis of the PROWESS shock study. J Crit Care 2018;47:70–9. in Google Scholar PubMed PubMed Central

37. HCUP Clinical Classifications Software (CCS) for ICD-9-CM. Healthcare Cost and Utilization Project (HCUP). 2013-2015. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ). [Accessed 1 June 2019].Search in Google Scholar

38. HCUP Clinical Classifications Software (CCS) for ICD-10-CM. Healthcare Cost and Utilization Project (HCUP). 2015-2018. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ). [Accessed 1 June 2019].Search in Google Scholar

39. Aragon, TJF, Michael, P, Wollschlaeger, D, Omidpanah, A. Epitools: Epidemiology Tools. 0.5-10.1 ed2020. p. R package.Search in Google Scholar

40. Moore, BJ, White, S, Washington, R, Coenen, N, Elixhauser, A. Identifying increased risk of readmission and in-hospital mortality using hospital administrative data: the AHRQ Elixhauser comorbidity index. Med Care 2017;55:698–705. in Google Scholar PubMed

41. Von Elm, E, Altman, DG, Egger, M, Pocock, SJ, Gøtzsche, PC, Vandenbroucke, JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007;147:573–7. in Google Scholar PubMed

42. Clifford, KM, Dy-Boarman, EA, Haase, KK, Maxvill, K, Pass, SE, Alvarez, CA. Challenges with diagnosing and managing sepsis in older adults. Expert Rev Anti Infect Ther 2016;14:231–41. in Google Scholar PubMed PubMed Central

Supplementary Material

The online version of this article offers supplementary material (

Received: 2020-10-22
Accepted: 2021-02-01
Published Online: 2021-02-25
Published in Print: 2021-11-25

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 24.2.2024 from
Scroll to top button