Experienced physicians must rapidly identify ill pediatric patients. We evaluated the ability of an illness rating score (IRS) to predict admission to a pediatric hospital and explored the underlying clinical reasoning of the gestalt assessment of illness.
We used mixed-methods to study pediatric emergency medicine physicians at an academic children’s hospital emergency department (ED). Physicians rated patients’ illness severity with the IRS, anchored by 0 (totally well) and 10 (critically ill), and shared their rationale with concurrent think-aloud responses. The association between IRS and need for hospitalization, respiratory support, parenteral antibiotics, and resuscitative intravenous (IV) fluids were analyzed with mixed effects linear regression. Area under the curve (AUC) receiver operator characteristic (ROC) curve and test characteristics at different cut-points were calculated for IRS as a predictor of admission. Think-aloud responses were qualitatively analyzed via inductive process.
A total of 141 IRS were analyzed (mean 3.56, SD 2.30, range 0–9). Mean IRS were significantly higher for patients requiring admission (4.32 vs. 3.13, p<0.001), respiratory support (6.15 vs. 3.98, p = 0.033), IV fluids (4.53 vs. 3.14, p < 0.001), and parenteral antibiotics (4.68 vs. 3.32, p = 0.009). AUC for IRS as a predictor of admission was 0.635 (95% CI: 0.534–0.737). Analysis of 95 think-aloud responses yielded eight categories that describe the underlying clinical reasoning.
Rapid assessments as captured by the IRS differentiated pediatric patients who required admission and medical interventions. Think-aloud responses for the rationale for rapid assessments may form the basis for teaching the skill of identifying ill pediatric patients.
Research funding: None declared.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: Authors state no conflict of interest.
Informed consent: Informed consent was obtained from all individuals included in this study.
Ethical approval: The study survey and protocol involving human subjects (Pro00011729) were submitted to the Institutional Review Board (IRB) at Children’s National Hospital and granted approval on July 24, 2019.
Previous presentations: The quantitative data were presented in written and oral abstract presentation form at the following conferences: Pediatric Academic Society (PAS) Annual Conference, April 2020, Virtual; Pediatric Hospital Medicine (PHM) Annual Conference, July 2020, Virtual; Society for Diagnosis in Medicine (SIDM) Annual Conference, October 2020, Virtual.
1. Spaeder, MC, Moorman, JR, Tran, CA, Keim-Malpass, J, Zschaebitz, JV, Lake, DE, et al.. Predictive analytics in the pediatric intensive care unit for early identification of sepsis: capturing the context of age. Pediatr Res 2019;86:655–61. https://doi.org/10.1038/s41390-019-0518-1.Search in Google Scholar
2. Topjian, AA, Berg, RA, Nadkarni, VM. Advances in recognition, resuscitation, and stabilization of the critically ill child. Pediatr Clin North Am 2013;60:605–20. https://doi.org/10.1016/j.pcl.2013.02.014.Search in Google Scholar
4. American Heart Association. 2005 American Heart Association (AHA) guidelines for cardiopulmonary resuscitation (CPR) and emergency cardiovascular care (ECC) of pediatric and neonatal patients: pediatric basic life support. Pediatrics 2006;117:e989–1004. https://doi.org/10.1542/peds.2006-0219.Search in Google Scholar PubMed
5. Corrales, AY, Starr, M. Assessment of the unwell child. Aust Fam Physician 2010;39:270–5.Search in Google Scholar
6. Mandell, IM, Bynum, F, Marshall, L, Bart, R, Gold, JI, Rubin, S. Pediatric Early Warning Score and unplanned readmission to the pediatric intensive care unit. J Crit Care 2015;30:1090–5. https://doi.org/10.1016/j.jcrc.2015.06.019.Search in Google Scholar PubMed
7. Chamberlain, JM. The Pediatric Risk of Hospital Admission Score: a second-generation severity-of-illness score for pediatric emergency patients. Pediatrics 2005;115:388–95. https://doi.org/10.1542/peds.2004-0586.Search in Google Scholar PubMed
8. Gold, DL, Mihalov, LK, Cohen, DM. Evaluating the Pediatric Early Warning Score (PEWS) system for admitted patients in the pediatric emergency department. Walthall J, editor. Acad Emerg Med 2014;21:1249–56. https://doi.org/10.1111/acem.12514.Search in Google Scholar PubMed PubMed Central
9. Sweney, JS, Poss, WB, Grissom, CK, Keenan, HT. Comparison of severity of illness scores to physician clinical judgment for potential use in pediatric critical care triage. Disaster Med Public Health Prep 2012;6:126–30. https://doi.org/10.1001/dmp.2012.17.Search in Google Scholar PubMed
10. Bonafide, CP, Roberts, KE, Weirich, CM, Paciotti, B, Tibbetts, KM, Keren, R, et al.. Beyond statistical prediction: qualitative evaluation of the mechanisms by which pediatric early warning scores impact patient safety: Early Warning Score Qualitative Study. J Hosp Med 2013;8:248–53. https://doi.org/10.1002/jhm.2026.Search in Google Scholar PubMed
11. Akre, M, Finkelstein, M, Erickson, M, Liu, M, Vanderbilt, L, Billman, G. Sensitivity of the pediatric early warning score to identify patient deterioration. Pediatrics 2010;125:e763–9. https://doi.org/10.1542/peds.2009-0338.Search in Google Scholar PubMed
12. Fenix, J, Gillespie, CW, Levin, A, Dean, N. Comparison of pediatric early warning score to physician opinion for deteriorating patients. Hosp Pediatr 2015;5:474–9. https://doi.org/10.1542/hpeds.2014-0199.Search in Google Scholar
13. Bradman, K, Borland, M, Pascoe, E. Predicting patient disposition in a paediatric emergency department: predicting patient disposition in paediatric emergency department. J Paediatr Child Health 2014;50:E39–44. https://doi.org/10.1111/jpc.12011.Search in Google Scholar
14. Rohacek, M, Nickel, CH, Dietrich, M, Bingisser, R. Clinical intuition ratings are associated with morbidity and hospitalisation. Int J Clin Pract 2015;69:710–7. https://doi.org/10.1111/ijcp.12606.Search in Google Scholar
15. Cabrera, D, Thomas, J, Wiswell, J, Walston, J, Anderson, J, Hess, E, et al.. Accuracy of ‘my gut feeling:’ comparing system 1 to system 2 decision-making for acuity prediction, disposition and diagnosis in an academic emergency department. West J Emerg Med 2015;16:653–7. https://doi.org/10.5811/westjem.2015.5.25301.Search in Google Scholar
16. Wiswell, J, Tsao, K, Bellolio, MF, Hess, EP, Cabrera, D. “Sick” or “not-sick”: accuracy of System 1 diagnostic reasoning for the prediction of disposition and acuity in patients presenting to an academic ED. Am J Emerg Med 2013;31:1448–52. https://doi.org/10.1016/j.ajem.2013.07.018.Search in Google Scholar
17. Stolper, E, Van de Wiel, M, Van Royen, P, Van Bokhoven, M, Van der Weijden, T, Dinant, GJ. Gut feelings as a third track in general practitioners’ diagnostic reasoning. J Gen Intern Med 2011;26:197–203. https://doi.org/10.1007/s11606-010-1524-5.Search in Google Scholar
18. Sibbald, M, Sherbino, J, Preyra, I, Coffin-Simpson, T, Norman, G, Monteiro, S. Eyeballing: the use of visual appearance to diagnose ‘sick’. Med Educ 2017;51:1138–45. https://doi.org/10.1111/medu.13396.Search in Google Scholar
19. Baumann, MR. Evaluation of the emergency severity index (version 3) triage algorithm in pediatric patients. Acad Emerg Med 2005;12:219–24. https://doi.org/10.1197/j.aem.2004.09.023.Search in Google Scholar
21. Battista, A, Konopasky, A, Ramani, D, Ohmer, M, Mikita, J, Howle, A, et al.. Clinical reasoning in the primary care setting: two scenario-based simulations for residents and attendings. MedEdPORTAL 2018;14:10773. https://doi.org/10.15766/mep_2374-8265.10773.Search in Google Scholar PubMed PubMed Central
22. Ohmer, M, Durning, SJ, Kucera, W, Nealeigh, M, Ordway, S, Mellor, T, et al.. Clinical reasoning in the ward setting: a rapid response scenario for residents and attendings. MedEdPORTAL 2019;15:10834. https://doi.org/10.15766/mep_2374-8265.10834.Search in Google Scholar PubMed PubMed Central
23. Harris, PA, Taylor, R, Thielke, R, Payne, J, Gonzalez, N, Conde, JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. https://doi.org/10.1016/j.jbi.2008.08.010.Search in Google Scholar PubMed PubMed Central
24. Feudtner, C, Feinstein, JA, Zhong, W, Hall, M, Dai, D. Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation [Internet]. BMC Pediatr 2014;14:199. https://doi.org/10.1186/1471-2431-14-199.Search in Google Scholar PubMed PubMed Central
25. Saldaña, J. The coding manual for qualitative researchers, 2nd ed. Los Angeles: SAGE; 2013:303 p.Search in Google Scholar
27. Van den Bruel, A, Thompson, M, Buntinx, F, Mant, D. Clinicians’ gut feeling about serious infections in children: observational study. BMJ 2012;345:e6144. https://doi.org/10.1136/bmj.e6144.Search in Google Scholar PubMed PubMed Central
28. Walsh, P, Thornton, J, Asato, J, Walker, N, McCoy, G, Baal, J, et al.. Approaches to describing inter-rater reliability of the overall clinical appearance of febrile infants and toddlers in the emergency department. PeerJ 2014;2:e651. https://doi.org/10.7717/peerj.651.Search in Google Scholar PubMed PubMed Central
30. Jauregui, J, Nelson, D, Choo, E, Stearns, B, Levine, AC, Liebmann, O, et al.. The BUDDY (Bedside Ultrasound to Detect Dehydration in Youth) study. Crit Ultrasound J 2014;6:15. https://doi.org/10.1186/s13089-014-0015-z.Search in Google Scholar PubMed PubMed Central
31. Bowen, L, Shaw, A, Lyttle, MD, Purdy, S. The transition to clinical expert: enhanced decision making for children aged less than 5 years attending the paediatric ED with acute respiratory conditions. Emerg Med J 2017;34:76–81. https://doi.org/10.1136/emermed-2015-205211.Search in Google Scholar PubMed PubMed Central
32. Douw, G, Schoonhoven, L, Holwerda, T, Huisman-de Waal, G, van Zanten, ARH, van Achterberg, T, et al.. Nurses’ worry or concern and early recognition of deteriorating patients on general wards in acute care hospitals: a systematic review. Crit Care 2015;19:230. https://doi.org/10.1186/s13054-015-0950-5.Search in Google Scholar PubMed PubMed Central
33. Fernández, A, Ares, MI, Garcia, S, Martinez-Indart, L, Mintegi, S, Benito, J. The validity of the Pediatric Assessment Triangle as the first step in the triage process in a pediatric emergency department. Pediatr Emerg Care 2017;33:234–8. https://doi.org/10.1097/PEC.0000000000000717.Search in Google Scholar PubMed
34. Horeczko, T, Enriquez, B, McGrath, NE, Gausche-Hill, M, Lewis, RJ. The Pediatric Assessment Triangle: accuracy of its application by nurses in the triage of children. J Emerg Nurs 2013;39:182–9. https://doi.org/10.1016/j.jen.2011.12.020.Search in Google Scholar PubMed PubMed Central
35. Kestner, V, Vogeley, E, Pitetti, RD, Sun, Q, Parker, KN, Maloney, L. A comparison of parental and nursing assessments of level of illness or injury in a pediatric emergency department. Pediatr Emerg Care 2009;25:633–5. https://doi.org/10.1097/pec.0b013e3181b9201d.Search in Google Scholar PubMed
37. Vanstone, M, Monteiro, S, Colvin, E, Norman, G, Sherbino, J, Sibbald, M, et al.. Experienced physician descriptions of intuition in clinical reasoning: a typology. Diagnosis (Berl). 2019;6:259–68. https://doi.org/10.1515/dx-2018-0069.Search in Google Scholar PubMed
38. Berry, JG, Hall, M, Hall, DE, Kuo, DZ, Cohen, E, Agrawal, R, et al.. Inpatient growth and resource use in 28 children’s hospitals: a longitudinal, multi-institutional study. JAMA Pediatr 2013;167:170. https://doi.org/10.1001/jamapediatrics.2013.432.Search in Google Scholar PubMed PubMed Central
39. Murphy, NA, Alvey, J, Valentine, KJ, Mann, K, Wilkes, J, Clark, EB. Children with medical complexity: the 10-year experience of a single center. Hosp Pediatr 2020;10:702–8. https://doi.org/10.1542/hpeds.2020-0085.Search in Google Scholar PubMed
41. Dosa, NP, Boeing, NM, Ms, N, Kanter, RK. Excess risk of severe acute illness in children with chronic health conditions. Pediatrics 2001;107:499–504. https://doi.org/10.1542/peds.107.3.499.Search in Google Scholar PubMed
42. Chan, T, Rodean, J, Richardson, T, Farris, RWD, Bratton, SL, Di Gennaro, JL, et al.. Pediatric critical care resource use by children with medical complexity. J Pediatr 2016;177:197–203.e1. https://doi.org/10.1016/j.jpeds.2016.06.035.Search in Google Scholar PubMed
43. Rudolf, F, Oyama, LC, Schwartz, K, Fernandez, JA, Hayden, SR. Teaching rapid assessment skills in triage for the emergency medicine clerkship. J Emerg Med 2021;61:76–81.10.1016/j.jemermed.2021.02.005Search in Google Scholar PubMed
44. Seki, M, Otaki, J, Breugelmans, R, Komoda, T, Nagata-Kobayashi, S, Akaishi, Y, et al.. How do case presentation teaching methods affect learning outcomes?-SNAPPS and the One-Minute preceptor. BMC Med Educ 2016;16:12. https://doi.org/10.1186/s12909-016-0531-6.Search in Google Scholar PubMed PubMed Central
45. Natesan, S, Bailitz, J, King, A, Krzyzaniak, S, Kennedy, S, Kim, A, et al.. Clinical teaching: an evidence-based guide to best practices from the council of emergency medicine residency directors. West J Emerg Med 2020;21:985–98. https://doi.org/10.5811/westjem.2020.4.46060.Search in Google Scholar PubMed PubMed Central
46. Sokol, K. Modifying the one-minute preceptor model for use in the emergency department with a critically ill patient. J Emerg Med 2017;52:368–9. https://doi.org/10.1016/j.jemermed.2016.11.030.Search in Google Scholar PubMed
47. Ravichandran, L, Sivaprakasam, E, Balaji, S, Swaminathan, N. Effectiveness of the 1-minute preceptor on feedback to pediatric residents in a busy ambulatory setting. J Grad Med Educ 2019;11:204–6. https://doi.org/10.4300/jgme-d-19-00440.Search in Google Scholar PubMed PubMed Central
© 2021 Walter de Gruyter GmbH, Berlin/Boston