Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter December 13, 2021

Evaluation of feedback modalities and preferences regarding feedback on decision-making in a pediatric emergency department

  • Jessica M.K. Graham EMAIL logo , Lilliam Ambroggio , Jan E. Leonard , Sonja I. Ziniel and Joseph A. Grubenhoff
From the journal Diagnosis



To compare pediatric emergency clinicians’ attitudes toward three feedback modalities and assess clinicians’ case-based feedback preferences.


Electronic survey sent to pediatric emergency medicine (PEM) physicians and fellows; general pediatricians; and advanced practice providers (APPs) with nine questions exploring effectiveness and emotional impact of three feedback modalities: case-based feedback, bounce-back notifications, and biannual performance reports. Additional questions used a four-point ordinal agreement response scale and assessed clinicians’ attitudes toward case review notification, case-based feedback preferences, and emotional support. Survey responses were compared by feedback modality using Pearson’s chi-squared.


Of 165 eligible providers, 93 (56%) responded. Respondents agreed that case-based feedback was timely (81%), actionable (75%), prompted reflection on decision-making (92%), prompted research on current clinical practice (53%), and encouraged practice change (58%). Pediatric Emergency Care Applied Research Network (PECARN) performance reports scored the lowest on all metrics except positive feedback. No more than 40% of providers indicated that any feedback modality provided emotional support. Regarding case-based feedback, 88% of respondents desired email notification before case review and 88% desired feedback after case review. Clinicians prefer receiving feedback from someone with similar or more experience/training. Clinicians receiving feedback desire succinctness, supporting evidence, consistency, and sensitive delivery.


Case-based feedback scored highest of the three modalities and is perceived to be the most likely to improve decision-making and promote practice change. Most providers did not perceive emotional support from any feedback modality. Emotional safety warrants purposeful attention in feedback delivery. Critical components of case-based feedback include succinctness, supporting evidence, consistency, and sensitive delivery.

Corresponding author: Jessica M.K. Graham, MD, Physician, Pediatric Emergency Medicine, Children’s Hospital of Colorado, Aurora, CO, USA; 13123 E 16th Avenue, Aurora, CO, 80045, E-mail:

Funding source: ELEVATE Fund for Pediatric Emergency Medicine, supported by The Children’s Hospital Colorado Foundation

Award Identifier / Grant number: 1802


Irina Topoz, M.D., Head of ED Case Review Committee. Lalit Bajaj, M.D., Children’s Hospital Colorado PECARN Hospital Emergency Department Affiliates (HEDA) Principal Investigator.

  1. Research funding: This project was funded by Grant Number: 1802. Elevate: The Children’s Hospital Colorado’s Level 1 Fund to Elevate Pediatric Emergency and Urgent Care,

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Not applicable.

  5. Ethical approval: The local Institutional Review Board deemed the study exempt from review.


1. Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, The National Academies of Sciences, Engineering, and Medicine, Balogh, EP, Miller, BT, Ball, JR, editors. Improving diagnosis in health care. Washington, D.C.: National Academies Press; 2015:21794 p.Search in Google Scholar

2. Grubenhoff, JA, Ziniel, SI, Cifra, CL, Singhal, G, McClead, RE, Singh, H. Pediatric clinician comfort discussing diagnostic errors for improving patient safety: a survey. Pediatr Qual Saf 2020;5:e259. in Google Scholar PubMed PubMed Central

3. Wachter, RM. Why diagnostic errors don’t get any respect—and what can Be done about them. Health Aff 2010;29:1605–10. in Google Scholar PubMed

4. Lavoie, CF, Plint, AC, Clifford, TJ, Gaboury, I. “I never hear what happens, even if they die”: a survey of emergency physicians about outcome feedback. Can J Emerg Med 2009;11:523–8. in Google Scholar PubMed

5. Lavoie, CF, Schachter, H, Stewart, AT, McGowan, J. Does outcome feedback make you a better emergency physician? A systematic review and research framework proposal. Can J Emerg Med 2009;11:545–52. in Google Scholar PubMed

6. Croskerry, P. The feedback sanction. Acad Emerg Med 2000;7:1232–8. in Google Scholar PubMed

7. Schiff, GD. Minimizing diagnostic error: the importance of follow-up and feedback. Am J Med 2008;121:S38–42. in Google Scholar PubMed

8. Graff, L, Stevens, C, Spaite, D, Foody, J. Measuring and improving quality in emergency medicine. Acad Emerg Med 2002;9:1091–107. in Google Scholar

9. Ivers, N, Jamtvedt, G, Flottorp, S, Young, JM, Odgaard-Jensen, J, French, SD, et al.. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259, in Google Scholar

10. McLellan, L, Dornan, T, Newton, P, Williams, SD, Lewis, P, Steinke, D, et al.. Pharmacist-led feedback workshops increase appropriate prescribing of antimicrobials. J Antimicrob Chemother 2016;71:1415–25. in Google Scholar PubMed

11. Lane, KP, Chia, C, Lessing, JN, Limes, J, Mathews, B, Schaefer, J, et al.. Improving resident feedback on diagnostic reasoning after handovers: the LOOP project. J Hosp Med 2019;14:622–5. in Google Scholar PubMed

12. Rogers, RLG, Narvaez, Y, Venkatesh, AK, Fleischman, W, Hall, MK, Taylor, RA, et al.. Improving emergency physician performance using audit and feedback: a systematic review. Am J Emerg Med 2015;33:1505–14. in Google Scholar PubMed

13. Ende, J. Feedback in clinical medical education. J Am Med Assoc 1983;250:777–81. in Google Scholar

14. Richardson, BK. Feedback. Acad Emerg Med 2004;11:e1–5. in Google Scholar PubMed

15. Lu, C-YE, Vinci, LM, Quinn, MT, Wilkes, AE, Chin, MH, Peek, ME. Using feedback to change primary care physician behavior. J Ambul Care Manag 2015;38:118–24. in Google Scholar PubMed PubMed Central

16. Mohanaruban, A, Flanders, L, Rees, H. Case-based discussion: perceptions of feedback. Clin Teach 2018;15:126–31. in Google Scholar PubMed

17. Isbell, LM, Boudreaux, ED, Chimowitz, H, Liu, G, Cyr, E, Kimball, E. What do emergency department physicians and nurses feel? A qualitative study of emotions, triggers, regulation strategies, and effects on patient care. BMJ Qual Saf 2020;29:1–2. in Google Scholar PubMed PubMed Central

18. Martín-Brufau, R, Martin-Gorgojo, A, Suso-Ribera, C, Estrada, E, Capriles-Ovalles, M-E, Romero-Brufau, S. Emotion regulation strategies, workload conditions, and burnout in healthcare residents. Int J Environ Res Publ Health 2020;17:E7816. in Google Scholar PubMed PubMed Central

19. Kluger, AN, DeNisi, A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119:254–84. in Google Scholar

20. Glatzeder, B, Goel, V, Müller, A, editors. Towards a theory of thinking. Berlin, Heidelberg: Springer Berlin Heidelberg; 2010.10.1007/978-3-642-03129-8Search in Google Scholar

21. Bernstein, SL, Aronsky, D, Duseja, R, Epstein, S, Handel, D, Hwang, U, et al.. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med 2009;16:1–10. in Google Scholar PubMed

22. Westbrook, JI, Coiera, E, Dunsmuir, WTM, Brown, BM, Kelk, N, Paoloni, R, et al.. The impact of interruptions on clinical task completion. Qual Saf Health Care 2010;19:284–9. in Google Scholar PubMed

23. Croskerry, P, Singhal, G, Mamede, S. Cognitive debiasing 1: origins of bias and theory of debiasing. BMJ Qual Saf 2013;22:ii58–64. in Google Scholar PubMed PubMed Central

24. Cifra, CL, Tigges, CR, Miller, SL, Herwaldt, LA, Singh, H. Updates to referring clinicians regarding critically ill children admitted to the pediatric intensive care unit: a state-wide survey. Diagnosis (Berl). 2020;7:123–8. in Google Scholar PubMed PubMed Central

25. Gupta, R. The use of outcome feedback by emergency medicine physicians: results of a physician survey. World J Emerg Med 2019;10:14. in Google Scholar PubMed PubMed Central

26. Deakyne Davies, S, Grundmeier, R, Campos, D, Hayes, K, Bell, J, Alessandrini, E, et al.. The Pediatric Emergency Care Applied Research Network Registry: a multicenter electronic health record registry of pediatric emergency care. Appl Clin Inf 2018;09:366–76. in Google Scholar PubMed PubMed Central

27. Bing-You, R, Ramesh, S, Hayes, V, Varaklis, K, Ward, D, Blanco, M. Trainees’ perceptions of feedback: validity evidence for two FEEDME (feedback in medical education) instruments. Teach Learn Med 2018;30:162–72. in Google Scholar PubMed

28. Converse, JM, Presser, S. Survey questions: handcrafting the standardized questionnaire. Thousand Oaks, CA, USA: SAGE Publications; 1986.10.4135/9781412986045Search in Google Scholar

29. Harris, PA, Taylor, R, Thielke, R, Payne, J, Gonzalez, N, Conde, JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inf 2009;42:377–81. in Google Scholar PubMed PubMed Central

30. Montoy, JCC, Tamayo-Sarver, J, Miller, GA, Baer, AE, Peabody, CR. Predicting emergency department “bouncebacks”: a retrospective cohort analysis. West J Emerg Med 2019;20:865–74. in Google Scholar PubMed PubMed Central

31. Pham, JC, Kirsch, TD, Hill, PM, DeRuggerio, K, Hoffmann, B. Seventy-two-hour returns may not be a good indicator of safety in the emergency department: a national study. Acad Emerg Med 2011;18:390–7. in Google Scholar PubMed

32. Seip, B, Frich, JC, Hoff, G. Doctors’ experiences with a quality assurance programme. Clin Govern Int J 2012;17:297–306. in Google Scholar

33. Powell, AA, White, KM, Partin, MR, Halek, K, Hysong, SJ, Zarling, E, et al.. More than a score: a qualitative study of ancillary benefits of performance measurement. BMJ Qual Saf 2014;23:651–8. in Google Scholar

34. Gude, WT, Brown, B, van der Veer, SN, Colquhoun, HL, Ivers, NM, Br ehaut, JC, et al.. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci 2019;14:39. in Google Scholar

35. Fainstad, T, McClintock, AA, Van der Ridder, MJ, Johnston, SS, Patton, KK. Feedback can Be less stressful: medical trainee perceptions of using the prepare to ADAPT (Ask-Discuss-Ask-Plan Together) framework. Cureus 2018;10:e3718. in Google Scholar

36. Telio, S, Ajjawi, R, Regehr, G. The “educational alliance” as a framework for reconceptualizing feedback in medical education. Acad Med 2015;90:609–14. in Google Scholar

37. Gaunt, A, Pawlikowska, T. Physicians actively engaging in seeking feedback for learning through patient outcomes. Med Educ 2018;52:354–6. in Google Scholar

38. McConnell, MM, Eva, KW. The role of emotion in the learning and transfer of clinical skills and knowledge. Acad Med 2012;87:1316–22. in Google Scholar

39. Luu, S, Leung, SOA, Moulton, C. When bad things happen to good surgeons: reactions to adverse events. Surg Clin 2012;92:153–61. in Google Scholar

40. West, CP, Huschka, MM, Novotny, PJ, Sloan, JA, Kolars, JC, Habermann, TM, et al.. Association of perceived medical errors with resident distress and empathy: a prospective longitudinal study. J Am Med Assoc 2006;296:1071–8. in Google Scholar

41. Croskerry, P, Abbass, AA, Wu, AW. How doctors feel: affective issues in patients’ safety. Lancet 2008;372:1205–6. in Google Scholar

Supplementary Material

The online version of this article offers supplementary material (

Received: 2021-08-31
Accepted: 2021-11-17
Published Online: 2021-12-13

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 20.2.2024 from
Scroll to top button