Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter September 21, 2018

“Closing the loop”: a mixed-methods study about resident learning from outcome feedback after patient handoffs

  • Edna C. Shenvi , Stephanie Feudjio Feupe , Hai Yang and Robert El-Kareh EMAIL logo
From the journal Diagnosis



Learning patient outcomes is recognized as crucial for ongoing refinement of clinical decision-making, but is often difficult in fragmented care with frequent handoffs. Data on resident habits of seeking outcome feedback after handoffs are lacking.


We performed a mixed-methods study including (1) an analysis of chart re-access rates after handoffs performed using access logs of the electronic health record (EHR); and (2) a web-based survey sent to internal medicine (IM) and emergency medicine (EM) residents about their habits of and barriers to learning the outcomes of patients after they have handed them off to other teams.


Residents on ward rotations were often able to re-access charts of patients after handoffs, but those on EM or night admitting rotations did so <5% of the time. Among residents surveyed, only a minority stated that they frequently find out the outcomes of patients they have handed off, although learning outcomes was important to both their education and job satisfaction. Most were not satisfied with current systems of learning outcomes of patients after handoffs, citing too little time and lack of reliable patient tracking systems as the main barriers.


Despite perceived importance of learning outcomes after handoffs, residents cite difficulty with obtaining such information. Systematically providing feedback on patient outcomes would meet a recognized need among physicians in training.

Corresponding author: Dr. Robert El-Kareh, MD, MS, MPH UCSD Health Department of Biomedical Informatics, University of California, San Diego, La Jolla, CA, USA, Phone: 858-822-7776

  1. Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

  2. Research funding: Dr. Shenvi and Ms. Feudjio Feupe were supported by the National Library of Medicine training grant T15LM011271, San Diego Biomedical Informatics Education and Research. Dr. El-Kareh was supported by K22LM011435-02, Funder Id: 10.13039/100000092, a career development award from the U.S. National Library of Medicine.

  3. Employment or leadership: None declared.

  4. Honorarium: None declared.

  5. Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.


1. Croskerry P. The feedback sanction. Acad Emerg Med 2000;7:1232–8.10.1111/j.1553-2712.2000.tb00468.xSearch in Google Scholar PubMed

2. Dhaliwal G. Annals for hospitalists inpatient notes-diagnostic excellence starts with an incessant watch. Ann Intern Med 2017;167:HO2–3.10.7326/M17-2447Search in Google Scholar PubMed

3. Lavoie CF, Schachter H, Stewart AT, McGowan J. Does outcome feedback make you a better emergency physician? a systematic review and research framework proposal. CJEM 2009;11:545–52.10.1017/S1481803500011829Search in Google Scholar

4. Rudolph JW, Morrison JB. Sidestepping superstitious learning, ambiguity, and other roadblocks: a feedback model of diagnostic problem solving. Am J Med 2008;121(5 Suppl):S34–7.10.1016/j.amjmed.2008.02.003Search in Google Scholar PubMed

5. Schmidt H, Norman G, Boshuizen H. A cognitive perspective on medical expertise: theory and implication [published erratum appears in Acad Med 1992;67:287]. Acad Med 1990;65:611–21.10.1097/00001888-199010000-00001Search in Google Scholar PubMed

6. Prober CG, Heath C. Lecture halls without lectures – a proposal for medical education. N Engl J Med 2012;366:1657–9.10.1056/NEJMp1202451Search in Google Scholar PubMed

7. Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005;39:418–27.10.1111/j.1365-2929.2005.02127.xSearch in Google Scholar PubMed

8. Schmidt HG, Rikers RM. How expertise develops in medicine: knowledge encapsulation and illness script formation. Med Educ 2007;41:1133–9.10.1111/j.1365-2923.2007.02915.xSearch in Google Scholar PubMed

9. Logio LS, Djuricich AM. Handoffs in teaching hospitals: situation, background, assessment, and recommendation. Am J Med 2010;123:563–7.10.1016/j.amjmed.2010.03.001Search in Google Scholar PubMed

10. Van Eaton EG, Tarpley JL, Solorzano CC, Cho CS, Weber SM, Termuhlen PM. Resident education in 2011: three key challenges on the road ahead. Surgery 2011;149:465–73.10.1016/j.surg.2010.11.007Search in Google Scholar PubMed

11. McNutt R, Johnson T, Kane J, Ackerman M, Odwazny R, Bardhan J. Cost and quality implications of discrepancies between admitting and discharge diagnoses. Qual Manag Health Care 2012;21:220–7.10.1097/QMH.0b013e31826d1ed2Search in Google Scholar PubMed

12. Schiff GD. Minimizing diagnostic error: the importance of follow-up and feedback. Am J Med 2008;121(5 Suppl):S38–42.10.1016/j.amjmed.2008.02.004Search in Google Scholar PubMed

13. Gildfind S, Egerton-Warburton D, Craig S. Remember that patient you saw last week? Emerg Med Australas 2014;26:303–4.10.1111/1742-6723.12234Search in Google Scholar PubMed

14. Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med 2006;355:2217–25.10.1056/NEJMra054782Search in Google Scholar PubMed

15. Dhaliwal G, Detsky AS. The evolution of the master diagnostician. J Am Med Assoc 2013;310:579–80.10.1001/jama.2013.7572Search in Google Scholar PubMed

16. Redelmeier DA. The cognitive psychology of missed diagnoses. Ann Intern Med 2005;142:115–20.10.7326/0003-4819-142-2-200501180-00010Search in Google Scholar PubMed

17. Singh H, Giardina TD, Forjuoh SN, Reis MD, Kosmach S, Khan MM, et al. Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2012;21:93–100.10.1136/bmjqs-2011-000304Search in Google Scholar PubMed PubMed Central

18. Brisson GE, Neely KJ, Tyler PD, Barnard C. Should medical students track former patients in the electronic health record? An emerging ethical conflict. Acad Med 2015;90:1020–4.10.1097/ACM.0000000000000633Search in Google Scholar PubMed

19. Dalseg TR, Calder LA, Lee C, Walker J, Frank JR. Outcome feedback within emergency medicine training programs: an opportunity to apply the theory of deliberate practice? CJEM 2015;17:367–73.10.1017/cem.2014.47Search in Google Scholar PubMed

20. Bowen JL, Ilgen JS, Irby DM, Ten Cate O, O’brien BC. “You have to know the end of the story”: motivations to follow up after transitions of clinical responsibility. Acad Med 2017;92: S48–54.10.1097/ACM.0000000000001919Search in Google Scholar PubMed

21. Lavoie CF, Plint AC, Clifford TJ, Gaboury I. “I never hear what happens, even if they die”: a survey of emergency physicians about outcome feedback. CJEM 2009;11:523–8.10.1017/S1481803500011787Search in Google Scholar PubMed

22. SurveyMonkey, Inc. San Mateo, California, USA. [Available from: in Google Scholar

23. Benjamini Y, Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Ann Stat 2001;29: 1165–88.10.1214/aos/1013699998Search in Google Scholar

24. Team RC. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2013.Search in Google Scholar

25. National Academies of Sciences Engineering, and Medicine. Improving diagnosis in health care. Washington, DC: The National Academies Press, 2015.Search in Google Scholar

26. Dhaliwal G, Shojania KG. The data of diagnostic error: big, large and small. BMJ Qual Saf 2018;27:499–501.10.1136/bmjqs-2018-007917Search in Google Scholar PubMed

27. Narayana S, Rajkomar A, Harrison JD, Valencia V, Dhaliwal G, Ranji SR. What happened to my patient? an educational intervention to facilitate postdischarge patient follow-up. J Grad Med Educ 2017;9:627–33.10.4300/JGME-D-16-00846.1Search in Google Scholar PubMed PubMed Central

Supplementary Material

The online version of this article offers supplementary material (

Article note

Parts of this work were presented as a podium abstract presentation at the Diagnostic Error in Medicine Conference, September 2014, in Atlanta, GA, and at the American Medical Informatics Association Annual Symposium, November 2014, in Washington, DC, USA.

Received: 2018-04-16
Accepted: 2018-08-21
Published Online: 2018-09-21
Published in Print: 2018-11-27

©2018 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 30.11.2023 from
Scroll to top button