Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter November 13, 2020

Identifying trigger concepts to screen emergency department visits for diagnostic errors

  • Prashant Mahajan EMAIL logo , Chih-Wen Pai , Karen S. Cosby , Cynthia J. Mollen , Kathy N. Shaw , James M. Chamberlain , Robert El-Kareh , Richard M. Ruddy , Elizabeth R. Alpern , Helene M. Epstein , Traber D. Giardina , Mark L. Graber , Laura N. Medford-Davis , Richard P. Medlin , Divvy K. Upadhyay , Sarah J. Parker and Hardeep Singh
From the journal Diagnosis



The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm.


We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED.


Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings.


We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Corresponding author: Prashant Mahajan, MD, MPH, MBA Department of Emergency Medicine, University of Michigan 1540 E. Hospital Drive, Ann Arbor, MI 48109-4260, USA, Phone: (734) 763-9309, E-mail:

Award Identifier / Grant number: R01HS024953


The authors would like to thank Elizabeth Duffy for her contributions to the project and this work.

  1. Research funding: This work was supported by the Agency for Healthcare Research and Quality [grant number R01HS024953]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality and Department of Veterans Affairs. Dr. Singh is funded in part by the Houston Veterans Administration (VA) Health Services Research and Development (HSR&D) Center for Innovations in Quality, Effectiveness, and Safety (CIN13-413), the VA HSR&D Service (CRE17-127 and the Presidential Early Career Award for Scientists and Engineers USA 14–274), the VA National Center for Patient Safety, the Agency for Healthcare Research and Quality (R01HS27363), the CanTest Research Collaborative funded by a Cancer Research UK Population Research Catalyst award (C8640/A23385) and the Gordon and Betty Moore Foundation.

  2. Author contributions: PM and HS conceived the study, designed the consensus meeting, and obtained research funding. PM identified and convened the expert panel and organized consensus meeting; CM, PM, and HS led the consensus discussion; all authors (excluding SJP) participated in consensus meeting. PM and CWP drafted the manuscript, and all authors contributed substantially to its revision. PM takes responsibility for the paper as a whole. All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Ethical approval: This project was determined to be exempt from ethics review by the Institutional Review Board.

  5. Meeting: Results of this research were presented as a poster at the Diagnostic Error in Medicine Conference in New Orleans, LA, on November 5, 2018.


1. Institute of Medicine. Emergency care for children: growing pains. Washington, DC: National Academies Press; 2007. in Google Scholar

2. Institute of Medicine. Hospital based emergency care: at the breaking point. Washington, DC: National Academies Press; 2007. in Google Scholar

3. Institute of Medicine. Emergency medical services: at the crossroads. Washington, DC: National Academies Press; 2007. in Google Scholar

4. National Academies of Sciences. Engineering, and Medicine. Improving diagnosis in health care. In: Balogh, EP, Miller, BT, Ball, JR, editors. Washington, DC: The National Academies Press; 2015.Search in Google Scholar

5. National Quality Forum. Improving diagnostic quality and safety. Washington, DC: National Quality Forum; 2017.Search in Google Scholar

6. Rui, P, Kang, K, Ashman, JJ. National hospital ambulatory medical care survey: 2016 emergency department summary tables. Atlanta, GA: National Center for Health Statistics; 2016. Available from: [Accessed 17 Feb 2018].Search in Google Scholar

7. Gurley, KL, Grossman, SA, Janes, M, Yu-Moe, CW, Song, E, Tibbles, CD, . Comparison of emergency medicine malpractice cases involving residents to nonresident cases. Acad Emerg Med 2018;25:980–6. in Google Scholar

8. Singh, H, Meyer, AN, Thomas, EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Qual Saf 2014;23:727–31. in Google Scholar

9. Singh, H, Giardina, TD, Forjuoh, SN, Reis, MD, Kosmach, S, Khan, MM, . Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2012;21:93–100. in Google Scholar

10. Singh, H, Thomas, EJ, Khan, MM, Petersen, LA. Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med 2007;167:302–8. in Google Scholar

11. Singh, H. Editorial: helping health care organizations to define diagnostic errors as missed opportunities in diagnosis. Joint Comm J Qual Patient Saf 2014;40:99–101. in Google Scholar

12. Murphy, DR, Laxmisan, A, Reis, BA, Thomas, EJ, Esquivel, A, Forjuoh, SN, . Electronic health record-based triggers to detect potential delays in cancer diagnosis. BMJ Qual Saf 2014;23:8–16. in Google Scholar PubMed

13. Adekoya, N. Patients seen in emergency departments who had a prior visit within the previous 72 h-National Hospital Ambulatory Medical Care Survey, 2002. Publ Health 2005;119:914–8. in Google Scholar PubMed

14. Martin-Gill, C, Reiser, RC. Risk factors for 72-hour admission to the ED. Am J Emerg Med 2004;22:448–53. in Google Scholar PubMed

15. Verelst, S, Pierloot, S, Desruelles, D, Gillet, JB, Bergs, J. Short-term unscheduled return visits of adult patients to the emergency department. J Emerg Med 2014;47:131–9. in Google Scholar PubMed

16. Sauvin, G, Freund, Y, Saidi, K, Riou, B, Hausfater, P. Unscheduled return visits to the emergency department: consequences for triage. Acad Emerg Med 2013;20:33–9. in Google Scholar PubMed

17. Medford-Davis, L, Park, E, Shlamovitz, G, Suliburk, J, Meyer, AN, Singh, H. Diagnostic errors related to acute abdominal pain in the emergency department. Emerg Med J 2016;33:253–9. in Google Scholar PubMed

18. Guly, H. Diagnostic errors in an accident and emergency department. Emerg Med J 2001;18:263–9. in Google Scholar PubMed PubMed Central

19. Warrick, C, Patel, P, Hyer, W, Neale, G, Sevdalis, N, Inwald, D. Diagnostic error in children presenting with acute medical illness to a community hospital. Int J Qual Health Care 2014;26:538–46. in Google Scholar PubMed

20. Sundberg, M, Perron, CO, Kimia, A, Landschaft, A, Nigrovic, LE, Nelson, KA, . A method to identify pediatric high-risk diagnoses missed in the emergency department. Diagnosis (Berl) 2018;5:63–9. in Google Scholar PubMed

21. Bhise, V, Sittig, DF, Vaghani, V, Wei, L, Baldwin, J, Singh, H. An electronic trigger based on care escalation to identify preventable adverse events in hospitalised patients. BMJ Qual Saf 2018;27:241–6. in Google Scholar PubMed PubMed Central

22. Menendez, ME, Janssen, SJ, Ring, D. Electronic health record-based triggers to detect adverse events after outpatient orthopaedic surgery. BMJ Qual Saf 2016;25:25–30. in Google Scholar PubMed

23. Murphy, DR, Meyer, AND, Sittig, DF, Meeks, DW, Thomas, EJ, Singh, H. Application of electronic trigger tools to identify targets for improving diagnostic safety. BMJ Qual Saf 2019;28:151–9. in Google Scholar PubMed PubMed Central

24. Aaronson, E, Borczuk, P, Benzer, T, Mort, E, Temin, E. 72h returns: a trigger tool for diagnostic error. Am J Emerg Med 2018;36:359–61. in Google Scholar PubMed

25. Calder, L, Pozgay, A, Riff, S, Rothwell, D, Youngson, E, Mojaverian, N, . Adverse events in patients with return emergency department visits. BMJ Qual Saf 2015;24:142–8. in Google Scholar PubMed PubMed Central

26. Mahajan, P, Mollen, C, Alpern, ER, Baird-Cox, K, Boothman, RC, Chamberlain, JM, . An operational framework to study diagnostic errors in emergency departments: findings from a consensus panel. J Patient Saf 2019 Nov 25. [Epub ahead of print].Search in Google Scholar PubMed

27. McMillan, SS, King, M, Tully, MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm 2016;38:655–62. in Google Scholar PubMed PubMed Central

28. Liberman, AL, Newman-Toker, DE. Symptom-Disease Pair Analysis of Diagnostic Error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data. BMJ Qual Saf 2018;27: 557–66. in Google Scholar PubMed PubMed Central

29. Rising, KL, Victor, TW, Hollander, JE, Carr, BG. Patient returns to the emergency department: the time-to-return curve. Acad Emerg Med 2014;21:864–71. in Google Scholar PubMed

30. Singh, H, Upadhyay, D, Korukonda, S, Hunt, J, Torretti, D. Implementation of e-triggers for measurement of diagnostic errors in a large health care system. The Diagnostic Error in Medicine 12th Annual International Conference. Diagnosis 2019;6:eA1–eA96. in Google Scholar PubMed

31. Mahajan, P, Basu, T, Pai, CW, Singh, H, Petersen, N, Bellolio, MF, . Factors associated with potentially missed diagnosis of appendicitis in the emergency department. JAMA Netw Open 2020;3:e200612.10.1001/jamanetworkopen.2020.0612Search in Google Scholar PubMed PubMed Central

32. Newman-Toker, DE, Moy, E, Valente, E, Coffey, R, Hines, AL. Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample. Diagnosis (Berl) 2014;1:155–66. in Google Scholar PubMed PubMed Central

33. Singh, H, Giardina, TD, Forjuoh, SN, Reis, MD, Kosmach, S, Khan, MM, . Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2012;21:93–100. in Google Scholar PubMed PubMed Central

34. Jung, WS, Kim, SH, Lee, H. Missed diagnosis of anaphylaxis in patients with pediatric urticaria in emergency department. Pediatr Emerg Care 2021;37:199–203. in Google Scholar

35. Singh, H, Bradford, A, Goeschel, C. Operational measurement of diagnostic safety: state of the science. Diagnosis (Berl) 2021;8:51–65. in Google Scholar

36. Cosby, KS, Roberts, R, Palivos, L, Ross C, Schaider J, Sherman S, . Characteristics of patient care management problems identified in emergency department morbidity and mortality investigations during 15 years. Ann Emerg Med 2008;51:251–61. in Google Scholar

37. Chu, D, Xiao, J, Shah, P, Todd, B. How common are cognitive errors in cases presented at emergency medicine resident morbidity and mortality conferences?. Diagnosis (Berl) 2018;5:143–50. in Google Scholar

38. Okafor, N, Payne, VL, Chathampally, Y, Miller, S, Doshi, P, Singh, H. Using voluntary reports from physicians to learn from diagnostic errors in emergency medicine. Emerg Med J 2016;33:245–52. in Google Scholar

39. Pinto, A, Reginelli, A, Pinto, F, Lo Re, G, Midiri, F, Muzj, C, . Errors in imaging patients in the emergency setting. Br J Radiol 2016;89:20150914. in Google Scholar

40. Mattsson, B, Ertman, D, Exadaktylos, AK, Martinolli, L, Hautz, WE. Now you see me: a pragmatic cohort study comparing first and final radiological diagnoses in the emergency department. BMJ Open 2018;8:e020230. in Google Scholar

41. Shojania, KG. The elephant of patient safety: what you see depends on how you look. Joint Comm J Qual Patient Saf. 2010;36:399-401. in Google Scholar

42. Cunningham, CT, Quan, H, Hemmelgarn, B, Noseworthy, T, Beck, CA, Dixon, E, .. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol 2015;15:32. in Google Scholar PubMed PubMed Central

43. Grava-Gubins, I, Scott, S. Effects of various methodologic strategies: survey response rates among Canadian physicians and physicians-in-training. Can Fam Physician 2008;54:1424–30.Search in Google Scholar

Supplementary Material

The online version of this article offers supplementary material (

Received: 2019-12-10
Accepted: 2020-09-17
Published Online: 2020-11-13
Published in Print: 2021-08-26

© 2020 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 22.2.2024 from
Scroll to top button