Medical error is a leading cause of death nationwide. While systems issues have been closely investigated as a contributor to error, little is known about the cognitive factors that contribute to diagnostic error in an emergency department (ED) environment.
Eight months of patient revisits within 72 h where patients were admitted on their second visit were examined. Fifty-two cases of confirmed error were identified and classified using a modified version of the Australian Patient Safety Foundation classification system for medical errors by a group of trained physicians.
Faulty information processing was the most frequently identified category of error (45% of cases), followed by faulty verification (31%). Faulty knowledge (6%) and faulty information gathering (18%) occurred relatively infrequently. “Misjudging the salience of a finding” and “premature closure” were the individual errors that occurred most frequently (13%).
Despite the complex nature of diagnostic reasoning, cognitive errors of information processing appear to occur at higher rates than other errors, and in a similar pattern to an internal medicine service despite a different clinical environment. Further research is needed to elucidate why these errors occur and how to mitigate them.
Medical errors were estimated in 2016 to cause 250,000 deaths per year in the US ; the rate of diagnostic error overall has been estimated at 10–15% , although reported rates are variable depending on the method used to estimate incidence . While a great deal of effort has been expended on the improvement of systems issues such as improving patient handoffs  and medication reconciliation , cognitive errors have proved more difficult to study and mitigate .
Previous work by Graber et al. showed that the majority of the cognitive errors identified among high-risk cases on an internal medicine service were errors of faulty synthesis (information processing and verification) and that relatively few errors were attributable to poor knowledge or data gathering . Though the diagnostic errors in an emergency department (ED) are likely to occur at around a similar rate as diagnostic errors in the rest of the health system , ED errors may be of a different character because of the fast pace  and frequent interruptions  that can lead to incomplete or unreliable gathering or transfer of information . One study identified an incomplete history and physical examination as a factor in 42% of ED malpractice claims .
The first step to reducing error in the ED is to understand the factors that contribute most frequently to medical error. One commonly examined source of potential medical errors are patient revisits, as a patient’s return to the ED within a short time frame suggests that an error may have occurred . The aim of this study was to determine the cognitive factors that contribute to error most frequently in the ED by examining revisits within 72 h.
Subjects and methods
The study was conducted at an adult ED in an urban academic public hospital in New York City with approximately 156,000 annual visits.
A revisit was defined for our study as two visits to the same ED within a 72 h window. Cases were excluded if the patient was under 18 or over the age of 89, the second visit was planned during the first visit (e.g. wound check follow-up), the patient was admitted on the first visit or if the patient was discharged on both visits. While it is possible that some of the excluded cases also contained instances of error, we felt that the most serious and important cases of error were likely to be found when the patient was discharged on the first visit and admitted on the second visit, as this suggests that the original disposition may have been made in error.
Cognitive error was defined as a delayed, incorrect or missed diagnosis due to an error in physician judgment as determined by information obtained later, a modification of the definition used by Graber et al. . Errors were classified using a modified version of the Australian Patient Safety Foundation classification system and abbreviated from the categorization described by Graber et al.  to focus only on the cognitive factors contributing to error, not on systems issues. This classification system describes four broad categories of factors which can lead medical decision-making to break down: faulty knowledge, faulty data gathering, faulty information processing and faulty verification. It also defines multiple subcategories for each of the factors. See Table 1 for the classification system used with descriptions of each factor.
|Knowledge base inadequate or defective||Insufficient knowledge of relevant condition||Providers not aware of Fournier gangrene|
|Skills inadequate or defective||Insufficient diagnostic skill for relevant condition||Missed diagnosis of complete heart block, clinician misread electrocardiogram|
|Faulty data gathering|
|Ineffective, incomplete or faulty workup||Problems in organizing or coordinating patient’s tests and consultations||Delayed diagnosis of drug-related lupus: failure to consult patient’s old medical records|
|Ineffective, incomplete or faulty history and physical examination||Failure to collect appropriate information from the initial interview and examination||Delayed diagnosis of abdominal aortic aneurysm: incomplete past history questioning|
|Faulty test or procedure techniques||Standard test/procedure is conducted incorrectly||Wrong diagnosis of myocardial infarction: electrocardio-graphic leads reversed|
|Failure to screen (pre-hypothesis)||Failure to perform indicated screening procedures||Missed prostate cancer: rectal examination and PSA testing never performed in a 55-year-old man|
|Poor etiquette leading to poor data quality||Failure to collect required information owing to poor interaction with patient||Missed CNS contusion after very abbreviated history, pejorative questioning|
|Faulty synthesis: faulty information processing|
|Faulty context generation||Lack of awareness/consideration of aspects of patient’s situation that are relevant to diagnosis||Missed perforated ulcer in a patient presenting with chest pain and laboratory evidence of myocardial infarction|
|Overestimating or underestimating usefulness or salience of a finding||Clinician is aware of symptom, but either focuses too closely on it to the exclusion of others or fails to appreciate its relevance||Wrong diagnosis of sepsis in a patient with stable leukocytosis in the setting of myelodysplastic syndrome|
|Faulty detection or perception||Symptom, sign or finding should be noticeable, but clinician misses it||Missed pneumothorax on chest radiograph|
|Failed heuristics||Failure to apply appropriate rule of thumb, or overapplication of such a rule under inappropriate/atypical circumstances||Wrong diagnosis of bronchitis in a patient later found to have pulmonary embolism|
|Failure to act sooner||Delay in appropriate data analysis activity||Missed diagnosis of ischemic bowel in a patient with a 12-week history of bloody diarrhea|
|Faulty triggering||Clinician considers inappropriate conclusion based on current data, or fails to consider conclusion reasonable from data||Wrong diagnosis of pneumonia in a patient with hemoptysis: never considered the eventual diagnosis of vasculitis|
|Misidentification of a symptom or sign||One symptom is mistaken for another||Missed cancer of the pancreas in a patient with pain, radiating to the back, attributed to GERD|
|Distraction by other goals or issues||Other aspects of patient treatment (e.g. dealing with an earlier condition) are allowed to obscure diagnostic process for current condition||Wrong diagnosis of panic disorder: patient with a history of schizophrenia presenting with abnormal mental status, found to have CNS metastases|
|Faulty interpretation of a test result||Test results are read correctly, but incorrect conclusions are drawn||Missed diagnosis of Clostridiumdifficile enteritis in a patient with a negative stool test result|
|Reporting or remembering findings not gathered||Symptoms or signs reported that do not exist, often findings that are typically present in the suspected illness||None encountered|
|Faulty synthesis: faulty verification|
|Premature closure||Failure to consider other possibilities once an initial diagnosis has been reached||Wrong diagnosis of musculoskeletal pain after a car crash: ruptured spleen ultimately found|
|Failure to order or follow-up on appropriate test||Clinician does not use an appropriate test to confirm a diagnosis, or does not take appropriate next step after test||Wrong diagnosis of urosepsis in a patient: bedside urinalysis never performed|
|Failure to consult||Appropriate expert is not contacted||Hyponatremia inappropriately ascribed to diuretics in a patient later found to have lung cancer; no consults requested|
|Failure to periodically review the situation||Failure to gather new data in order to determine whether situation has changed since initial diagnosis||Missed colon cancer in a patient with progressively declining hematocrit attributed to gastritis|
|Failure to gather other useful information to verify diagnosis||Appropriate steps to verify diagnosis are not taken||Wrong diagnosis of osteoarthritis in a patient found to have drug-induced lupus after ANA testing|
|Overreliance on someone else’s findings or opinion||Failure to check previous clinician’s diagnosis against current findings||Outpatient followed with diagnosis of CHF, admitted with increased shortness of breath, later found to have lung cancer as the cause|
|Failure to validate findings with patient||Clinician does not check with patient concerning additional symptoms that might confirm/disconfirm diagnosis||Wrong diagnosis of bone metastases in a patient with many prior broken ribs|
|Confirmation bias||Tendency to interpret new results in a way that supports one’s previous diagnosis||Wrong diagnosis of pulmonary embolism: positive test for D-dimer taken to support this diagnosis in a patient with respiratory failure due to ARDS and Gram-negative sepsis|
ANA, antinuclear antibody; ARDS, adult respiratory distress syndrome; CHF, congestive heart failure; CNS, central nervous system; GERD, gastroesophageal reflux disease; PSA, prostate-specific antigen. From Graber ; used with permission.
As part of routine ED quality improvement measures, 72 h revisit case summaries are identified at our institution, with pertinent data included from the history, examination, laboratory results and imaging tests. If it remained unclear whether an error had occurred after review of the case summary, or further information was required to identify the type of error, the full medical record was examined and the original providers were contacted for further clarification when possible. Eight months of 72 h revisits from 2013 to 2014 were included in our analysis. Within the 8-month study period, 271 cases met the inclusion criteria and were examined.
A team of physicians, including two physicians with cognitive science experience, was trained on the modified Australian Patient Safety Foundation classification and its use over several hours prior to the start of the study. All team members assigned classifications to several example cases to ensure that all participants shared a similar understanding on how to use the classification system.
Identifying cognitive errors took place in two steps. In the first step, each case that met the inclusion criteria was examined by two independent reviewers to determine whether, in their clinical judgment, the case could have contained cognitive error as defined by our study criteria. Cases where the two reviewers disagreed were adjudicated by the full group. In the second step, all cases that were identified as possibly containing a cognitive error were examined in more detail by at least three members of the study team, who analyzed the details of each case and reached a consensus as a group as to whether an error occurred and if so, what categories of error occurred.
Data recording and analysis
Cases were categorized by error class and body system, as well as screened for known high-risk conditions that could predispose patients to medical error or poor outcomes, such as substance abuse or psychiatric illness. Cases were recorded in a secure database, accessible only to the researchers. Descriptive statistics were calculated using Numbers (Apple, Cupertino, CA, USA).
The study was approved by the Institutional Review Board at the participating hospital.
A total of 271 cases of revisits within 72 h met our inclusion criteria. A total of 131 (48%) cases were determined by both reviewers to contain no potential cognitive error; 140 (52%) were identified by at least one reviewer as containing a potential cognitive error and flagged for further review by the study team. Of the 140 cases where potential issues were identified, 52 cases (19%) were verified to represent instances of cognitive error as determined by group consensus (see Figure 1).
Within the 52 cases identified as containing a cognitive error, there were 120 cognitive factors identified as contributing to error (each case could be assigned more than one factor which contributed to the error). Among the four general categories of cognitive error, faulty information processing was the most common, representing 45% [95% confidence interval (CI) 36–54%] of the identified errors. Faulty verification was the next most frequently identified, representing 31% (95% CI 23–40%) of factors. Finally, factors of faulty data gathering and faulty knowledge occurred least commonly, representing 18% (95% CI 12–26%) and 6% (95% CI 3–12%) of errors, respectively. Table 2 represents the relative frequency of each of the four types of cognitive factors determined to contribute to error. The mean number of factors identified per case containing error was 2.3.
|Type of error||Error type||Number of times error identified|
|Knowledge||Knowledge base defective/inadequate||3|
|Skills inadequate or defective||4|
|Data gathering||Ineffective, incomplete or faulty workup||13|
|Ineffective, incomplete or faulty history and physical examination||9|
|Faulty test or procedure techniques||0|
|Failure to screen||0|
|Poor etiquette leading to poor data quality||0|
|Information processing||Faulty context generation||13|
|Overestimating or underestimating usefulness or salience of a finding||15|
|Faulty detection or perception||0|
|Failure to act sooner||0|
|Misidentification of a symptom or sign||1|
|Distraction by other goals or issues||3|
|Faulty interpretation of a test result||9|
|Reporting or remembering findings not gathered||0|
|Failure to order or follow-up on appropriate test||6|
|Failure to consult||4|
|Failure to periodically review the situation||5|
|Failure to gather other useful information to verify diagnosis||1|
|Overreliance on someone else’s findings or opinion||4|
|Failure to validate findings with patient||0|
Of the 25 described specific cognitive factors which can contribute to error, the most commonly occurring were “misjudging the salience of a finding” and “premature closure”, which each represented 13% (95% CI 8–20%) of the identified types of error. In cases with confirmed error, the most common body system involved was hepatobiliary (e.g. missed cholecystitis), representing 19% of the cases, followed by pulmonary (e.g. pneumonia too sick to discharge), representing 13% of the cases; see Table 3 for the percentage of errors by body system. We also noted that a significant portion of the cases involved patients with a history of substance abuse (12% of cases) and patients with psychiatric illness and congestive heart failure (each 8% of cases); Table 4 shows the incidence of errors associated with risk factors identified in our study.
|Primary system involved||Number of cases||Percentage of total cases, %|
|Risk factor||Number of cases||Percentage of total cases, %|
|Congestive heart failure||4||8|
|Human immunodeficiency virus (HIV)||3||6|
To the best of our knowledge, this is the first study to categorize and quantify the cognitive factors that contribute to errors in the ED. Our study shows that, similar to an inpatient internal medicine environment, the cognitive factors that contribute to error most often in 72 h revisits are faulty information processing and faulty verification of data.
Previously published literature from other medical disciplines shows mixed results regarding what types of cognitive errors are most common in diagnosis. An analysis of admitted patients found that errors of knowledge application were most frequent , similar to the results of this study. However, a study of primary care visits found a large number of errors involving history taking and examination of the patient  and a study of intensive care unit patients found that “failure to carry out the intended treatment” was the most frequent type of error . While some of these differences may be accounted for by the differences in clinical environments, the lack of an effective, comprehensive and agreed-upon methodology for measuring cognitive error is likely a significant contributing factor to these discrepancies . The presence of trainees in an academic ED theoretically increases the likelihood that inadequate knowledge is contributing to error ; over half of the residents in one survey acknowledged that insufficient knowledge may have contributed to a recent error in their care . Despite trainees’ limited clinical experience, errors of data gathering and knowledge do not appear in our study to occur more frequently in the ED setting.
Past literature is also mixed on the question of what types of cardinal presentations are most likely to be prone to error and revisits. One study in Taiwan found that patients with abdominal symptoms were at highest risk to return , similar to the high prevalence of errors in return visits categorized as ‘hepatobiliary’ or ‘gastrointestinal’ in our study. A Dutch study found a similarly high rate of returns for abdominal pathology, but also a high rate of returns for patients with urinary symptoms , which our study did not find. Upper respiratory tract infections were the most common reason for a revisit in a Hong Kong ED study , and a study of physician-recalled errors found that the most frequently reported errors involved pulmonary problems . Our study did not calculate the distribution of complaints by system for all visits to the ED at our study site, but the rates of error found in our study for hepatobiliary complaints (19% of cases) and gastrointestinal complaints (10%) were significantly higher than the rate of ED visits nationally for all gastrointestinal complaints (6% of all visits) , suggesting that intra-abdominal complaints may be more vulnerable to diagnostic and cognitive error. While the poor localization of visceral pain lends biologic plausibility to this idea, cultural and health care delivery system differences may also play a role and more study is needed in this area.
Our study concurred with previous findings that patient factors can be a significant contributor to diagnostic and cognitive error . Specifically, patients with psychiatric disease , substance abuse  and congestive heart failure  are known to have a high frequency of ED revisits; diagnosis and ideal management may be difficult in these populations. In our study, errors in each of these at-risk populations appeared more frequently than the rate of ED visits nationally for these issues. Substance abuse was the source of 12% of errors identified but represented only 7% of ED visits nationally. Psychiatric disease was tied to 8% of errors in our study but represented only 4% of ED visits nationally. Congestive heart failure was related to 8% of errors but was the source of 3% of national ED visits, and patients with human immunodeficiency virus (HIV) were tied to 6% of errors, while making up only 0.4% of ED visits . However, the overall number of cases analyzed in this study is small, which may limit the generalizability of these conclusions.
A variety of techniques have been explored to attempt to reduce errors of information processing. Classically, two types of diagnostic thinking have been described, with System 1, or rapid pattern recognition, being vulnerable to error and System 2, or effortful logical reasoning, being a safety net which can catch errors . Consequently, some efforts at error reduction have focused on cognitive forcing strategies, designed to reorient clinicians to alternative diagnostic possibilities at pre-specified points in the workup (such as pop-up reminders in the electronic medical record). Unfortunately, while these interventions succeed in making physicians more deliberate, they may not reduce cognitive error , , , suggesting that System 2 may not be as effective as suggested at eliminating information synthesis problems. Additionally, experienced physicians may be able to effectively use System 1 thinking to make their workflow more efficient by rapidly categorizing patients without making errors . While some research shows that only additional clinical knowledge and experience reduces cognitive error , other evidence has suggested promise for guided reflection and cognitive forcing strategies , . Ultimately, the effectiveness of interventions to reduce error may be context dependent; checklists have been shown to be effective in some contexts but not others . More research is still needed to determine what strategies may be most effective for reducing cognitive errors in the clinical environment.
There are several potential sources of bias for this study. Though many of our errors were attributed to either premature closure or misjudging the salience of a finding, managing patients is a complex and dynamic interaction between knowledge and interpretation and it is possible that knowledge issues may affect information processing and verification. Additionally, while Croskerry and others have described many of the types of cognitive errors that occur  as well as the clinical situations in which they are most likely to be found, in practice, it can be difficult to categorize real errors that occur in the clinical setting as descriptions overlap and the error types lack strict criteria; we did not categorize distal causes of error in this analysis . Our definition of a revisit may have also systematically affected our results, as a missed important finding such fracture would be unlikely to have been admitted on their second visit.
The results of this study are also based on a retrospective review, which may have overestimated the incidence of errors, as the reviewers have the benefit of hindsight. In an effort to reduce hindsight bias and give maximum deference to providers, we attempted to include only cases of clear cognitive error. Our study is also limited by its single center nature, as patients may have returned and been admitted at other hospitals, as well as our documentation, as what was documented in the original medical record and the quality review case summaries may not fully reflect potential issues. As there is no gold standard for determining whether an error occurred or what cognitive factors may have contributed, we relied on group consensus, which can be subject to bias. Finally, as the classification system we used for cognitive factors contained more factors categorized as information processing and verification factors than those categorized as knowledge or data gathering factors, this may have contributed to the increased frequency that these factors were seen in our study.
Errors of information processing and verification were the most commonly identified errors in a study of patients with 72 h revisits to an academic ED. Patients with abdominal complaints were at highest risk for cognitive errors in diagnosis. Further standardization around describing and quantifying cognitive errors is needed to further elucidate how these errors impact care in the ED environment.
We thank Dr. Candice Cruz, Dr. Courtney Cassella, Dr. Angela Hua, Dr. Zara Mathew, Dr. Clark Owyang, Dr. Bradley Shy and Dr. Sumintra Wood for their invaluable assistance with completing this project.
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
1. Makary MA, Daniel M. Medical error-the third leading cause of death in the US. Br Med J 2016;353:i2139. Search in Google Scholar
2. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008;121:S2–23. Search in Google Scholar
3. Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013;22(Suppl 2):ii21–7. Search in Google Scholar
4. Riesenberg LA, Leitzsch J, Massucci JL, Jaeger J, Rosenfeld JC, Patow C, et al. Residents’ and attending physicians’ handoffs: a systematic review of the literature. Acad Med 2009;84:1775–87. Search in Google Scholar
5. Mueller SK, Sponsler KC, Kripalani S, Schnipper JL. Hospital-based medication reconciliation practices: a systematic review. Arch Intern Med 2012;172:1057–69. Search in Google Scholar
6. Zwaan L, Monteiro S, Sherbino J, Ilgen J, Howey B, Norman G. Is bias in the eye of the beholder? a vignette study to assess recognition of cognitive biases in clinical case workups. BMJ Qual Saf 2017;26:104–10. Search in Google Scholar
7. Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med 2005;165:1493–9. Search in Google Scholar
8. Chellis M, Olson J, Augustine J, Hamilton G. Evaluation of missed diagnoses for patients admitted from the emergency department. Acad Emerg Med 2001;8:125–30. Search in Google Scholar
9. Hamden K, Jeanmonod D, Gualtieri D, Jeanmonod R. Comparison of resident and mid-level provider productivity in a high-acuity emergency department setting. Emerg Med J 2014;31:216–9. Search in Google Scholar
10. Berg LM, Kallberg AS, Goransson KE, Ostergren J, Florin J, Ehrenberg A. Interruptions in emergency department work: an observational and interview study. BMJ Qual Saf 2013;22:656–63. Search in Google Scholar
11. Stiell A, Forster AJ, Stiell IG, van Walraven C. Prevalence of information gaps in the emergency department and the effect on patient outcomes. Can Med Assoc J 2003;169:1023–8. Search in Google Scholar
12. Kachalia A, Gandhi TK, Puopolo AL, Yoon C, Thomas EJ, Griffey R, et al. Missed and delayed diagnoses in the emergency department: a study of closed malpractice claims from 4 liability insurers. Ann Emerg Med 2007;49:196–205. Search in Google Scholar
13. Calder L, Pozgay A, Riff S, Rothwell D, Youngson E, Mojaverian N, et al. Adverse events in patients with return emergency department visits. BMJ Qual Saf 2015;24:142–8. Search in Google Scholar
14. Zwaan L, de Bruijne M, Wagner C, Thijs A, Smits M, van der Wal G, et al. Patient record review of the incidence, consequences, and causes of diagnostic adverse events. Arch Intern Med 2010;170:1015–21. Search in Google Scholar
15. Singh H, Giardina TD, Meyer AN, Forjuoh SN, Reis MD, Thomas EJ. Types and origins of diagnostic errors in primary care settings. JAMA Intern Med 2013;173:418–25. Search in Google Scholar
16. Rothschild JM, Landrigan CP, Cronin JW, Kaushal R, Lockley SW, Burdick E, et al. The critical care safety study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med 2005;33:1694–700. Search in Google Scholar
17. Young JQ, Ranji SR, Wachter RM, Lee CM, Niehaus B, Auerbach AD. “July effect”: impact of the academic year-end changeover on patient outcomes: a systematic review. Ann Intern Med 2011;155:309–15. Search in Google Scholar
18. Hobgood C, Hevia A, Tamayo-Sarver JH, Weiner B, Riviello R. The influence of the causes and contexts of medical errors on emergency medicine residents’ responses to their errors: an exploration. Acad Med 2005;80:758–64. Search in Google Scholar
19. Wu CL, Wang FT, Chiang YC, Chiu YF, Lin TG, Fu LF, et al. Unplanned emergency department revisits within 72 h to a secondary teaching referral hospital in Taiwan. J Emerg Med 2010;38:512–7. Search in Google Scholar
20. van der Linden MC, Lindeboom R, de Haan R, van der Linden N, de Deckere ER, Lucas C, et al. Unscheduled return visits to a Dutch inner-city emergency department. Int J Emerg Med 2014;7:23. Search in Google Scholar
21. Ng C, Chung C. An analysis of unscheduled return visits to the accident and emergency department of a general public hospital. Hong Kong J Emerg Med 2003;10:153–61. Search in Google Scholar
22. Schiff GD, Hasan O, Kim S, Abrams R, Cosby K, Lamber BL, et al. Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch Intern Med 2009;169:1881–7. Search in Google Scholar
23. National Hospital Ambulatory Medical Care Survey: 2015 Emergency Department Summary Tables. Available at: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2015_ed_web_tables.pdf. Accessed 14 Jun 2018. Search in Google Scholar
24. Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA, et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med 2006;145:488–96. Search in Google Scholar
25. van Nieuwenhuizen A, Henderson C, Kassam A, Graham T, Murray J, Howard LM, et al. Emergency department staff views and experiences on diagnostic overshadowing related to people with mental illness. Epidemiol Psychiatr Sci 2013;22:255–62. Search in Google Scholar
26. Smith MW, Stocks C, Santora PB. Hospital readmission rates and emergency department visits for mental health and substance abuse conditions. Community Ment Health J 2015;51:190–7. Search in Google Scholar
27. Chun S, Tu JV, Wijeysundera HC, Austin PC, Wang X, Levy D, et al. Lifetime analysis of hospitalizations and survival of patients newly admitted with heart failure. Circ Heart Fail 2012;5:414–21. Search in Google Scholar
28. Stanovich KE, West RF. Individual differences in reasoning: implications for the rationality debate? In: Gilovich T, Griffin D, Kahneman D, editors. Heuristics and biases: the psychology of intuitive judgment. New York, NY: Cambridge University Press, 2002:421–40, Chapter xvi, 857 Pages. Search in Google Scholar
29. Sherbino J, Kulasegaram K, Howey E, Norman G. Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: a controlled trial. Can J Emerg Med 2014;16:34–40. Search in Google Scholar
30. Monteiro SD, Sherbino J, Patel A, Mazzetti I, Norman GR, Howey E. Reflecting on diagnostic errors: taking a second look is not enough. J Gen Intern Med 2015;30:1270–4. Search in Google Scholar
31. Sherbino J, Yip S, Dore KL, Siu E, Norman GR. The effectiveness of cognitive forcing strategies to decrease diagnostic error: an exploratory study. Teach Learn Med 2011;23:78–84. Search in Google Scholar
32. Kovacs G, Croskerry P. Clinical decision making: an emergency medicine perspective. Acad Emerg Med 1999;6:947–52. Search in Google Scholar
33. Monteiro SD, Sherbino JD, Ilgen JS, Dore KL, Wood TJ, Young ME, et al. Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians? Acad Med 2015;90:511–7. Search in Google Scholar
34. Graber ML, Kissam S, Payne VL, Meyer AN, Sorensen A, Lenfestey N, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf 2012;21:535–57. Search in Google Scholar
35. Lambe KA, O’Reilly G, Kelly BD, Curristan S. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual Saf 2016;25:808–20. Search in Google Scholar
36. Shimizu T, Matsumoto K, Tokuda Y. Effects of the use of differential diagnosis checklist and general de-biasing checklist on diagnostic performance in comparison to intuitive diagnosis. Med Teach 2013;35:e1218–29. Search in Google Scholar
37. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med 2003;78:775–80. Search in Google Scholar
38. Croskerry P. Our better angels and black boxes. Emerg Med J 2016;33:242–4. Search in Google Scholar
©2018 Walter de Gruyter GmbH, Berlin/Boston