Abstract
Diagnostic error is increasingly recognized as a major patient safety concern. Efforts to improve diagnosis have largely focused on safety and quality improvement initiatives that patients, providers, and health care organizations can take to improve the diagnostic process and its outcomes. This educational policy brief presents an alternative strategy for improving diagnosis, centered on future healthcare providers, to improve the education and training of clinicians in every health care profession. The hypothesis is that we can improve diagnosis by improving education. A literature search was first conducted to understand the relationship of education and training to diagnosis and diagnostic error in different health care professions. Based on the findings from this search we present the justification for focusing on education and training, recommendations for specific content that should be incorporated to improve diagnosis, and recommendations on educational approaches that should be used. Using an iterative, consensus-based process, we then developed a driver diagram that categorizes the key content into five areas. Learners should: 1) Acquire and effectively use a relevant knowledge base, 2) Optimize clinical reasoning to reduce cognitive error, 3) Understand system-related aspects of care, 4) Effectively engage patients and the diagnostic team, and 5) Acquire appropriate perspectives and attitudes about diagnosis. These domains echo recommendations in the National Academy of Medicine’s report Improving Diagnosis in Health Care. The National Academy report suggests that true interprofessional education and training, incorporating recent advances in understanding diagnostic error, and improving clinical reasoning and other aspects of education, can ultimately improve diagnosis by improving the knowledge, skills, and attitudes of all health care professionals.
Introduction
A hypothesis: we can improve diagnosis by improving education
Establishing an accurate and timely diagnosis is the foundation of safe, effective patient care. The diagnostic process is complicated, nuanced, and uncertain. Clinicians require years of education, experience, and coaching to develop their diagnostic ability. Given the complexity of the task, it is remarkable how often the process succeeds. However, a substantial body of evidence suggests that healthcare professionals and systems could do better; diagnostic errors are a common and important problem that carry enormous costs – in both dollars and human suffering. Diagnostic error was defined by the National Academy of Medicine’s (NAM’s) seminal report Improving Diagnosis in Health Care as “the failure to provide an accurate and timely explanation of the patient’s health problem(s) or communicate that explanation to the patient”, and includes diagnoses that were wrong, missed entirely, or should have been made much earlier [1], [2]. According to best estimates, one person in 20 will experience a diagnostic error each year in ambulatory clinics [3], [4]. Diagnostic errors account for the largest fraction of malpractice claims in most specialties [5], and at least 40,000–80,000 hospitalized patients die annually from diagnostic error in the United States [6]. The National Academy report concluded that “Each of us is likely to experience one or more diagnostic errors in our lifetime, sometimes with devastating consequences” [1].
The National Academy report outlines a broad range of interventions that could improve the safety of diagnosis. Most of these recommendations focus on the factors that influence the quality and safety of care in actual practice. For example, system-related aspects of care could be studied to improve the diagnostic process, such as more effective use of the electronic medical record and decision-support resources. Improving teamwork in diagnosis was cited as a particularly attractive approach to improve the process. Interventions to improve the cognitive aspects of diagnosis include suggestions to ‘debias’ decision making, and improve the environment for diagnosis, for example, by providing more time for the patient encounter. In this educational policy brief we focus specifically on the report’s recommendation to shift focus to training the next generation of clinicians. The foundational hypothesis is that diagnosis could be improved by improving the education of these future practitioners.
Every health care profession contributes to diagnosis and every health care professional will make or aid in the diagnosis of a patient in his or her career. Many health professionals have direct responsibility for diagnosis (medicine, advanced practice nursing, physician assistants, dentists, optometrists, podiatrists, and many others), and clinicians in other health care professions (nursing, pharmacy, social work service, physical therapy, and many others) participate by contributing to assessment and helping to recognize and avoid diagnostic errors. Many professional schools include instruction on elements of the diagnostic process, focusing on things such as taking an accurate history, contributing to a comprehensive patient assessment, or clinical reasoning. Rarely, however, is this training explicit, comprehensive, or focused on preventing diagnostic error. Although comparable data is lacking from other schools, education on clinical reasoning in schools of medicine is inadequate; in a recent survey of clerkship directors, 84% reported that students entered clinical clerkships with poor, or at best fair, appreciation of key clinical reasoning concepts, and 57% acknowledged that there was no formal training on these topics in the preclinical years [7]. In schools of medicine, trainees learn the process of diagnosis implicitly through experience and apprenticeship. In schools of nursing and other health professions, instruction may be less idiosyncratic and may include classroom sessions, readings, and simulation exercises. As the paradigm shifts across all health professions towards a focus on defining, measuring, and ensuring the competence of trainees (competency-based education), there is a need to ensure that learners are able to demonstrate, high-quality, safe diagnostic practices and abilities appropriate for their profession [8], [9], [10].
For this report we conducted a literature search to clarify why education should be a priority in efforts to improve diagnosis, and how education can play this important role. First, results from the literature review provide the justification for developing a competency-based, interprofessional curriculum to improve diagnosis. Second, the literature search identified new content that would be appropriately incorporated in education and training to produce these competencies in learners as well as specific educational approaches that could be used in delivering this content. We convened an interprofessional group of advisors to identify the most important content elements, and organized these into a driver diagram that groups these into five general areas, each with the potential to help reach the ultimate goal of improving diagnostic performance in practice and reducing diagnostic errors.
We propose that the reforms we are advocating apply to all health professions. Some of this education and training will, by pragmatic necessity, occur within each profession’s own educational programs. However, the NAM report emphasizes the large gains that could be realized by improving interprofessional collaboration in diagnosis. The NAM report suggests that diagnostic quality would benefit by reconceptualizing the process as a team effort, not just the responsibility of a single clinician. Accordingly, we believe that diagnostic error can be reduced by providing at least some, or perhaps most of the recommended content in every health profession, and through joint education and training experiences, so that trainees can become skilled in diagnosis both individually and as members of diagnostic teams. If diagnoses are best made by teams, we advocate that the diagnostic process should be learned in teams.
Methods
Literature search
We conducted a 10-year survey (2007–2017) of the health professions literature to identify English language, full-text publications focused on education or training to improve clinical reasoning or diagnosis. The general search strategy employed this logic: [(Education OR training OR residency) NOT (continuing education)] AND (diagnosis OR clinical reasoning OR interprofessional teamwork OR competencies) and the specific search strategies are presented in the Appendix (Supplementary Material). Databases searched (and abstracts retrieved) included Ovid-Medline (163 abstracts), Embase (2757 abstracts), Academic Search Complete (864 abstracts), and SCOPUS (1583 abstracts). After removing duplicate listings, just over 3500 abstracts were reviewed; 481 were selected as being project relevant, and these were assigned to general topic areas: clinical reasoning (262), system-related issues (75), diagnostic error (106), interprofessional issues (16), and assessment (83), with some abstracts being assigned to two areas. These abstracts described a wide range of topics, focused on different learners at different stages of their education and training, different educational approaches, and using variable research designs. Accordingly, no effort was made to systematically organize the findings other than to provide examples, synthesize findings when appropriate, and identify possible model programs.
Driver diagrams create a visual representation of the primary and secondary factors that are relevant to achieving a goal [11]. As a first step toward developing competencies to recommend, we created a driver diagram to identify factors in the educational process relevant to improving diagnosis in practice. An interprofessional consensus committee of 32 individuals was recruited to help develop and refine this driver diagram. In addition to a patient representative and one medical student, represented professions included medicine (internal medicine, pediatrics, emergency medicine), nursing, physician assistants, pharmacy, laboratory medicine, and health professions educators. Professional boards and organizations represented included the Accreditation Council for Graduate Medical Education, the American Association of Colleges of Nursing, the Alliance for Academic Internal Medicine, the American Association of Medical Colleges, the American Association of Nurse Practitioners, the American Board of Internal Medicine, the American Board of Medical Specialties, the American Board of Pediatrics, the American Medical Association Council on Graduate Medical Education, and the Association of American Medical Colleges. A five-member steering committee coordinated the involvement of the consensus group as well as the overall project flow.
The consensus committee was divided by self-selection into three groups (clinical reasoning, team-related topics, and system-related topics related to diagnostic error) with the assignment to identify primary and secondary drivers through which education could improve the quality and safety of diagnosis. Each group was provided access to the relevant abstracts retrieved by the literature search. Leaders were identified for each working group and, through a series of conference calls, the groups reviewed the relevant abstracts and combined the findings of this literature base with their own experiences and viewpoints to inform preliminary driver development. The groups also developed draft competencies based on the drivers which are being used in later stages of the project. These preliminary driver suggestions were collected and used to create a draft driver diagram using a modified Delphi process [12], in which the diagram was presented and then serially refined through iterative cycles of group conference calls. Feedback obtained during these calls as well as that obtained asynchronously via email was incorporated anonymously between calls, leading to a revised driver diagram then presented on the next call. The final version was approved unanimously on the final teleconference call of the consensus committee.
Findings: justification for education reform
In reviewing the published literature on education topics relevant to diagnosis, the authors and consensus committee identified two major reasons why education reform to improve diagnosis is imperative:
Existing training programs may not provide adequate education regarding diagnostic safety.
Most health professions education programs provide and require courses on the basic elements of the diagnostic process, and the specific medical knowledge and clinical skills necessary for developing diagnostic ability in that profession. Physician competency expectations both in the US [13] and Canada [14] emphasize the primary goal of skillful and comprehensive patient assessment to arrive at a diagnosis, and also now speak to the more contemporary goals of working collaboratively with patients in the diagnostic process, acknowledging uncertainty, and the value of learning from errors to improve performance.
However, there is typically little or no content in health professions education today on cognitive psychology, the impact of health care systems on the diagnostic process, or the mechanisms by which diagnostic errors arise and can be avoided. The actual competence of trainees in health professions education programs even with respect to performing elements of the diagnostic process that are explicitly addressed (such as history and physical examination), have also been drawn into question. An increasing number of publications are calling attention to the substandard performance on even basic elements of obtaining the history and performing the physical examination [15], [16]. Further, skills in clinical reasoning are suboptimal, even in schools of medicine [7].
There is a clear mandate to improve diagnostic safety training in health professions education.
Stark and Fins describe the need to include specific training on diagnostic error as an ethical imperative, given the aggregate harm from diagnostic error, the clear evidence how often cognitive shortcomings are involved, and the reality that current curricula do not provide effective training on how to optimize clinical decision-making. “Continued failure to address pervasive thinking errors in medical decision-making imperils patient safety and professionalism, as well as beneficence and non-maleficence, fairness and justice” [17].
Patient safety authorities have echoed this mandate for curricular reform. A major white paper from the Lucien Leape Institute called out the deficiencies in the educational process in regard to patient safety: “…substantive improvements in patient safety will be difficult to achieve without major medical education reform at the medical school and residency training program levels. Specifically, schools of medicine are not doing an adequate job of facilitating student understanding of basic knowledge and the development of skills required for the provision of safe patient care, to wit: systems thinking, problem analysis, application of human factors science, communication skills, patient-centered care, interprofessional collaborative practice and skills, and dealing with feelings of doubt, fear, and uncertainty with respect to medical errors” [18]. Although training on clinical reasoning and the elements of diagnosis are standard components of training for nurses, physician assistants, and many other health professions, it is probably reasonable to assume that these more advanced competencies we are proposing are shortchanged or absent from those education and training programs, just as they are deficient in schools and training programs of medicine.
The National Academy of Medicine’s Improving Diagnosis in Health Care explicitly called for enhanced education and training related to diagnosis, identifying clear gaps and shortcomings, and many opportunities to improve diagnostic performance. A major recommendation of the report was to “Enhance health care professional education and training in the diagnostic process” (Figure 1) [1]. The Millennium Conference 2011 similarly emphasized the importance of incorporating specific training on clinical reasoning and critical thinking into the curriculum of health professionals [19].
National Academy of Medicine GOAL #2: enhance health care professional education and training in the diagnostic process.
Findings: curriculum content and delivery approaches
The literature search identified important advances that have been made in understanding patient safety in general, and diagnostic safety specifically, that merit incorporation in teaching and training programs. The search also identified specific suggestions on how this content should be delivered to optimize learning and the successful attainment of relevant competencies. The consensus group identified five primary drivers and multiple relevant secondary drivers through which education and training could ultimately improve diagnosis (Figure 2). Learners should demonstrate the ability to …:
A ‘driver diagram’ illustrating how education could improve diagnostic performance.
Five primary drivers are indicated by the circles, and secondary drivers in the accompanying lists.
Acquire and effectively use a relevant knowledge base
Optimize clinical reasoning to reduce cognitive error
Understand system-related aspects of care
Effectively engage patients and members of the diagnostic team
Acquire appropriate perspectives and attitudes (acknowledging complexity and uncertainty, incorporating the patient’s values, understanding patient safety and diagnostic error concepts, etc.)
1) Acquire and effectively use a relevant knowledge base
There is universal agreement that the most important determinant of effective and safe diagnosis is a comprehensive base of medical knowledge, including content from both the basic sciences (anatomy, physiology, etc.) and clinical sciences (pathophysiology, organ system diseases) that forms the core of education for physicians, nurses, physician assistants, and other health professionals [20], [21], [22], [23]. The ability to construct an appropriately prioritized differential diagnosis is considered the most essential element of diagnostic competency [24], and this skill derives from mastering the content relevant to the conditions being considered. Obviously, knowledge is necessary but not alone sufficient: successful diagnosis also requires appropriate use of laboratory and imaging data, integration of epidemiology data, and a host of other elements and skills.
Incorporating recent advances in understanding how to improve uptake of medical knowledge content would be a meaningful and likely effective way to improve diagnosis in its own right, and would complement the new curricular elements dealing specifically with improving diagnostic quality and safety [25]. Schools are already using some of these novel approaches, including test-enhanced learning, interleaving, spaced education, and deliberate practice in realistic simulations. Cognitive load theory may have specific value in learning diagnosis, given the complexity of the diagnostic process [26]. Cognitive load theory stresses that task training in the individual components of diagnosis (e.g. history-taking, differential diagnosis) is not enough to learn the whole task of diagnosis, as the interaction and concurrence of these components is a source of much difficulty. Thus, this complex process can only be effectively learned with whole task training. Thus, for team-based diagnosis, simulation with a typical team of health professionals involved in a given diagnostic context will be important.
The realization that diagnosis is context-specific and highly dependent on the typicality (or lack thereof) of the presenting features suggests that learning can be optimized by starting with prototypical cases to develop skills in describing the case (problem representation), and an ‘illness script’ for that condition [22], [27], [28]. Expertise then develops by introducing variants and atypical presentations progressively, perhaps aided by simulation, ideally in a learner-centered, progressively adapted approach [29], [30], [31].
2) Optimize clinical reasoning to reduce cognitive error
Substantial progress has been made over the past several decades in better describing the cognitive psychology of clinical reasoning and diagnostic performance [32], [33]. Further, studies of the causes of diagnostic error reliably point to breakdowns in the diagnostic process that relate to suboptimal clinical reasoning. In parallel, an emerging body of education research offers the potential to improve the way ‘diagnosis’ and clinical reasoning can and should be taught [23], [27], [34, 35, 36, 37, 38, 39]. A sufficient body of knowledge now exists to define competence in clinical reasoning and diagnosis, delineate assessment practices, and develop educational programs through which learners may attain these competencies during health professions education.
The importance of clinical decision making has been underscored by healthcare leaders since the time of Flexner. “Decision making is implicit or explicit in every clinical activity encountered during undergraduate or graduate medical education because it is a fundamental to medical practice itself” [40]. Redelmeier has argued that cognitive psychology should be taught alongside the basic sciences, because it addresses the problem that humans tend to make mistakes in solving complex problems, and that these mistakes are both predictable and understandable [41]. According to Redelmeier and colleagues, teaching principles of cognitive psychology would accomplish four things:
It would sensitize future clinicians to the predictable pitfalls of judgment and decision making;
It would provide a ‘language and a logic’ for discussing diagnostic quality and safety;
It would encourage the kind of reflective practice that has been advocated as an effective antidote for diagnostic errors [42]; and
It would introduce the concept of metacognition, an implicit skill required to recognize one’s own limitations, the need for including team members and consultants in the diagnostic process, and the importance of lifelong learning to improve diagnostic performance.
The complexity and ambiguity involved in the diagnostic process makes it necessary that we study and understand factors underlying and enabling differential diagnosis. Much of the understanding about how humans perform differential diagnosis and other ill-defined categorization tasks has been organized within a theoretical framework referred to as the “dual processing” paradigm. Dual processing suggests that two distinctly different cognitive systems drive the performance of categorization tasks. The first is a rapid, intuitive process based on pattern recognition (System 1). The second is a slower, analytically-driven system (System 2). The dual processing paradigm is increasingly accepted as a helpful way to understand not only clinical decision making, but human decision making in general [32], [43]. Both non-analytic and analytic decision-making processes play a role in diagnosis, and health professions education programs must emphasize that the attainment of expertise in clinical reasoning invariably involves the use of both processes [43].
The intuitive, System 1 pathway is prone to the effects of cognitive bias, and there is a consensus that cognitive biases and shortcomings play a significant role in cognitive errors [44]. Debiasing in practice centers on the use of reflection (System 2) to monitor the intuitive (System 1) thinking, and reflection is one of the most promising strategies to improve diagnostic performance in health professions education [42], [45]. Although several early experiments were unable to demonstrate an impact of reflection to improve diagnosis [46], there are also successful demonstrations [45], [47], [48]. A recent systematic review of both cognitive and technologial interventions to improve diagnostic decision-making concluded that on balance debiasing was effective [49], but it seems likely that a great deal of experimental work lies ahead to understand what distinguishes successful from ineffective interventions. Over 40 separate mitigation strategies have been described by this point, many already in use in every-day life and other professions [50]. An important point is that even if de-biasing is unsuccessful in preventing cognitive errors, learning the most common cognitive tendencies that underlie diagnostic error may allow one to recognize faulty decision-making and avoid harm [51].
Understanding that diagnosis always takes place in a given context, it is fundamental for health care professionals to understand how contextual and ‘human factors’ elements impact diagnostic quality [52], [53], [54]. Trainees would also benefit from learning how expertise develops, in particular the critical role of feedback and self-monitoring [55], [56], [57]. The tenets of deliberate, reflective practice have only recently been discussed in health professions education, including goal-setting, authentic practice experiences with coached feedback, and a commitment to improvement over the long term. Given the demonstration of the effectiveness of deliberate practice in attaining expertise in most other studied fields of human performance, this requires further study and implementation in health professions education [58]. Similarly, exercises to develop skill in rationality and critical thinking may be beneficial [59], [60]. Lastly, given the variation and ambiguity that exist daily in every clinical encounter, it is important that health professions education programs explicitly address uncertainty and humility.
Our consensus committee also emphasized the need for educators to have access to high-quality assessment tools with validity and reliability evidence to determine learner competence with respect to diagnostic performance. Assessment is important not only for making summative determinations about learner competence (entrustability decisions) but also enables students to improve by helping them to identify knowledge and performance gaps (assessment for learning). Health professions education has long relied on high-stakes assessments of medical knowledge (e.g. board examinations and other multiple choice tests). Such assessments are likely necessary to gauge acquisition and accessibility of basic medical knowledge and its organization, but given the context-specific and context-dependent nature of diagnosis and diagnostic performance, it is increasingly clear that the medical education community must identify valid and reliable means of workplace-based assessment of diagnostic performance [61], [62], and repeat these over time. Effective strategies will likely include a combination of workplace-based assessments (e.g. end-of-rotation global assessments, direct observation) and non-workplace-based assessments, including multiple-choice question examinations, simulation [including Objective Structured Clinical Examinations (OSCEs), and virtual cases]. In addition to assessment of individual performance, it is also clear that there must be reliable and valid means of measuring team performance (Team-OSCEs) with respect to diagnosis; there are many existing assessments of team-based decision-making in other domains [63], but no published assessment tools focus on the performance of diagnosis teams.
3) Understand system-related aspects of care
Education and training on patient safety topics generally could also be improved. A systematic review published in 2010 identified 41 published reports of patient safety curricula in the US, but only 16 (39%) included content on systems thinking, only three (7%) included content on human factors, and none reported content specifically on diagnostic error generally or cognitive error more specifically [64]. Another recent review concluded that the safety curricula that have been studied generally have not provided appropriate course evaluation, and have not focused on topics relevant to diagnostic quality [65]. The NAM report, and others [66], [67], have emphasized the importance of understanding the diagnostic process from a human factors perspective, and how the success or failure of the process is determined by the relationships between the individual, the task, and the environment.
System-related breakdowns can be identified in the majority of diagnostic errors [2], and interventions to address system-related errors have been collected and summarized [68]. Communication breakdowns are the most commonly-identified contributing factor, and efforts to improve communication (e.g. handoffs used at transitions of care) are important early targets to improve diagnosis. Ensuring communication of test results is another area ripe for improvement, including tests pending at discharge, and trainees should be sensitized to the likelihood of breakdowns in these areas. Trainees should also have the opportunity to learn about and use ‘system’ resources that could help improve diagnostic accuracy, including appropriate consultation from subspecialists, obtaining second opinions [69], and using health informatics tools such as differential diagnosis generators [70]. Finally, trainees would benefit from training in learning health systems, where they are exposed to incident reporting, root cause analysis of cases of diagnostic error or success, ‘morbidity and mortality’ case discussions, and reviews of diagnostic performance data.
4) Effectively engage patients and members of the diagnostic team
Many health professional schools have developed and/or expanded their interprofessional educational experiences, recognizing the importance of teamwork in high-quality patient care. With regard to clinical reasoning specifically, the first recommendation in the NAM report on Improving Diagnosis in Healthcare likewise focused on the diagnostic process as a team-based activity, replacing the physician-centric paradigm. The concept that team training can improve safety was pioneered in aviation, and the same benefit has been replicated in healthcare (e.g. in surgical settings) [71]. The TeamSTEPPS model, based on the crew resource management training provided in aviation, has been implemented in thousands of hospitals in the United States, particularly in environments where the team composition is stable, such as the operating room or emergency departments [72], [73], [74]. Creating effective diagnostic teams, however, is novel [75], [76], [77]. It is appropriate to study and understand diagnosis as an example of distributed cognition [78], where all of the stakeholders play a different but ultimately coordinated role in diagnosis. The new diagnostic team envisions more direct involvement in the diagnostic process by the patient [79], family, nurses, pharmacists, and every other health professional who touches the patient. There is accumulating evidence that engaged patients have better health outcomes, and their activation to partner in ‘co-producing’ diagnoses could help ensure that patients and the clinical staff share the same mental model of the diagnostic process and the uncertainty therein.
With breakdowns in communication and care coordination leading the list of system-related diagnostic errors [80], improving the extent of interprofessional collaboration can be especially impactful in improving diagnostic outcomes [81].
To be most effective, team training should not just be a topic in the education program of every health profession’s individual silo. Education and training should be provided to groups of trainees from different professions, and should provide authentic experiences for trainees to undertake diagnostic exercises jointly.
5) Acquire appropriate perspectives and attitudes (acknowledging complexity and uncertainty, incorporating the patient’s values, understanding patient safety and diagnostic error concepts, etc.)
The past decades have produced abundant knowledge on the incidence, etiology, and possible prevention of diagnostic errors, summarized in the NAM report on Improving Diagnosis in Health Care[1]. Although this remains an unproven assumption, it is reasonable to propose that learners who are aware of how diagnostic errors arise as well as error prevention strategies will be better equipped to employ this knowledge in practice, and achieve higher levels of diagnostic performance.
Students (and their teachers) would do well to appreciate how diagnosis plays out over time, and the importance of being able to triage which diagnoses need to be made immediately, or can safely wait. Similarly, trainees need to learn that diagnosis is not always beneficial; labelling tends to discourage further thinking about a case, and can in fact be dangerous, especially if the label is wrong. Further, there is very real harm resulting from overdiagnosis, where assigning a diagnosis is unlikely to effect a difference (or lead to harm) in a patient’s life [82], [83], [84].
Students must appreciate that uncertainty is an ever-present factor in the diagnostic process [85], [86], and the reality that this process is inherently error-prone. Diagnostic errors are inevitable; the challenge then is to minimize their likelihood and their impact.
Conclusions
The current paradigm for teaching diagnostic reasoning in medical education is based on recommendations dating from the Flexner report that trainees should acquire an extensive baseline body of medical knowledge and then try to apply this by emulating their teachers. While other health professions have adopted more prospective, intentional approaches, there remains substantial room for improvement. Trainees learn the basic steps of the diagnostic process and can arrive at a diagnosis but may have little or no appreciation for the underlying cognitive psychology, or on the many ways that the process can be degraded in the actual contexts of providing care. The explosion of cognitive psychology literature over the 40 years has led to a revolution in the way that we think about problem solving and decision making but health professions programs have been slow to incorporate this knowledge into their curricula.
Although the apprenticeship model has been the norm in physician education for over a century, we propose that the reliability and safety of diagnosis can be improved by incorporating more specific training on topics relevant to the diagnostic process and diagnostic error. The growing complexity of diagnosis itself (over 8000 diseases are now catalogued by the National Library of Medicine’s MESH system, and new entities are added every year) and our healthcare organizations, along with growing pressures on productivity and performance are all factors that speak to the need for updating our educational programs.
Competency requires a trainee to acquire and demonstrate the requisite knowledge, skills, and attitudes for the task at hand. This project, using a consensus-driven process, identified five key ‘drivers’ through which education could improve diagnostic performance. We believe that every school and program responsible for the training of healthcare professionals should consider which of these competencies are relevant and essential for their trainees. For medicine, nursing, physician assistant trainees, and for professions where diagnosis is a primary responsibility (e.g. podiatry, optometry, dentistry, etc.), all of the competencies will be relevant and appropriate. Even in professions where diagnosis is not a primary responsibility (e.g. social work services, etc.), some of the competencies will be relevant, for example by emphasizing the importance of communication and coordination amongst all the healthcare professionals a patient will encounter. All health professionals should be trained to identify breakdowns in communication and/or understanding between patients and their care team, and bring these to attention.
Pioneering efforts are already emerging at selected schools to incorporate the educational reforms we are advocating [57], [87]. Many medical residency programs have long-standing curriculum about diagnosis and diagnostic error, and selected medical schools have recently developed and refined courses in diagnosis and clinical reasoning. Schools that are contemplating how to incorporate these new concepts may be able to derive guidance from these early endeavors. Further, many other health professions programs have robust simulation curricula focused on teamwork and communication. A major unmet need for all programs is a clear delineation of the specific competencies required in each of the cardinal dimensions of the driver diagram, as well as assessment tools to measure progress towards attaining competence.
The focus of this project was on undergraduate and postgraduate health professions training and the next generations of clinical providers. It is likely, however, that continuing education programs directed at health professionals already in practice could also produce improvements in diagnostic quality and safety.
In summary, addressing diagnostic error in practice is an urgent concern. A promising avenue for improving the diagnostic process is ensuring that the next generation of healthcare professionals are equipped with a more appropriate set of knowledge, skills, attitudes that focus on diagnostic quality and safety in the modern, team-based setting.
Acknowledgments
We are grateful for the funding for this project from the Josiah Macy Jr Foundation to the Society to Improve Diagnosis in Medicine. We sincerely appreciate input and assistance from many individuals who contributed to the success of this work: Anne Pereira, Assistant Dean for Curriculum, University of Minnesota Medical School; Cori Smith, Administrator, Society to Improve Diagnosis in Medicine; Daniel Persky, Project Manager, Society to Improve Diagnosis in Medicine; Chloe Laabs, Administrative Assistant, University of Minnesota Medical School; and the members of the Consensus Committee. Consensus Committee members: Dan Berg, Patient Advocate; Eliana Bonifacino, MD, University of Pittsburgh; Richard Botner, PA, University of Texas; Barbara Brandt, PhD, NCIP, University of Minnesota; Wallace Carter, MD, ABEM; Cristin Colford, MD, University of North Carolina; Cindy Cooke, DNP, FNP-C, AANP; Karen Cosby, MD, University of Chicago; Pat Croskerry, PhD, MD, Dalhousie University; Carol Durham Fowler, EdD, RN, UNC Chapel Hill; Beth Fox, MD, MPH, AAFP; Rick Garcia, RN, PhD, AACN; Laura Gerstner, MHA, PAC, Campbell University; Michael Giuliano, MD, Seton Hall; Mark Graber, MD, SIDM; Gloria Grice, Pharm D, St Louis, College Pharmacy; Gene Harkless, DNSc, APRN, University of New Hampshire; Richard Hawkins, MD, AMA; Joy Higgs, AM, PhD, Charles Sturt University – AU; Eric Holmboe, MD, ACGME; Lisa Howley, MD, AAMC; Mira Irons, MD, ABMS; Mark Johnson, Medical student University of Minnesota Student; Valerie Lang, MD, AAIM; Craig Lenz, DO, Am. College of Osteopathic Medicine; Louis Ling, MD, ACGME; John Luk, MD, University of Texas – Austin; Prashant Mahajan, MD, MPH, AAP; Phil Masters, MD, ACP; Furman McDonald, MD, MPH, ABIM; Karen McDonough, MD, University of Washington; Stacy Melanson, MD, PhD, Brigham & Women’s Hospital; Alexa Miller, Arts Practica; Sandra Monteiro, PhD, MSc McMaster University; Virginia Moyer, MD, MPH, Am Board Peds; Jen Myers, MD, University of Pennsylvania; Beth Nelson, MD, University of Texas – Austin; Andrew P.J. Olson, MD, University of Minnesota; Dianne Padden, PhD, CRNP, AANP; Frank Papa, DO, PhD, TCOM; Elissa Passiment, CDC; Anne Pereira, MD, MPH, University of Minnesota; Joseph Rencic, MD, Tufts; Emily Ruedinger, MD, Seattle Childrens; Diana Rusz, MPH, SIDM; Steve Schoenbaum, MD, Macy Foundation; Joan Stanley, PhD, CRNP, AACN; Michael Tang, MD, PhD, NY-Presbyterian Hospital; Joan Von Feldt, MD, MSEd, University of Pennsylvania; Brenda Zierler, PhD, RN, University of Washington; Meg Zomorodi, PhD, RN, CNL, University of North Carolina Chapel Hill.
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: Josiah Macy Jr. Foundation, Funder Id: 10.13039/100006152, B17-02.
Employment or leadership: None declared.
Honorarium: None declared.
References
1. Institute of Medicine. Improving Diagnosis in Health Care. Washington, DC: National Academies Press, 2015.Search in Google Scholar
2. Graber ML, Gordon RR, Franklin N. Diagnostic error in internal medicine. Annual Meeting: National Patient Safety Foundation, 2004.10.1001/archinte.165.13.1493Search in Google Scholar PubMed
3. Singh H, Traber D, Meyer A, Forjuoh S, Reis M, Thomas E. Types and origins of diagnostic errors in primary care settings. JAMA Internal Med 2013;173:418–25.10.1001/jamainternmed.2013.2777Search in Google Scholar PubMed PubMed Central
4. Singh H, Meyer A, Thomas E. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Qual Saf 2014;23:727–31.10.1136/bmjqs-2013-002627Search in Google Scholar PubMed PubMed Central
5. Troxel D. Diagnostic Error in Medical Practice by Specialty. The Doctor’s Advocate. 2014; Third Quarter. Available at: http://www.thedoctors.com/KnowledgeCenter/Publications/.Search in Google Scholar
6. Leape L, Berwick D, Bates D. Counting deaths from medical errors. J Am Med Assoc 2002;288:2405.10.1001/jama.288.19.2405-JLT1120-2-3Search in Google Scholar
7. Rencic J, Trowbridge R, Fagan M, Szauter K, Durning S. Clinical reasoning education at US Medical Schools: results from a National Survey of Internal Medicine Clerkship Directors. J Gen Intern Med 2017;32:1242–6.10.1007/s11606-017-4159-ySearch in Google Scholar PubMed PubMed Central
8. Powell D, Caraccio C. Toward competency based medical education. N Engl J Med 2018;378:1–5.10.1056/NEJMp1712900Search in Google Scholar
9. Pereira A, Woods M, Olson A, van den Hoogenhof S, Duffy B, Englander R. Criterion-based assessment in a norm-based world: how can we move past grades? Acad Med 2018;93:560–4.10.1097/ACM.0000000000001939Search in Google Scholar PubMed
10. Graber M. The incidence of diagnostic error. BMJ Qual Saf 2013;22, Part 2:ii21–ii7.10.1136/bmjqs-2012-001615Search in Google Scholar PubMed PubMed Central
11. Reed J, McNicholas C, Woodcock T, Issen L, Bell D. Designing quality improvement initiatives: the action effect method, a structured approach to identifying and articulating programme theory. BMJ Qual Saf 2014;23:1040–8.10.1136/bmjqs-2014-003103Search in Google Scholar PubMed
12. Humphrey-Murto S, Varpio L, Wood T, Gonsalves C, Ufholz L-A, Mascioli K, et al. The use of the delphi and other consensus group methods in medical education research: a review. Acad Med 2017;92:1491–8.10.1097/ACM.0000000000001812Search in Google Scholar PubMed
13. Englander R, Cameron T, Ballard A, Dodge J, Bull J, Aschenbrenner C. Toward a common taxonomy of competency domains for the health professions and competencies for physicians. Acad Med 2013;88:1088–94.10.1097/ACM.0b013e31829a3b2bSearch in Google Scholar PubMed
14. Frank J, Snell L, Sherbino J. CanMEDS 2015 Physician Competency Framework. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada, 2015.Search in Google Scholar
15. Verghese A, Charlton B, Kassirer J, Ramsey M, Ionnidis J. Inadequacies of physical examination as a cause of medical errors and adverse events: a collection of Vignettes. Am J Med 2015;128:1322–4.10.1016/j.amjmed.2015.06.004Search in Google Scholar PubMed
16. Schattner A. Revitalizing the history and clinical examination. Am J Med 2012;125:e1–3.10.1016/j.amjmed.2011.05.013Search in Google Scholar PubMed
17. Stark M, Fins J. The ethical imperative to think about thinking. Cambridge Quart Healthc Ethics 2014;23:386–96.10.1017/S0963180114000061Search in Google Scholar PubMed
18. Lucien Leape Institute. Unmet needs – teaching physicians to provide safe patient care. Available at: wwwihiorg. 2010.Search in Google Scholar
19. Huang G, Newman L, Schwartzstein R. Critrical thinking in health professions education: summary and consensus statements of the Millenium Conference 2011. Teach Learn Med 2014;26:95–102.10.1080/10401334.2013.857335Search in Google Scholar PubMed
20. Dhaliwal G. Premature closure? Not so fast. BMJ Qual Saf 2016; doi: 10.1136/bmjqs2016005267, Available at: http://dx.doi.org/10.1136/bmjqs-2015-005014.10.1136/bmjqs2016005267Search in Google Scholar
21. Dhaliwal G. The evolution of the master diagnostician. J Am Med Assoc 2013;310:579–60.10.1001/jama.2013.7572Search in Google Scholar PubMed
22. Eva KW. What every teacher needs to know about clinical reasoning. Med Educ 2005;39:98–106.10.1111/j.1365-2929.2004.01972.xSearch in Google Scholar PubMed
23. Norman G. Building on experience – the development of clinical reasoning. N Engl J Med 2006;355:2251–2.10.1056/NEJMe068134Search in Google Scholar PubMed
24. Maude J. Differential diagnosis: the key to reducing diagnosis error, measuring diagnosis, and a mechanism to reduce healthcare costs. Diagnosis 2014;1:107–9.10.1515/dx-2013-0009Search in Google Scholar PubMed
25. Papa F. Learning sciences principles that can inform the construction of new approaches to diagnostic training. Diagnosis 2014;1:125–9.10.1515/dx-2013-0013Search in Google Scholar PubMed
26. van Merrienboer J, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ 2010;44:85–93.10.1111/j.1365-2923.2009.03498.xSearch in Google Scholar PubMed
27. Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med 2006;355:2217–25.10.1056/NEJMra054782Search in Google Scholar PubMed
28. Eva KW. Diagnostic error in medical education: where wrongs can make rights. Adv Health Sci Educ Theory Pract 2009;14(Suppl 1):71–81.10.1007/s10459-009-9188-9Search in Google Scholar PubMed
29. Dhaliwal G. Developing teachers of clinical reasoning. Clin Teacher 2013;10:313–7.10.1111/tct.12082Search in Google Scholar PubMed
30. Papa F, Stone R, Aldrich D. Further evidence of the relationshp between case typicallity and diagnostic performance: implications for medical education. Acad Med 1996;71:510–12.10.1097/00001888-199601000-00028Search in Google Scholar PubMed
31. Papa F, Li F. Evidence for the preferential use of disease prototypes over case exemplars among year one medical students prior to and following diagnostic training. Diagnosis 2015;2:217–25.10.1515/dx-2015-0024Search in Google Scholar PubMed
32. Croskerry P. A universal model of diagnostic reasoning. Acad Med 2009;84:1022–8.10.1097/ACM.0b013e3181ace703Search in Google Scholar PubMed
33. Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what’s the goal? Acad Med 2002;77:981–92.10.1097/00001888-200210000-00009Search in Google Scholar PubMed
34. Friedlander M, Andrews L, Armstrong E, Aschenbrenner C, Kass J, Ogden P, et al. What can medical education learn from the neurobiology of learning? Acad Med 2011;86:415–20.10.1097/ACM.0b013e31820dc197Search in Google Scholar PubMed
35. Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005;39:418–27.10.1111/j.1365-2929.2005.02127.xSearch in Google Scholar PubMed
36. Eva KW, Hatala RM, LeBlanc VR, Brooks LR. Teaching from the clinical reasoning literature: combined reasoning strategies help novice diagnosticians overcome misleading information. Med Educ 2007;41:1152–8.10.1111/j.1365-2923.2007.02923.xSearch in Google Scholar PubMed
37. Ilgen J, Bowen J, Yarris L, Fu R, Lowe R, Eva K. Adjusting our lens: can developmental differences in diagnostic reasoning be harnessed to improve health professional and trainee assessment. Acad Emerg Med 2011;18(Suppl 2):S79–86.10.1111/j.1553-2712.2011.01182.xSearch in Google Scholar PubMed PubMed Central
38. Ilgen J, Humbert A, Kuhn G, Hanse M, Norman G, Eva K, et al. Assessing diagnostic reasoning: a consensus statement summarizing theory, practice, and future needs. Acad Emerg Med 2012;19:1454–61.10.1111/acem.12034Search in Google Scholar PubMed
39. Croskerry P. A model for decision-making in medicine. Med Sci Educator 2017;27(Suppl 1):9–13.10.1007/s40670-017-0499-9Search in Google Scholar
40. Schwartz A. Medical decision making and medical education: challenges and opportunities. Perspect Biol Med 2011;54:68–74.10.1353/pbm.2011.0010Search in Google Scholar PubMed
41. Redelmeier DA, Ferris LE, Tu JV, Hux JE, Schull MJ. Problems for clinical judgement: introducing cognitive psychology as one more basic science. Can Med Assoc J 2001;164:358–60.Search in Google Scholar
42. Mamede S, Schmidt HG, Rikers R. Diagnostic errors and reflective practice in medicine. J Eval Clin Pract 2007;13:138–45.10.1111/j.1365-2753.2006.00638.xSearch in Google Scholar PubMed
43. Papa F. A dual processing theory based approach to instruction and assessment of diagnostic competencies. Med Sci Educ 2016;26:787–95.10.1007/s40670-016-0326-8Search in Google Scholar
44. Croskerry P. Diagnostic failure: a cognitive and affective approach. Advances in patient safety: from research to implementation. 2. AHRQ Publication Nos. 050021: Agency for Health Care Research and Quality, Rockville, MD, 2005:241–54.Search in Google Scholar
45. Lambe K, O’Reilly G, Kelly B, Curristan S. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual Saf 2016;25:808–20.10.1136/bmjqs-2015-004417Search in Google Scholar PubMed
46. Norman G, Monteiro D, Sherbino J, Ilgen J, Schmidt H, Mamede S. The causes of errors in clinical reasoning: cogntive biases, knowledge deficits, and dual process thinking. Acad Med 2016;92:23–30.10.1097/ACM.0000000000001421Search in Google Scholar PubMed
47. Mamede S, Schmidt HG, Penaforte JC. Effect of reflective practice on accuracy of medical diagnosis. In: Soares SM, editor. Reflective practice in medicine. Brazil, Fortaleza, 2006.Search in Google Scholar
48. Croskerry P, Singhal G, Mamede S. Cognitive debiasing 2: impediments to and strategies for change. BMJ Qual Saf 2013;22(Suppl 2):ii65–72.10.1136/bmjqs-2012-001713Search in Google Scholar PubMed PubMed Central
49. Ludolf R, Schulz P. Debiasing health-related judgments and decision-making: a systematic review. Med Decis Making 2017;38:3–13.10.1177/0272989X17716672Search in Google Scholar PubMed
50. Croskerry P, Cosby K, Graber M, Singh H. Diagnosis – interpreting the shadows. Boca Raton, FL: CRC Press, 2017.10.1201/9781315116334Search in Google Scholar
51. Dror I. A novel approach to minimize error in the medical domain: cognitive neuroscientific insights into training. Med Teacher 2011;33:34–8.10.3109/0142159X.2011.535047Search in Google Scholar PubMed
52. Holmboe E, Durning S. Assessing clinical reasoning: moving from in vitro to in vivo. Diagnosis 2014;1:111–8.10.1515/dx-2013-0029Search in Google Scholar PubMed
53. Henriksen K, Brady J. The pursuit of better diagnostic performance: a human factors perspective. BMJ Qual Saf 2013;22(Suppl 2):1–5.10.1136/bmjqs-2013-001827Search in Google Scholar PubMed PubMed Central
54. Weiner S, Schwartz A. Contextual errors in medical decision making: overlooked and understudied. Acad Med 2016;91: 657–62.10.1097/ACM.0000000000001017Search in Google Scholar PubMed
55. Chi MT, Glaser R, Farr MJ. The Nature of Expertise. Hillsdale, NJ: Lawrence Erlbaum Associates, 1988.Search in Google Scholar
56. Ericsson K. Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med 2015;90:1471–86.10.1097/ACM.0000000000000939Search in Google Scholar PubMed
57. Ruedinger E, Olson M, Yee J, Borman-Shoap E, Olson C. Education for the next frontier in patient safety: a longitudinal resident curriculum on diagnostic error. Am J Med Qual 2017;32:625–31.10.1177/1062860616681626Search in Google Scholar PubMed
58. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 2008;15:988–94.10.1111/j.1553-2712.2008.00227.xSearch in Google Scholar PubMed
59. Sharples J, Oxman A, Mahtani K, Chalmers I, Oliver S, Collins K, et al. Critical thinking in healthcare and education. BMJ 2017;357:j2234.10.1136/bmj.j2234Search in Google Scholar PubMed
60. Croskerry P. Critical thinking and reasoning in emergency medicine. In: Croskerry P, Cosby K, Schenkel S, Wears R, editors. Patient safety in emergency medicine. New York, NY: Wolters Kluwer, Lippincott, 2013.Search in Google Scholar
61. Warm E, Mathis B, Held J, Tolentino J, Ashbrook L, Lee C, et al. Entrustment and mapping of observable practice activities for resident assessment. J Gen Int Med 2014;29:1177–82.10.1007/s11606-014-2801-5Search in Google Scholar PubMed PubMed Central
62. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ 2005;39:1176–77.10.1111/j.1365-2929.2005.02341.xSearch in Google Scholar PubMed
63. Cooke N, Salas E, Kiekel P, Bell B. Advances in measuring team cognition. In: Salas E, Fiore S, editors. Team cognition. Washington, DC: American Psychological Association, 2004.10.1037/10690-005Search in Google Scholar
64. Wong B, Etchells E, Kuper A, Levinson W, Shojania K. Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med 2010;85:1425–39.10.1097/ACM.0b013e3181e2d0c6Search in Google Scholar PubMed
65. Kirkman M, Sevdalis N, Arora S, Baker P, Vincent C, Ahmed M. The outcomes of recent patient safety education interventions for trainee physicians and medical students: a systematic review. BMJ Open 2015;5:e007705.10.1136/bmjopen-2015-007705Search in Google Scholar PubMed PubMed Central
66. Glavin R, Maran N. Integrating human factors into the medical curriculum. Med Educ 2003;37(Suppl 1):59–64.10.1046/j.1365-2923.37.s1.5.xSearch in Google Scholar PubMed
67. Bleetman A, Sanusi S, Dale T, Brace S. Human factors and error prevention in emergency medicine. Emerg Med 2012;29:389–93.10.1136/emj.2010.107698Search in Google Scholar PubMed
68. Singh H, Graber M, Kissam S, Sorensen AV, Lenfestey NF, Tant EM, et al. System-related interventions to reduce diagnostic errors: a narrative review. BMJ Qual Saf 2012;21:160–70.10.1136/bmjqs-2011-000150Search in Google Scholar PubMed PubMed Central
69. Payne V, Singh H, Meyer A, Levey L, Harrison D, Graber M. Patient-initiated second opinions: systematic review of characteristics and impact on diagnosis, treatment, and satisfaction. Mayo Clin Proc 2014;89:687–96.10.1016/j.mayocp.2014.02.015Search in Google Scholar PubMed
70. Riches N, Panagioti M, Alam R, Cheraghi-Sohi S, Campbell S, Esmail A, et al. The effectiveness of electronic differential diagnosis (DDX) generators: a systematic review and meta-analysis. PLoS One 2016;11:e0148991.10.1371/journal.pone.0148991Search in Google Scholar PubMed PubMed Central
71. Duclos A, Peix J, Piriou V, Occelli P, Denis A, Bourdy S, et al. Cluster randomized trial to evaluate the impact of team training on surgical outcomes. Br J Surg 2016;103:1804–13.10.1002/bjs.10295Search in Google Scholar PubMed
72. Agency for Healthcare Research and Quality. TeamSTEPPS. Available at: teamsteppsahrqgov. Accessed: 27 Aug 2017.Search in Google Scholar
73. Klein G. The power of the team mind. Sources of power: how people make decisions. Cambridge, MA: MIT Press, 1998:233–57.Search in Google Scholar
74. Baker D, Salas E, Barach P, Battles J, King H. The relation between teamwork and patient safety. In: Carayon P, editor. Handbook of human factors and ergonomics in health care and patient safety. Mahwah, NJ: Lawrence Erlbaum Associates, Inc, 2006:259–71.Search in Google Scholar
75. Graber M, Rusz D, Jones M, Farm-Franks D, Jones B, Gluck J, et al. The new diagnostic team. Diagnosis 2017;4:225–38.10.1515/dx-2017-0022Search in Google Scholar PubMed
76. Gleason K, Davidson P, Tanner E, Baker D, Baptiste D, Rushton C, et al. Defining the critical role of nurses in diagnostic error prevention: a conceptual framework and a call to action. Diagnosis 2017;4:201–10.10.1515/dx-2017-0015Search in Google Scholar PubMed
77. McDonald K. The diagnostic field’s players and interactions: from the inside out. Diagnosis 2014;1:55–8.10.1515/dx-2013-0023Search in Google Scholar PubMed
78. Smith M, Hughes A, Brown C, Russo E, Giardina T, Mehta P, et al. Test result management and distributed cognition in electronic health record-enabled primary care. Health Inform J 2018; doi:10.1177/1460458218779114.10.1177/1460458218779114Search in Google Scholar PubMed
79. McDonald K, Bryce C, Graber M. The patient is in: patient involvement strategies for diagnostic error mitigation. BMJ Qual Saf 2013;22, Part 2:30–6.10.1136/bmjqs-2012-001623Search in Google Scholar PubMed PubMed Central
80. Graber ML, Franklin N, Gordon RR. Diagnostic error in internal medicine. Arch Int Med 2005;165:1493–9.10.1001/archinte.165.13.1493Search in Google Scholar PubMed
81. Macy Foundation; ABIM Foundation; Robert Wood Johnson Foundation. Team-Based Competencies. Building a Shared Foundation for Education and Clinical Practice. www.aacnncheedu/leading-initiatives/IPECProceedingspdf; 2014.Search in Google Scholar
82. Rosenbaum L. Living unlabeled – Diagnosis and disorder. N Engl J Med 2008;359:1650–3.10.1056/NEJMp0804984Search in Google Scholar PubMed
83. Rosenbaum L. The art of doing nothing. N Engl J Med 2011;365:782–5.10.1056/NEJMp1108914Search in Google Scholar PubMed
84. Dangelo M. The dangers of labeling. Acad Med 2016;91:759.10.1097/ACM.0000000000001191Search in Google Scholar PubMed
85. Luther V, Crandall S. Ambiguity and uncertainty: neglected elements of medical education curricula? Acad Med 2011;86:877–82.10.1097/ACM.0b013e31821da915Search in Google Scholar PubMed
86. Kassirer J. Our stubborn quest for diagnostic certainty. N Engl J Med 1989;320:1489–91.10.1056/NEJM198906013202211Search in Google Scholar PubMed
87. Reilly J, Ogdie A, Von Feldt J, Myers J. Teaching about how doctors think: a longitudinal curriculum in cognitive bias and diagnostic error for residents. BMJ Qual Saf 2013;22:1044–50.10.1136/bmjqs-2013-001987Search in Google Scholar PubMed
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/dx-2018-0033).
©2018 Walter de Gruyter GmbH, Berlin/Boston
