Diagnostic errors represent one of the most pressing patient safety and quality concerns in health care today. In a landmark report, the National Academy of Medicine (National Academy) reviewed the epidemiology and etiology of diagnostic errors, concluding that they are common and harmful. In view of estimates that diagnostic error contributes to 40,000–80,000 deaths annually in the US, the National Academy designated the need to address diagnostic error as an urgent national priority . A reasonable conclusion drawn from these statistics is that the outcomes of our current education programs with respect to diagnosis are not enough; that is, the products of our current education programs – health care professionals in practice – are not achieving desirable levels of diagnostic performance.
The National Academy report outlined a number of possible interventions to reduce the likelihood of diagnostic error. While these interventions are wide-ranging, the top two recommendations of the report were to facilitate more effective teamwork in the diagnostic process among health care professionals, patients, and their families; and enhance health care professional education and training in the diagnostic process. These recommendations aligned with those previously made by the Lucien Leape Institute , the Millenium Report , the Macy Foundation , and others . While these two recommendations at first may seem unrelated, there are important connections between them and the diagnostic process. Diagnosis in modern health care is a team-based social activity, and education programs in the health professions have a moral obligation to equip tomorrow’s health care professionals to be effective members of diagnostic teams. Improving the education and training of health care professionals concerning diagnosis could have both an immediate effect, as well as a sustained impact on reducing harm from diagnostic error for future generations of patients.
There are substantial gaps in health professions education with respect to explicit education about the diagnostic process as well as the team-based and system-related aspects of care. We have previously identified five primary drivers through which education could ultimately improve the practice of diagnosis by addressing these gaps . The five action items identify the need to (1) acquire and effectively use a relevant knowledge base; (2) optimize clinical reasoning to reduce cognitive error; (3) understand system-related aspects of care; (4) effectively engage patients and the diagnostic team; and (5) acquire the appropriate perspectives and attitudes relating to diagnostic quality and safety.
An important contemporary trend that must inform any curricular recommendation or innovation in health professions education is the shift toward competency-based education , . This shift reflects the need to define the outcomes (competencies) that education programs must ensure their learners meet before graduation and measure attainment of those outcomes. This model is in stark contrast to the traditional time-based framework of health professions education, in which learners are generally assumed to have attained competence if they complete the requisite number of years allocated for their education and satisfactorily pass requisite knowledge-based examinations. By better defining these competencies, educators and educational leaders can design curricula to achieve them and assessments to measure this attainment. While diagnosis is implicit – or marginally explicit – in many competency sets, there generally remains a gap in defining competencies for diagnosis and diagnostic teamwork. Defining these competencies is the first step in the curricular change necessary to improve both the quality and safety of diagnosis in health care.
Steering committee and consensus committee
Under the auspices of the Society to Improve Diagnosis in Medicine (SIDM), a five-member steering committee coordinated the efforts of the competency-development process. We selected an interprofessional consensus committee of 32 individuals representing multiple health care professions based on their experience, expertise, position within stakeholder organizations, and/or history of innovation. The committee included professionals and educators from the following fields: laboratory medicine, medicine (including internal medicine, pediatrics, and emergency medicine), nursing, pharmacy, and physician assistants. Students and a patient representative were also key members of the consensus committee. Professional boards and organizations represented on the committee are shown in Table 1.
Foundational concepts and desirable attitudes for diagnosis.
|–||The primacy of medical knowledge|
|–||The dual-processing paradigm as a framework for understanding clinical reasoning|
|–||The value of rational thought and reflection|
|–||The value of experience and of feedback: learning from outcomes of decisions|
|–||The dynamic nature of diagnosis – the inevitable problems of uncertainty and complexity|
|–||The inherent power of teamwork, of patient engagement, and of health information technology|
|–||The relevancy of human factors and “the system” in determining diagnostic performance|
|–||The limits and biases of human cognition|
|–||The value of recognizing high risk and “red flag” situations|
|–||The need to prioritize and triage appropriately|
|–||The need to focus on the needs of the patient|
|–||Assessment of diagnosis should include measures of skillful differential diagnosis and diagnostic efficiency|
|Desirable attitudes (See Appendix C for expanded definitions)|
|Humility||Intellectual autonomy||Resilience and adaptability||Respect|
|Empathy||Kindness||Tolerance of uncertainty||Patience|
Frameworks and definitions
For the purposes of this project, we defined diagnosis as relating to the evaluation of an individual with new symptomatic health concerns and not to screening. The consensus committee agreed on appropriate frameworks to guide the competency-development process, definitions, and guiding principles. (See Supplementary material, Appendix A). We strategically decided to identify general competencies for health care professionals that can be customized for different health professions. Furthermore, our competencies aim to complement (not replace or compete with) existing national competency recommendations on interprofessional health care and communication , . Accordingly, our competencies focus on the co-produced outcome of interprofessional teams in addressing both the quality and safety of diagnosis, setting the expectation that it be timely, accurate, and safe.
The consensus committee was initially divided by self-selection into three working groups (clinical reasoning, diagnostic error, and system-related) with the assignment to identify candidate competencies for diagnosis that should be achieved in the education of health professionals. A scoping literature review was conducted by project staff to identify published articles and reports relevant to the topics of diagnosis or diagnostic error, as well as health care education or training . Each group was provided access to the relevant abstracts retrieved by the literature search and the existing national program competency sets for medicine, nursing, and physician assistant training programs (Supplementary material, Appendix B). Then, through a series of conference calls and approval at a consensus meeting, the committee agreed upon a set of foundational concepts and desirable attitudes relevant to developing diagnostic skill (Table 1 and Supplementary material, Appendix C). While not competencies themselves, these attributes are, in some cases, preconditions necessary for a learner to eventually demonstrate competence in the areas we define.
Next, 52 potential competencies were considered by the three working groups. A 2-day, in-person meeting was used to review, discuss, and revise the draft competency concepts. The original three groups (clinical reasoning, diagnostic error, and system-related issues) were deemed inappropriate for competency categories, and all further work was conducted in three new categories: individual, team-based, and system-related competencies.
A formal Q-sort process was used to rank-order the potential competencies in these three domains with respect to both feasibility (F score) and importance (I score) . The F and I scores were summed to determine an overall score for each potential competency. Subsequently, the highest rated competencies were re-evaluated for wording and, when appropriate, combined to achieve a draft set of 12 competencies. These were further refined and clarified by the executive committee for wording and specificity, and ratified by an email vote of the consensus committee members: 81% approved of all the competencies as written, and the other 19% identified minor changes in wording that were incorporated to produce the final set.
To enable assessment of a learners’ progression toward competence, we developed a prototype milestone set for one profession (medicine). This was developed using a modified Dreyfus model  with progression of observable behaviors from novice to expert.
IRB approval is not applicable to this project.
The major work product developed through the consensus-building process is a set of 12 recommended competencies. We propose that, if met broadly by health care professionals, these competencies would promote and improve diagnostic quality and safety (Table 2).
Twelve key competencies to improve diagnostic quality and safety.
|Individual competencies for diagnosis (I-competencies)|
|I. Demonstrate clinical reasoning to arrive at a justifiable diagnosis (an explanation for a health-related condition)|
|I-1. Accurately and efficiently collect key clinical findings needed to inform diagnostic hypotheses.|
|Use these tools appropriately and efficiently in the diagnostic process: effective interpersonal communication skills, history-taking, the physical examination, and record review, diagnostic testing, and the electronic health record and health IT resources.|
|I-2. Formulate, or contribute to, an accurate problem representation expressed in a concise summary statement that includes essential epidemiological, clinical, and psychosocial information.|
|I-3. Produce, or contribute to, a correctly prioritized, relevant differential diagnosis, including “can’t-miss” diagnoses.|
|I-4. Explain and justify the prioritization of the differential diagnosis by comparing and contrasting the patient’s findings and test results with accurate knowledge about prototypical or characteristic disease manifestations and atypical presentations, and considering pathophysiology, disease likelihood, and clinical experience.|
|I-5. Use decision support tools, including point-of-care resources, checklists, consultation, and second opinions to improve diagnostic accuracy and timeliness.|
|I-6. Use reflection, surveillance, and critical thinking to improve diagnostic performance and mitigate detrimental cognitive bias throughout the clinical encounter. Discuss and reflect on the strengths and weaknesses of cognition, the impact of contextual factors on diagnosis, and the challenges of uncertainty. Demonstrate awareness of atypical presentations, information that is missing, and key findings that don’t “fit.”|
|Team-based competencies for diagnosis (T-competencies)|
|T. Partner effectively as part of an interprofessional diagnostic team. Communicate effectively and solicit information from all members of the team (including the patient and family) to create a shared mental model of a patient’s illness and the plan for diagnostic evaluation.|
|T-1. Engage and collaborate with patients and families, in accordance with their values and preferences when making a plan for diagnostic evaluation. Listen actively, encourage questions, and be alert to new or changing information. Explain the diagnostic process, including the patient’s and family’s role in helping to identify the most likely diagnosis. Share appropriately when diagnostic uncertainty exists.|
|T-2. Collaborate with other health care professionals (including nurses, physicians, physician assistants, radiologists, laboratory professionals, pharmacists, social workers, physical therapists, medical librarians, and others) and communicate effectively throughout the diagnostic process. Acknowledge and challenge authority gradients constructively, especially between clinicians, patients, and their families.|
|T-3. Apply effective strategies at transitions of care to facilitate accurate and sufficient information transfer about the diagnosis, including any pending workup and areas of uncertainty. Close the loop on test result communication and clarify expectations with the team for test result follow-up.|
|System-related competencies for diagnosis (S-competencies)|
|S. Identify and understand the systems factors that facilitate and contribute to timely, accurate diagnoses and error avoidance.|
|S-1. Discuss how human factors contribute to diagnostic safety and error by identifying how the work environment influences human performance. Take steps to mitigate common systems factors that detract from diagnostic quality and safety.|
|Use local resources (including people, teams, and technology – especially the electronic health record) effectively and efficiently to optimize patients’ access to health care, diagnostic testing services, and appropriate experts for consultation.|
|S-2. Advance a culture of diagnostic safety that encourages open dialogue and continuous learning from analysis and discussion of excellent diagnostic performance, near misses, and errors.|
|Give and receive feedback at an individual and team level to improve subsequent diagnostic performance.|
|S-3. Disclose diagnostic errors and missed opportunities transparently and in a timely manner to patients, families, team members, supervisors, and appropriate quality and risk management staff.|
The literature review conducted in the early phases of the project highlighted several key findings. First, the dual process paradigm (composed of intuitive and rational components) provides a useful and now well-accepted general framework for understanding the cognitive processes that underlie and enable clinical reasoning . Second, the practice environment and health care system both have a dramatic impact on individual and team performance, and ultimately on the diagnostic process itself , . An additional important theme is that improving the safety and quality of diagnosis could be aided by engaging patients more effectively in the diagnostic process . In this concept, diagnosis is co-produced by patients and health professionals. Each of these themes is reflected in one or more of the 12 competencies.
The 12 key competencies ultimately accepted are listed in Table 2, divided into three domains:
- –Individual competencies relate to the knowledge, skills, and attitudes that a health care professional must demonstrate on an individual level in order to contribute in their specific role to the diagnostic process.
- –Team-based competencies relate to the knowledge, skills, and attitudes that a health care professional must demonstrate in collaboration as a member of the diagnostic team.
- –Systems-based competencies relate to the knowledge, skills, and attitudes that a health care professional must demonstrate in relation to how the diagnostic process operates within a particular health care system.
An extensively referenced set of learning objectives was created for each competency (Appendix E). These include specific learning objectives for diagnostic testing (both laboratory and imaging), and using health information technology. The specificity of these learning objectives facilitates curricular development and innovation in each competency domain.
In order to inform assessment of competence, we developed a prototype milestone for Competency I-1 for the profession of medicine (Table 3). Milestones for the remaining competencies and professions can be constructed using this prototype, with the understanding that these may differ, or be less relevant, across the various health care professions.
Example of a five-tiered milestone progression for Competency I-1.
|Performs the clinical examination accurately and thoroughly but in a rote, non-hypothesis-driven manner. Relies heavily on data obtained and impressions formed by others and incompletely confirms clinical findings||Performs the clinical examination accurately and comprehensively and sometimes in a hypothesis-driven manner. Confirms most key findings and forms independent impressions||Performs the clinical examination in an efficient, hypothesis-driven manner. Confirms all key findings and forms independent impressions that take patient viewpoint into account||Role models performing all elements of the clinical examination in an efficient, evidence-based, hypothesis-driven manner. Confirms key findings and forms independent impressions that take patient viewpoint into account|
We believe that one of the most promising and sustainable ways to improve diagnosis is to improve education and training in the health professions. The first step in this process is to define the outcomes that trainees in each profession must achieve in order to be effective members of a diagnostic team in the modern health care setting, and in this paper we define these competencies for the first time.
The consensus committee endorsed the primacy of content-specific knowledge as a fundamental requirement for effective diagnostic reasoning. Factual knowledge alone, however, is necessary but not sufficient. Transmitting this knowledge base and incorporating it into practice is the current focus of education and training in every health care profession. What is novel in our set of competency recommendations is the expectation that education and training need to impart additional knowledge, skills, and attitudes that are not typically covered. In the domain of individual competencies, this includes, for example, knowing how and when to effectively get help with diagnosis (Competency I-5) through second opinions or taking advantage of decision-support tools for differential diagnosis. Similarly, Competency I-6 conveys the expectation to employ critical thinking and reflection in the diagnostic process.
While the recommended competencies for individual diagnostic performance reflect and refine much of what has been written in recent decades about the development of decision-making in health professions education, the new competency set also specifies team-based and system-related competencies that are novel, and not typically included in current curricular objectives. Working in teams was the top recommendation for reducing diagnostic errors in the National Academy report. This area is markedly underdeveloped from a curricular standpoint. It must be noted that we define the health care team very broadly and believe firmly that this team always involves the patient. Achieving effective communication is a key attribute of high-functioning teams, and breakdowns in communication are the most commonly encountered problems contributing to diagnostic error .
It is also increasingly apparent that education programs in the health professions must equip their trainees to practice within the modern health care system. Health systems science is an emerging area of emphasis, and our system-related competencies define three key aspects that relate to diagnostic performance. These include calls for transparency in disclosing diagnostic error and promoting a culture of diagnostic safety. Obtaining feedback on diagnostic performance is essential to improvement, and this feedback must consistently occur at the level of the individual, teams, and health care systems.
In this spirit, we believe that these new competencies are appropriate for the education and training of every health care professional involved in diagnosis. This includes professionals in fields that traditionally have been viewed as peripheral to diagnosis, such as pharmacists, social workers, physical therapists, and a wide range of additional health professionals. These professionals contribute to the diagnostic process through their own unique observations, and to the safety of diagnosis by participating in communication, health care coordination, patient monitoring, and helping to detect diagnostic errors.
There are limitations to our methodology and competencies. First, the competency recommendations are generally based on expert opinion and not empirical studies. As an example, while it seems likely that team-based strategies will improve the outcome of the diagnostic process, this has not been studied on a large scale. In addition, it is possible that, despite our inclusive approach, there are voices and opinions of other stakeholders that are not represented in our final project. A final issue concerns how priorities were assigned; a different group might have chosen other competencies to include, for example items that we classified as “learning objectives”.
There are three major limitations that, if not addressed, will limit the impact of this project:
First, the curricula in health professions education programs are already “full.” Most educational programs are extremely limited in what new concepts and activities can be added to their existing programming. Thus, these competencies must be mapped to existing activities whenever possible.
Second, much of the content of educational programs in health professions education programs is driven by accreditors and other stakeholder organizations. Thus, we must engage with these organizations to gain their endorsement. A “top-down” approach will encourage curricular innovation and give innovative faculty members more impetus for their drive to improve diagnosis through education.
The final limitation is that there are significant gaps with respect to valid and pragmatic assessment tools to measure attainment of these competencies. This will be a focus of much work for many in the coming years, and we believe that this assessment gap may now be filled once the outcomes (competencies) have been better defined. In particular, there is a dearth of instructional and assessment tools focused on team-based diagnosis, a novel concept for most health care professions , , . Although there are many well-described instructional activities and programs aimed at improving teamwork, these generally do not address the final outcome that the team achieves, particularly as this relates to diagnostic performance. Although there are some exciting early demonstrations in the literature , , there are currently no widely adopted, validated means to assess the diagnostic performance of a team.
Identifying the desirable diagnosis-related competencies of health professions education programs is necessary, but is just the first step. To actually improve education (and ultimately diagnostic performance and health), specific, bold steps must be taken by major stakeholders responsible for defining, delivering, and measuring educational outcomes. We suggest the following:
First, each profession and its relevant overarching educational organization(s) need to review the competencies and determine how each competency relates to existing program competencies and the role of that profession. The competencies we propose are potentially relevant to every health profession, but each profession will need to conduct their own crosswalk to understand what is already included, and what needs to be added or modified.
Second, each board that certifies health professionals and each body that accredits schools and programs will need to consider how their requirements need to be clarified or modified to explicitly recognize competence in diagnosis. New competency expectations should be added to the educational blueprint that schools follow in designing their own curriculum.
Third, each school will need to find existing content or develop their own to teach the relevant competencies, and provide both formative and summative assessment, including tactics such as simulation and direct observation.
Members of the education community could facilitate these actions by developing relevant teaching modules and identifying standardized ways to assess the new competencies, such as those addressing team-based diagnosis.
Diagnosis is the foundation of medical practice. It is remarkable that in the training of most health professionals, there is no specific course on diagnosis. Diagnosis education mirrors diagnostic practice: both are often idiosyncratic and not standardized. The past two decades have seen important growth in understanding the mechanisms that contribute to the harm associated with diagnostic error. The diagnostic process is complex, plays out over time and space, and uncertainty is the norm. Various cognitive and system-related contributions to diagnostic error have now been identified. We believe that incorporating this new knowledge into health professions education and training has the potential to substantially improve the diagnostic process and reduce the harm associated with diagnostic errors.
This project was supported by an award from the Josiah Macy Jr. Foundation to the Society to Improve Diagnosis in Medicine. Consensus committee members: Dan Berg, Patient Advocate; Eliana Bonifacino, MD, University of Pittsburgh; Richard Botner, PA, University of Texas; Barbara Brandt, PhD, NCIP, University of Minnesota; Wallace Carter, MD, American Board of Emergency Medicine; Cristin Colford, MD, University of North Carolina; Jeannine M Conway, PharmD, Universitiy of Minnesota; Cindy Cooke, DNP, FNP-C, American Association of Nurse Practitioners; Karen Cosby, MD, Rush Medical College; Pat Croskerry, PhD, MD, Dalhousie University; Carol Durham Fowler, EdD, RN, University of North Carolina, Chapel Hill; Beth Fox, MD, MPH, American Association of Family Practitioners; Rick Garcia, RN, PhD, American Association of Colleges of Nursing; Laura Gerstner, MHA, PAC, Campbell University; Michael Giuliano, MD, MHPE, Hackensack Meridian School of Medicine at Seton Hall; Mark Graber, MD, Society to Improve Diagnosis in Medicine; Gloria Grice, Pharm D, St Louis College of Pharmacy; Gene Harkless, DNSc, APRN, University of New Hampshire; Richard Hawkins, MD, American Medical Association; Eric Holmboe, MD, Accreditation Council for Graduate Medical Education; Lisa Howley, MD, Association of American Medical Colleges; Mira Irons, MD, American Board of Medical Specialties; Mark Johnson, Medical student, University of Minnesota; Valerie Lang, MD, Alliance for Academic Internal Medicine; Louis Ling, MD, Accreditation Council for Graduate Medical Education; John Luk, MD, University of Texas–Austin; Prashant Mahajan, MD, MPH, Association of American Physicians; Phil Masters, MD, American College of Physicians; Furman McDonald, MD, MPH, American Board of Internal Medicine; Karen McDonough, MD, University of Washington; Stacy Melanson, MD, PhD, Brigham and Women’s Hospital; Alexa Miller, Arts Practica; Sandra Monteiro, PhD, MSc McMaster University; Virginia Moyer, MD, MPH, American Board of Pediatrics; Jen Myers, MD, University of Pennsylvania; Beth Nelson, MD, Univ Texas – Austin; Andrew Olson, MD, University of Minnesota; Dianne Padden, PhD, CRNP, American Association of Nurse Practitioners; Frank Papa, DO, PhD, Texas College of Osteopathic Medicine; Elissa Passiment, Centers for Disease Control; Anne Pereira, MD, MPH, University of Minnesota; Joseph Rencic, MD, Tufts University School of Medicine; Emily Ruedinger, MD, Seattle Children’s Hospital; Diana Rusz, MPH, Society to Improve Diagnosis in Medicine; Steve Schoenbaum, MD, Macy Foundation; Joan Stanley, PhD, CRNP, American Association of Colleges of Nursing; Michael Tang, MD, PhD, Presbyterian Hospital, NY; Joan Von Feldt, MD, MSEd, University of Pennsylvania; Brenda Zierler, PhD, RN, University of Washington; Meg Zomorodi, PhD, RN, CNL, University of North Carolina, Chapel Hill.
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Institute of Medicine. Improving diagnosis in health care. Washington, DC: National Academies Press, 2015.
Huang G, Newman L, Schwartzstein R. Critrical thinking in health professions education: Summary and consensus statements of the Millenium Conference 2011. Teach Learn Med Int J 2014;26:95–102.
Macy Foundation. Team-based competencies. building a shared foundation for education and clinical practice, 2014. https://macyfoundation.org/news-and-commentary/ipec-reports.
Graber M, Rencic J, Rusz D, Papa F, Croskerry P, Zierler B, et al. Improving diagnosis by improving education: a policy brief on education in healthcare professions. Diagnosis 2018;5:107–18.
ten Cate O, Billet S. Competency-based medical education: origins, perspectives, and potentialities. Med Educ 2014;48:325–32.
Bataldan P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff 2002;21:103–11.
NQF endorses practices for safe lab medicine. Healthcare Benchmarks Qual Improv 2009;16:76–8.
Singh H, Sittig D. Advancing the science of measurement of diagnostic errors in healthcare: the Safer DX framework. BMJ Qual Saf 2015;24:103–10.
Carayon P, Schoofs Hunt A, Karsh B-T, Gurses A, Alvarado C, Smith M, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006;15(Suppl 1):i50–8.
McDonald K, Bryce C, Graber M. The Patient is in: patient involvement strategies for diagnostic error mitigation. BMJ Qual Saf 2013;22(Part 2):30–6.
Graber M, Rusz D, Jones M, Farm-Franks D, Jones B, Gluck J, et al. The new diagnostic team. Diagnosis 2017;4:225–38.
Thomas D, Newman-Toker D. Diagnosis is a team sport – partnering with allied health professionals to reduce diagnostic errors: a case study on the role of a vestibular therapist in diagnosing dizziness. Diagnosis 2016;3:49–59.
Gleason K, Davidson P, Tanner E, Baker D, Baptiste D, Rushton C, et al. Defining the critical role of nurses in diagnostic error prevention: a conceptual framework and a call to action. Diagnosis 2017;4:201–10.
Weingart S, Yaghi O, Wetherall M, Sweeney M. Measuring medical housestaff teamwork performance using multiple direct observation instruments: comparing apples and apples. Acad Med 2018;93:1064–70.
Blondon KS, Maitre F, Muller-Juge V, Bochatay N, Cullati S, Hudelson P, et al. Interprofessional collaborative reasoning by residents and nurses in internal medicine: evidence from a simulation study. Med Teach 2017;39:360–7.
The online version of this article offers supplementary material (