A workshop to train medicine faculty to teach clinical reasoning

Verity Schaye 1 , 2 , Michael Janjigian 2 , 3 , Kevin Hauck 3 , 4 , Neil Shapiro 3 , 5 , Daniel Becker 3 , 5 , Penelope Lusk 3 , Khemraj Hardowar 3 , Sondra Zabar 3 ,  and Anne Dembitzer 3 , 5
  • 1 Department of Medicine, New York University School of Medicine, 462 1st Avenue, New York, NY 10016, USA
  • 2 NYC Health and Hospitals/Bellevue, New York, NY, USA
  • 3 New York University School of Medicine, New York, NY, USA
  • 4 NYU Langone Health, New York, NY, USA
  • 5 VA New York Harbor Healthcare System, New York, NY, USA
Verity Schaye
  • Corresponding author
  • Department of Medicine, New York University School of Medicine, 462 1st Avenue, New York, NY 10016, USA
  • NYC Health and Hospitals/Bellevue, New York, NY, USA
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Michael Janjigian
  • NYC Health and Hospitals/Bellevue, New York, NY, USA
  • New York University School of Medicine, New York, NY, USA
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Kevin Hauck, Neil Shapiro
  • New York University School of Medicine, New York, NY, USA
  • VA New York Harbor Healthcare System, New York, NY, USA
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Daniel Becker
  • New York University School of Medicine, New York, NY, USA
  • VA New York Harbor Healthcare System, New York, NY, USA
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Penelope Lusk, Khemraj Hardowar, Sondra Zabar and Anne Dembitzer
  • New York University School of Medicine, New York, NY, USA
  • VA New York Harbor Healthcare System, New York, NY, USA
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar

Abstract

Background

Clinical reasoning (CR) is a core competency in medical education. Few studies have examined efforts to train faculty to teach CR and lead CR curricula in medical schools and residencies. In this report, we describe the development and preliminary evaluation of a faculty development workshop to teach CR grounded in CR theory.

Methods

Twenty-six medicine faculty (nine hospitalists and 17 subspecialists) participated in a workshop that introduced a framework to teach CR using an interactive, case-based didactic followed by role-play exercises. Faculty participated in pre- and post-Group Observed Structured Teaching Exercises (GOSTE), completed retrospective pre-post assessments (RPPs), and made commitment to change statements (CTCs).

Results

In the post-GOSTE, participants significantly improved in their use of problem representation and illness scripts to teach CR. RPPs revealed that faculty were more confident in their ability and more likely to teach CR using educational strategies grounded in CR educational theory. At 2-month follow-up, 81% of participants reported partially implementing these teaching techniques.

Conclusions

After participating in this 3-h workshop, faculty demonstrated increased ability to use these teaching techniques and expressed greater confidence and an increased likelihood to teach CR. The majority of faculty reported implementing these newly learned educational strategies into practice.

Background

In 2015, the National Academies of Sciences and Medicine put forth several calls to action to decrease diagnostic errors given their significant impact on the healthcare system, including to “Enhance health care professional education and training in the diagnostic process [1]”. There is much debate about which educational strategies can improve the diagnostic process. Strategies grounded in script theory and dual process theory (DPT), focusing on knowledge reorganization and guided reflection, have shown promise [2], [3], [4]. DPT stipulates that diagnostic reasoning arises from two interacting cognitive processes – intuitive, pattern recognition System 1 and slow, analytical System 2; the latter can be engaged through diagnostic time outs and by reflection on the diagnostic process [5]. Educational strategies related to script theory focus on knowledge organization and include teaching about problem representations (sentence summaries of the patient’s presentation in abstract terms) and illness scripts (organized mental representations of diseases stored in memory) [2], [4], [6].

Many medical schools lack an explicit curriculum in clinical reasoning (CR) citing limited faculty expertise among the barriers [7]. Learners value faculty who can explain their CR in a structured way, but faculty struggle when demonstrating their own reasoning without a framework [8], [9], [10], [11], [12]. Therefore, faculty development efforts are needed to train faculty to teach CR. Few reports of such programs exist [10], [11], [12]. In this report, we describe the development and preliminary evaluation of a workshop to train faculty to teach CR using educational strategies from the CR literature.

Methods

This workshop was one of three in a year-long faculty development program (FDP) (from November 2017 to November 2018) to improve teaching skills. Each workshop focused on a different skill (CR, observation and feedback, and assessing the struggling learner). Topics covered were based on a needs assessment of key stakeholders including interviews with medical school deans and program directors as well as a survey of program participants. This work met the New York University School of Medicine’s criteria for certification as a quality improvement project (not a human subjects research project) using a self-certification process to ensure that the data were not collected for research purposes, there was minimal risk of harm, and the primary goal of the project was to improve teaching performance.

Group Observed Structured Teaching Exercise

To assess baseline performance, faculty participated in a Group Observed Structured Teaching Exercise (GOSTE) in November 2017. Based on the work of Holmboe et al., we chose a GOSTE rather than an OSTE to foster collaborative and peer-assisted learning [13], [14]. The GOSTE case simulated ward round teaching: participants were tasked with leading a discussion with three standardized learners (SLs) (medical student, intern, and resident) focusing on the learners’ CR in their approach to a case. SLs evaluated faculty performance on a scoring rubric on a three-point Likert scale with behavioral anchors for “not done”, “partially done”, and “well done” on four skills: use of problem representation and illness scripts, approaches to broadening the differential diagnosis, strategies to prioritize the differential diagnosis, and use of diagnostic time outs. All SLs underwent two, 3-h training sessions including one session focused on rubric scoring and calibration of scores. One year after the pre-GOSTE, participants completed a post-GOSTE (the same case as the pre-GOSTE) to assess change in performance.

Workshop

We developed the content for this 3-h workshop using our needs assessment, educational literature on teaching CR, and our years of experience implementing CR curriculum in our medicine residency program (Table 1) [4], [15], [16]. We introduced participants to the impact diagnostic errors have on the healthcare system and evidence-based educational strategies to teach CR. We reviewed the concepts of problem representation, illness scripts, and diagnostic time outs in a case-based interactive didactic. Subsequently, participants engaged in role-play exercises on teaching CR on ward rounds and received guided feedback from trained facilitators and program participants.

Table 1:

Clinical reasoning workshop outline.

Workshop introduction (15 min):
Overview of agenda, impact of diagnostic errors, review of pre-GOSTE findings
Clinical reasoning framework (1 h):
Case-based interactive didactic introducing a framework to teach clinical reasoning
Problem representation

Defining characteristics of the patient summary in abstract terms, unified medical concepts, and semantic qualifiers
Broaden the differential

Systematic approach

Worst case scenario
Meant to aid in recall of diagnosis in the way it was originally learned (illness script)
Illness script

Mental representation of a disease including:

– Predisposing conditions

– Pathophysiologic insult

– Cinical consequences
Prioritize the differential

Compare/contrast problem representation to illness scripts of top diagnoses
Diagnostic time out

What else could this be?

Is there any data that doesn’t fit?

Is there more than one process going on?

What cognitive biases am I prey to?
Application of teaching using clinical reasoning framework (1 h):
In groups of 3–4 participants, guided by trained facilitators, participants engaged in role-play exercises with participants each taking turns playing the role of teacher and learners (student, intern, resident, and/or fellow) on ward rounds.
Debrief (10 min):
Reflection on the workshop and discussion of take home points to be incorporated into practice
Completion of retrospective pre-post assessments and commitment to change statements (15 min):
Brief introduction of CTCs and importance of creating specific plans and then time to complete RPPs and CTCs

At the end of the workshop, participants completed (1) retrospective pre-post assessments (RPPs) that measured change in confidence and likelihood of using their newly acquired skills on an eight-point Likert scale (ranging from “not at all likely/confident” to “very likely/confident”) and (2) commitment to change statements (CTCs). After 2 months, participants self-reported whether they had fully, partly, or not yet implemented (without anchoring statements) their CTCs as well as barriers to implementation.

Data analysis

Results of the RPPs were analyzed using a Wilcoxon signed-rank test. Results of the GOSTE were dichotomized into “not done/partly done” and “well done” (allowing the investigators to accommodate for rater bias) and results were reported as percent “well done”. An independent sample t-test, adjusted according to Levene’s test of homogeneity of variance, was conducted to compare the effect of the GOSTE from pre- to post- on each domain of the scoring rubric.

Results

Twenty-six medicine faculty (nine hospitalists and 17 subspecialists), practicing for 1–17 years (median of 5), completed the CR workshop in December 2017. Twenty faculty completed both the pre- and post-GOSTE. Participants significantly improved in their use of problem representation and illness scripts from pre- to post-GOSTE. In the pre-GOSTE, 10% [standard deviation (SD)=30.5%] received a rating of “well-done” compared to 38% (SD=49.8%) in the post-GOSTE (p=0.028). There was no significant difference in the “well done” ratings between the pre- to post-GOSTE for use of diagnostic time outs, teaching approach to broadening the differential diagnosis, or strategies to prioritize the differential diagnosis.

After participating in this workshop, faculty were significantly more confident in their ability and more likely to teach CR using problem representations, illness script theory, and diagnostic time outs with a large effect size in all domains (Figure 1).

Figure 1:
Figure 1:

Retrospective pre-post assessment results.

*PR, problem representation; IS, illness script.

Citation: Diagnosis 6, 2; 10.1515/dx-2018-0059

In CTCs, participants created specific plans for incorporating CR teaching strategies into their educational settings. At 2-month follow-up, 81% of participants had partially enacted their CTCs. Frequently cited barriers to implementation were insufficient time, logistical barriers, and forgetting. Only 4% of faculty reported insufficient skills.

Representative comments from CTCs include:

“When with a fellow who struggles with organizing their thoughts, [I will] focus on using (and teaching) the concept of problem representation as a means to systematically think through a case and create a differential.”

“In pivotal moments in case presentations, I will take a diagnostic time-out to discuss what fits/doesn’t fit.”

Discussion

Similar to national reports, our faculty lacked a framework for teaching CR [7], [8], [9], [10], [11], [12]. After participating in this 3-h workshop, faculty demonstrated increased ability to use problem representation and illness scripts in their CR teaching in the post-GOSTE. Faculty also expressed greater confidence and an increased likelihood to teach CR using a framework. Through CTCs, participants created specific plans for incorporating CR strategies into their educational settings and at 2-month follow-up the majority had partially implemented their CTCs.

Although educational strategies to teach CR are widely reported in the literature, few reports describe successful interventions to train faculty to teach using these methods [2], [10], [12]. Our study adds to the findings of Dhaliwal (2013) who reported on a similarly structured workshop which increased faculty’s understanding of CR teaching methods and their intent to apply those methods in practice [10]. More recently, Addy et al. (2016) published their experiences with training faculty to teach CR using a modified Bayesian approach. They observed five faculty and evaluated their implementation of CR skills by reviewing video recordings of teaching sessions. They reported variable degrees of implementation and similar barriers to what we found including lack of time, forgetting to incorporate, and insufficient skills [12].

One of our study limitations is that we have not directly observed whether faculty are implementing skills learned in this workshop in the real-world clinical environment. This is a challenge shared by many FDPs where learning occurs in workshops [2], [10], [12]. Other limitations of this study include a small sample size, a self-selected group of faculty, and the use of self-reported data. Future evaluations of this program aim to include learner evaluations and direct observations of faculty workplace teaching skills. We plan to reevaluate the workshop structure and content for the next iteration to bring about improvement in all skills taught, including broadening the differential diagnosis, strategies to prioritize the differential diagnosis, and use of diagnostic time outs.

There is debate on how to best teach CR. Issues of context specificity generate questions such as: can we teach learners how to reason generally, or is reasoning best learned by repeated exposure to cases, thus building illness scripts? [8], [9]. Providing learners with an array of strategies to approach cases, including guided reflection and knowledge reorganization, can help learners navigate the diagnostic process [2], [3]. Training faculty to teach CR with these methods can reinforce learner efforts and build a shared language to give learners feedback on their CR.

In conclusion, a faculty development workshop on teaching CR increased faculty performance, confidence, and likelihood of teaching CR. Participants demonstrated improved use of problem representation and illness scripts as observed in the post-GOSTE. Participants also reported implementing changes in their CR teaching in their educational environments. A theory-based CR workshop can be an effective strategy to build a cadre of faculty empowered with the skills and the confidence to teach the next generation of physicians this core skill.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Conflicts of interest: None of the authors on this paper have conflicts of interest to disclose.

Research funding: This faculty development program was funded by the generous donation of an anonymous donor.

Competing interests: The funding Organization(s) played no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References

  • 1.

    National Academies of Sciences, Engineering, and Medicine. Improving diagnosis in health care. Washington, DC: National Academies Press, 2016.

  • 2.

    Lambe KA, O’Reilly G, Kelly BD, Curristan S. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual Saf 2016;25:808–20.

    • Crossref
    • PubMed
    • Export Citation
  • 3.

    Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S. The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking. Acad Med 2017;92:23–30.

    • Crossref
    • PubMed
    • Export Citation
  • 4.

    Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med 2006;355:2217–25.

    • Crossref
    • PubMed
    • Export Citation
  • 5.

    Kahneman D, Egan P. Thinking, fast and slow. New York: Farrar, Straus and Giroux, 2011.

  • 6.

    Atkinson K, Ajjawi R, Cooling N. Promoting clinical reasoning in general practice trainees: role of the clinical teacher. Clin Teach 2011;8:176–80.

    • Crossref
    • PubMed
    • Export Citation
  • 7.

    Rencic J, Trowbridge RL, Fagan M, Szauter K, Durning S. Clinical reasoning education at US medical schools: results from a national survey of internal medicine clerkship directors. J Gen Intern Med 2017;32:1242–6.

    • Crossref
    • PubMed
    • Export Citation
  • 8.

    Eva KW. What every teacher needs to know about clinical reasoning. Med Educ 2005;39:98–106.

    • Crossref
    • PubMed
    • Export Citation
  • 9.

    Trowbridge RL, Olson AP. Becoming a teacher of clinical reasoning. Diagnosis (Berl) 2018;5:11–4.

    • Crossref
    • PubMed
    • Export Citation
  • 10.

    Dhaliwal G. Developing teachers of clinical reasoning. Clin Teach 2013;10:313–7.

  • 11.

    Audetat MC, Dory V, Nendaz M, Vanpee D, Pestiaux D, Junod Perron N, et al. What is so difficult about managing clinical reasoning difficulties? Med Educ 2012;46:216–27.

    • Crossref
    • PubMed
    • Export Citation
  • 12.

    Addy TM, Hafler J, Galerneau F. Faculty development for fostering clinical reasoning skills in early medical students using a modified bayesian approach. Teach Learn Med 2016;28:415–23.

    • Crossref
    • PubMed
    • Export Citation
  • 13.

    Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents’ clinical competence: a randomized trial. Ann Intern Med 2004;140:874–81.

    • Crossref
    • PubMed
    • Export Citation
  • 14.

    Ludwig AB, Raff AC, Lin J, Schoenbaum E. Group observed structured encounter (GOSCE) for third-year medical students improves self-assessment of clinical communication. Med Teach 2017;39:931–5.

    • PubMed
    • Export Citation
  • 15.

    Trowbridge RL. Twelve tips for teaching avoidance of diagnostic errors. Med Teach 2008;30:496–500.

    • Crossref
    • PubMed
    • Export Citation
  • 16.

    Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med 2002;9:1184–204.

    • Crossref
    • PubMed
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • 1.

    National Academies of Sciences, Engineering, and Medicine. Improving diagnosis in health care. Washington, DC: National Academies Press, 2016.

  • 2.

    Lambe KA, O’Reilly G, Kelly BD, Curristan S. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual Saf 2016;25:808–20.

    • Crossref
    • PubMed
    • Export Citation
  • 3.

    Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S. The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking. Acad Med 2017;92:23–30.

    • Crossref
    • PubMed
    • Export Citation
  • 4.

    Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med 2006;355:2217–25.

    • Crossref
    • PubMed
    • Export Citation
  • 5.

    Kahneman D, Egan P. Thinking, fast and slow. New York: Farrar, Straus and Giroux, 2011.

  • 6.

    Atkinson K, Ajjawi R, Cooling N. Promoting clinical reasoning in general practice trainees: role of the clinical teacher. Clin Teach 2011;8:176–80.

    • Crossref
    • PubMed
    • Export Citation
  • 7.

    Rencic J, Trowbridge RL, Fagan M, Szauter K, Durning S. Clinical reasoning education at US medical schools: results from a national survey of internal medicine clerkship directors. J Gen Intern Med 2017;32:1242–6.

    • Crossref
    • PubMed
    • Export Citation
  • 8.

    Eva KW. What every teacher needs to know about clinical reasoning. Med Educ 2005;39:98–106.

    • Crossref
    • PubMed
    • Export Citation
  • 9.

    Trowbridge RL, Olson AP. Becoming a teacher of clinical reasoning. Diagnosis (Berl) 2018;5:11–4.

    • Crossref
    • PubMed
    • Export Citation
  • 10.

    Dhaliwal G. Developing teachers of clinical reasoning. Clin Teach 2013;10:313–7.

  • 11.

    Audetat MC, Dory V, Nendaz M, Vanpee D, Pestiaux D, Junod Perron N, et al. What is so difficult about managing clinical reasoning difficulties? Med Educ 2012;46:216–27.

    • Crossref
    • PubMed
    • Export Citation
  • 12.

    Addy TM, Hafler J, Galerneau F. Faculty development for fostering clinical reasoning skills in early medical students using a modified bayesian approach. Teach Learn Med 2016;28:415–23.

    • Crossref
    • PubMed
    • Export Citation
  • 13.

    Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents’ clinical competence: a randomized trial. Ann Intern Med 2004;140:874–81.

    • Crossref
    • PubMed
    • Export Citation
  • 14.

    Ludwig AB, Raff AC, Lin J, Schoenbaum E. Group observed structured encounter (GOSCE) for third-year medical students improves self-assessment of clinical communication. Med Teach 2017;39:931–5.

    • PubMed
    • Export Citation
  • 15.

    Trowbridge RL. Twelve tips for teaching avoidance of diagnostic errors. Med Teach 2008;30:496–500.

    • Crossref
    • PubMed
    • Export Citation
  • 16.

    Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med 2002;9:1184–204.

    • Crossref
    • PubMed
    • Export Citation
FREE ACCESS

Journal + Issues

Diagnosis aims at answering the question how diagnosis determines the quality of medical care. It focuses on how diagnosis can be advanced, how it is taught, and how and why it can fail, leading to diagnostic errors. The journal welcomes both fundamental and applied works, improvement initiatives, opinions, and debates to encourage new thinking on improving this critical aspect of healthcare quality.

Search

  • View in gallery

    Retrospective pre-post assessment results.

    *PR, problem representation; IS, illness script.