Diagnostic error, including delayed, wrong or missed diagnoses, has become a central focus within the patient safety movement , . Although estimates vary, studies suggest that a diagnosis is wrong between 10 and 15% of the time , , and that every human, at least once in their life, will be exposed to a diagnostic error . Thus, an urgent and yet unmet need to improve diagnostic reasoning, accuracy and efficiency exists.
Current research has attempted to explain diagnostic errors in two domains: cognitive or system-based causes. Graber et al. published a useful taxonomy for such errors, separating cognitive errors into those stemming from faulty knowledge, faulty data gathering, faulty information processing and faulty information verification and systems errors into technical and organizational . Examples of cognitive errors include anchoring bias (where health care providers “lock on” to a diagnosis made early in the diagnostic process despite contrary evidence), or availability bias (where the likelihood of the diagnosis relates to how readily it comes to mind) . Conversely, lack of standardized policy, discontinuous care and interruptions are often cited as system-based problems that contribute to errors in diagnosis , , , . While a convenient classification scheme, these two domains never occur in isolation; in fact, one retrospective study of 93 diagnostic errors identified nearly six system- or cognitive-based factors per case, with almost half stemming from both domains .
While the contextual contributions to cognitive error, as well as tools to identify and organize systems and cognitive contributions to error, have been previously studied , , efforts to understand diagnostic errors and interventions to curb them have often treated systems and cognitive errors individually , . The drawbacks of this segregated approach (from a human factors perspective) have been recently highlighted in the literature . Despite this, there continues to be a paucity of data exploring the interrelatedness of these domains – or, more specifically, how system factors may contribute to cognitive missteps in diagnosis. This gap is important as some systems-based issues might be more amenable to change, thus preventing cognitive errors downstream. Therefore, we aimed to evaluate systems-based contributions to cognitive error so as to identify high-yield interventions to curb diagnostic errors in patient care.
Subjects and methods
We designed a focused ethnographic study to examine factors contributing to diagnostic decision-making in the inpatient setting. Based on qualitative methods, a focused ethnography is the practice of studying a group of people who have a “specific knowledge about an identified problem”, with the goal of providing useful application of the findings to the setting under study .
Settings and participants
We observed members of inpatient internal medicine teams at two affiliated academic medical centers. Teaching teams consisted of an attending physician, one senior resident, two interns and medical students. Teams were selected at random and were followed longitudinally on a weekly basis so that work routines on both call days (i.e. admitting new patients to the hospital) and non-call days (i.e. rounding without admitting) could be observed.
Observations were conducted by a multi-disciplinary team including clinicians (physicians, nurses) and non-clinicians (qualitative researchers, social scientists and health care engineers). Typically, one clinician and one non-clinician were paired to observe each teaching team. Observers were oriented to the field of diagnosis and diagnosis errors through team meetings prior to observations. Team members were observed during pre-round work, rounds and after rounds in activities such as order entry, documentation, communication and processes such as admitting and discharging patients. Explanations for observed behaviors were elicited and documented. Observational data were collected using both open field notes as well as templated field notes modeled on components of the National Academy of Science model for diagnosis, which categorizes phases of the diagnostic process (e.g. data gathering and integration, formulation of a working diagnosis and treatment delivery and outcomes) and components of the work system , , .
Focus groups and interviews
To understand participant rationales for diagnostic decisions and factors potentially contributing to errors, qualitative scientists (MH, MQ) and the study principal investigator (PI) (VC) conducted focus groups and interviews with interns, resident and attending physicians after team observations. This allowed us to elicit thoughts and opinions on what we observed regarding systems factors related to diagnosis. Attending physicians were interviewed separately from resident and interns to eliminate any power differentials during discussions. Within focus groups, residents and interns were asked about barriers to diagnosis, systems-based contributions to diagnosis and challenges in making diagnoses. Attending physicians were also asked questions related to oversight, recognition and management of errors made by the team. All focus groups and interviews were audio recorded and transcribed verbatim for analysis.
Field notes, focus group and interview transcripts were aggregated and analyzed using inductive content analysis by a qualitative specialist (MH) and verified by the study PI (VC) . Inductive codes were derived from reading a sample of field notes and transcripts, which were then systematically applied to the remaining data. All data were coded using NVivo v10 (QSR International, Melbourne, Australia) , after which code reports were generated to check for coding consistency and identify themes-related systems and cognitive contributions to error. Once these themes were identified, we returned to the literature and identified Graber’s taxonomy of diagnostic error  as conceptually useful way of organizing the data, an approach that has been utilized previously in qualitative literature , . Consistent with this taxonomy, codes were organized thematically via internal discussion amongst investigators into those representing systems-based contributions to: (a) faulty data gathering (e.g. ineffective or incomplete history and physical, workup, procedures or techniques), (b) faulty information processing (e.g. failed context generation, overestimating/underestimating usefulness of salient findings, failed heuristics) and (c) faulty information verification (e.g. premature closure). Because additional pertinent codes remained, a separate category, (d) systems-based contributions affecting multiple cognitive processes, was created.
Ethical and regulatory oversight
This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).
Between January 2016 and April 2017, a total of 10 teaching teams were observed during and after rounds for a total duration of 286 h. A total of 31 residents and interns participated in focus groups and 10 attending physicians participated in interviews.
System-based contributions to faulty data gathering
A key cognitive problem was the inability to obtain accurate information in a timely fashion to make clinical decisions. Trainees and attending physicians most often highlighted system challenges related to interdisciplinary communication and the electronic medical record (EMR) as key drivers of this problem. For example, despite recognizing the importance of interdisciplinary care, team members reported limited nursing involvement during morning rounds. We observed that nursing had competing priorities during medical rounds (such as giving hand-off to oncoming nurses or passing medications); yet, the knowledge of real-time information by nursing staff was felt paramount for diagnosis (e.g. overnight response to treatment, family input, concerns regarding behavior or sleep). Furthermore, the inability to communicate with nurses in-person was exacerbated by barriers to information gathering. For example, one resident noted that paging character limits often led to incomplete messages. Others cited lack of nurse phones as straining communication, as calls were redirected to a unit clerk who then had to locate the nurse to relay the message.
…even something as simple as, “has the patient peed today?”. It takes 30 minutes to hear back an answer because I can’t actually talk to a nurse. (Resident; Focus group)
Challenges with communication were also felt to arise from reliance on the EMR. For example, lack of updated vital signs or laboratory tests impaired diagnosis on rounds. Similarly, inability to provide context to orders via the EMR was felt to adversely affect the diagnostic process.
…a lot of times we put the orders in the system…and the assumption is, oh, they’ll look and see it in the system and so they’ll do it. And at times there’s context that needs to be taken into account, like when should I get this? How quickly should I do this? And we don’t always have time to be paging and make those kinds of communications as quickly as we may need to… (Resident; Focus group)
System-based contributions to faulty information processing
An important theme that linked system-based factor to impaired cognitive processes was problems in functionality within the EMR. In fact, we observed how even physicians highly trained in the EMR frequently sought help from others for some tasks (e.g. locating appropriate imaging studies, data from prior admissions and certain orders). “Hard stops” in the EMR (i.e. screens that had to be addressed before further activities could be performed), were similarly felt to create interruptions in diagnosis and treatment. Furthermore, navigation of the EMR was felt to be taxing. For example, we observed that residents often spent upwards of 30 min combing through charts in search of historical data, imaging, lab results and documentation prior to admitting patients.
Organizational structures of other services were also viewed as limiting data synthesis and leading to delays or inaccurate diagnoses. A commonly cited example was the discrepancy observed between preliminary resident radiology reads and final reports. Final reports were entered several hours after the preliminary read, sometimes after management and treatment decisions were made. Knowing that this delay was possible, team members often deferred care decisions or had to rely on potentially incomplete or inaccurate information. Team members also cited organizational structures of consulting teams as confounding diagnosis, as consultants failed to recognize organ systems outside of their domain and rarely communicated with each another to reach mutual conclusions.
[Patient] had six different teams consulted and all the teams were saying this is a bone marrow problem except Hematology was saying this is not a bone marrow problem. So, it then becomes just everyone fighting against each other and the primary team is just stuck between a rock and a hard place. (Intern; Focus group)
System-based contributions to faulty data verification
Handoffs of care between day and night teams, including newly admitted overnight patients, frequently arose as a system-based theme fueling cognitive errors. Similarly, transitions at the end of the month for interns and residents and every 2 weeks for attendings facilitated anchoring bias and premature closure (i.e. the failure to consider reasonable alternatives following the initial diagnosis). We observed these biases regularly, as teams accepted emergency room findings, consultant conclusions or pieces of data without questioning assumptions.
…we are constantly signing out to one another and somebody else is getting the history and the emergency department is getting the history and they call them up and say, oh, this is what’s happening. So it’s easy like to hook onto what they said and just like kind of take it like at face value and not investigate it further. (Resident; Focus group)
….in July, I picked up service. The patient’s hypercalcemic. The reasoning behind the hypercalcemia was high PTH…. It was a guy getting treated for pneumonia and treated for pneumonia on several occasions. The intern sort of said, “hey, his calcium level is up on the first day. It was like 11.5. That’s kind of weird….” The first couple of days they just gave him fluids, thinking, you know, he was going to get better but not understanding why. And then, probably about day three, we actually went back….and found he had a baseball in his lung. And I said, “That’s not pneumonia; that’s lung cancer.” (Attending; Interview)
System-based contributions affecting multiple cognitive processes
Several system-based factors were noted to affect multiple cognitive domains. These included interruptions, time constraints and the physical environment.
One of the most cited system barriers to accurate diagnosis was frequent interruptions from pagers. On rounds, residents and interns were typically paged away at least once per morning, often having to play “catch up” upon returning. Teams described pages as “incessant” and “distracting” and stated that the high volume of pages led to cognitive dissociations between tasks. One resident recounted being paged 120 times within a 24-h shift, dramatically affecting his ability to think critically about patients.
Imagine you are having these sorts of educated discussions about what you want to do for your patient, thinking about mechanisms and lab testing and diagnostics, and right in the middle of it, you have to keep closing out the chart of that patient to go order somebody else’s bowel regimen or to follow up on the family wanting to know about why the patient is on Vitamin D supplements. (Intern; Focus group)
Nearly all team members reported a lack of time to think critically about diagnosis. This was particularly notable on call days, when senior residents were trying to admit patients amidst responsibilities such as managing patients on the floor and discharging existing patients. On top of this, residents were expected to attend educational conferences, leading to being rushed and fueling errors in decisions.
I would consolidate my thoughts and my thinking at the end of the day when I finish writing my note and I’ve actually written everything down and then like I said, 20 percent, I find 20 percent new problems. And now it’s 7 or 8 PM. It’s too late to deal with these. If I had actually taken – if I had actually pushed that process up a few hours… and found maybe these few problems, I could have a consult that day instead of the next day. I could have gotten that test done today as opposed to tomorrow morning and something could have happened overnight. (Intern; Focus group)
Many team members described the physical space as “suboptimal” for diagnosis. For example, team rooms were noted to be cluttered and disorganized. A lack of dedicated space for every team member was commonly observed, leading to an insufficient number of computers. Team rooms were noisy and shared amongst multiple medicine teams and nursing staff. In fact, some resorted to donning headphones to drown out ambient noise and allow them to think clearly.
…we share our team room with another [team] and they’re doing their kind of table rounds and we’re trying to do our table rounds and it can get loud and cacophonous and I lose my train of thought… (Intern; Focus group)
In this focused ethnography, we identified and categorized how systems-based factors contribute to cognitive error. First, trainees and attending physicians noted that challenges with interdisciplinary communication, including in-person communication and communication within the EMR contributed to faulty data gathering. Second, inefficiencies within the EMR (such as “hard stops”), as well as operations within the hospital (such as timing and siloed role of consulting services), contributed to faulty data processing. Third, frequent handoffs between outgoing and incoming physicians provided opportunities for premature closure and anchoring, resulting in faulty data verification. Finally, constant interruptions, time constraints and a cluttered physical environment impacted multiple cognitive processes.
Interventions aimed at curbing cognitive contributions to diagnostic error include those aimed at increasing knowledge and improving clinical reasoning . Such efforts have focused on Kahneman and Tversky’s System 1 (intuitive) and System 2 (effortful) domains of reasoning . System 1-based interventions include aspects such as cognitive de-biasing, metacognition and use of checklists to encourage stopping and thinking. In contrast, system 2- based interventions emphasize principles of clinical reasoning, evidence-based medicine and knowledge regarding specific disease conditions . Importantly, studies testing these principles to prevent errors are often devised and tested in non-clinical settings, where factors such as interruptions, time constraints, physical environment and the quality and accessibility of necessary information are tightly controlled . Similarly, they assume a puritan view when it comes to cognitive processes and system-based factors. In accordance with Henriksen et al.’s human factors perspective, our findings suggest that separating the two – as has been done in many studies to date – may not be wise . Rather, targeting both system and cognitive factors concurrently may provide greater opportunities to implement sustainable changes.
Our ‘real-world’ study, performed on the wards and outside the confines of a simulated environment, suggests that interventions targeting systems factors and cognitive factors simultaneously should be prioritized. One example of such an approach are cancer tumor boards, implementation of which has resulted in marked change in diagnoses, evaluation and treatment strategies in a variety of malignancies , , . Despite necessitating upfront time from physicians across specialties, these boards are successful in part due to the time-savings from care coordination and improvements in interdisciplinary communication, allowing for improved data gathering and processing. In similar fashion, “diagnosis boards” – aimed at improving diagnosis by conglomerating multiple specialties – may yield benefit by minimizing information silos and maximizing communication – a classic system and cognitive trap observed in this study.
Targeted interventions aimed at curbing the number of interruptions may also be important in preventing diagnostic errors . For instance, separating admitting from rounding teams may minimize interruptions and improve rounding efficiency by uncoupling pages on existing patients from new admits, allowing more time to synthesize and interpret data. Similarly, rethinking timing and delivery of didactic medical education (e.g. consolidation during one afternoon rather than every day) may help alleviate time constraints that prevent critical thinking. Finally, “intersectional innovations”  that include rethinking the design of physical space for diagnoses (e.g. ergonomically designed workrooms with more table and computer space) and collaboration with information technology professionals to improve EMR functionality would be welcomed. These examples are but a few of how system and cognitive processes might be linked to help improve diagnostic decisions.
Our study has limitations. First, the study was performed at two academic centers, which may limit the generalizability of our findings. Second, our study did not specifically identify cases of diagnostic error; rather, we report on observations and views of trainees and attending physicians on contributors to these events. Third, teams may have modified aspects of their behavior in the setting of being observed, leading to a potential Hawthorne effect. This effect may be limited, however, because people do not act in ways that deviate from habits for long . Fourth, although we specifically examined for overlap between systems and cognitive factors, there may be some aspects that we could not observe. For example, the association between inadequate knowledge and systems-based factors and reasons underlying these relationships could not be explored using ethnographic techniques. Additionally, we acknowledge that lack of key data in the EMR is a system-based problem that may limit cognitive processing; however, it was not possible for us to evaluate this through our observational approach. These limitations were necessary given the broad scope of this work. Finally, while we aggregated data from observations and followed a methodologically rigorous approach to link system and cognitive processes, our findings should be viewed as hypothesis-generating. Future studies are needed to assess the veracity of our conclusions.
Our study also has important strengths. First, data were collected over 280 h, spanning 14 months, with physicians of varying backgrounds and experiences – lending a high degree of internal validity. Second, data collected in real time through observations were triangulated with focus groups and interviews, leading to a better understanding of workflow and the diagnostic process. Third, our team was highly multidisciplinary and included clinicians and experts trained in ethnographic methods. Finally, we utilized a published framework to organize our findings, allowing for identification of interventions using these data. Aligning systems with cognitive-based processes is novel and offers new insights into tackling diagnostic errors.
Efforts to address diagnostic error have yielded limited results to date. Recognizing systems and cognitive factors as overlapping processes and designing interventions aimed at concurrently addressing both domains may provide physicians with the time and tools necessary to improve diagnosis.
The authors would like to acknowledge Jane Forman, ScD, MHS for her assistance in qualitative method development and review.
Singh H, Graber ML. Improving diagnosis in health care – the next imperative for patient safety. N Engl J Med 2015;373:2493–5. Google Scholar
Improving Diagnosis in Healthcare. National Academy Press. Available at: http://www.nap.edu/21794. Accessed: 18 Sep 2017.
Elstein AS. Clinical reasoning in medicine. Woburn, MA: Butterworth-Heinemann, 1995. Google Scholar
Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013;22(Suppl 2):ii21–7. Google Scholar
Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med 2005;165:1493–9. Google Scholar
Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med 2003;78:775–80. Google Scholar
Singh H, Petersen LA, Thomas EJ. Understanding diagnostic errors in medicine: a lesson from aviation. Qual Saf Health Care 2006;15:159–64. Google Scholar
Gandhi TK. Fumbled handoffs: one dropped ball after another. Ann Intern Med 2005;142:352–8. Google Scholar
Petersen LA, Brennan TA, O’Neil AC, Cook EF, Lee TH. Does housestaff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med 1994;121:866–72. Google Scholar
Ogdie AR, Reilly JB, Pang WG, Keddem S, Barg FK, Von Feldt JM, et al. Seen through their eyes: residents’ reflections on the cognitive and contextual components of diagnostic errors in medicine. Acad Med 2012;87:1361–7. Google Scholar
Reilly JB, Myers JS, Salvador D, Trowbridge RL. Use of a novel, modified fishbone diagram to analyze diagnostic errors. Diagnosis (Berl) 2014;1:167–71. Google Scholar
Graber ML, Kissam S, Payne VL, Meyer AN, Sorensen A, Lenfestey N, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf 2012;21:535–57. Google Scholar
Singh H, Graber ML, Kissam SM, Sorensen AV, Lenfestey NF, Tant EM, et al. System-related interentions to reduce diagnostic errors: a narrative review. BMJ Qual Saf 2012;21:160–70. Google Scholar
Henriksen K, Brady J. The pursuit of better diagnostic performance: a human factors perspective. BMJ Qual Saf 2013;22(Suppl 2):ii1–5. Google Scholar
Higginbottom GM, Pillay PJ, Broadu NY. Guidance on performing focused ethnographies with an emphasis on healthcare research. The Qualitative Report 2013;18:1–6. Google Scholar
Mulhall A. In the field: notes on observation in qualitative research. J Adv Nurs 2003;41:306–13. Google Scholar
Chopra V, Harrod M, Winter S, Forman J, Quinn M, Krein SL, et al. Focused ethnography of diagnosis in academic medical centers. J Hosp Med 2018. doi: 10.12788/jhm.2966. [Epub ahead of print]. Google Scholar
Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs 2008;62:107–15. Google Scholar
Bazeley P, Jackson K, editors. Qualitative data analysis with NVivo. London: Sage Publications Limited, 2013. Google Scholar
Harrod M, Kowalski CP, Saint S, Forman J, Krein SL. Variations in risk perceptions: a qualitative study of why unnecessary urinary catheter use continues to be problematic. BMC Health Serv Res 2013;13:151. Google Scholar
Sandelowski M. Theory unmasked: the uses and guises of theory in qualitative research. Res Nurs Health 1993; 16:213–8. Google Scholar
Croskerry P. A universal model of diagnostic reasoning. Acad Med 2009;84:1022–8. Google Scholar
Lambe KA, O’Reilly G, Kelly BD, Curristan S. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual Saf 2016;25:808–20. Google Scholar
Wheless SA, McKinney KA, Zanation AM. A prospective study of the clinical impact of a multidisciplinary head and neck tumor board. Otolaryngol Head Neck Surg 2010;143:650–4. Google Scholar
Petty JK, Vetto JT. Beyond doughnuts: tumor board recommendations influence patient care. J Cancer Educ 2002;17:97–100. Google Scholar
van Hagen P, Spaander MC, van der Gaast A, van Rij CM, Tilanus HW, van Lanschot JJ, et al. Impact of a multidisciplinary tumour board meeting for upper-GI malignancies on clinical decision making: a prospective cohort study. Int J Clin Oncol 2013;18:214–9. Google Scholar
Westbrook JI, Raban M, Walter SR, Douglas H. Task errors by emergency physicians are associated with interruptions, multitasking, fatigue and working memory capacity: a prospective, direct observation study. BMJ Qual Saf 2018. doi: 10.1136/bmjqs-2017-007333. [Epub ahead of print]. Google Scholar
Johansson F. The medici effect: breakthrough insights at the intersection of ideas, concepts, and cultures. Boston: Harvard Business School Press, 2004. Google Scholar
Iedema R. Visualizing health care practice improvement: innovation from within. London, UK: Radcliffe Publishing Ltd., 2013. Google Scholar
About the article
Published Online: 2018-07-14
Published in Print: 2018-09-25
Funding Source: Agency for Healthcare Research and Quality
Award identifier / Grant number: P30HS024385
U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. Funder Id: 10.13039/100000133, Grant Number: P30HS024385. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Singh is supported by the VA Health Services Research and Development Service (Presidential Early Career Award for Scientists and Engineers USA 14-274), the VA National Center for Patient Safety, the Agency for Health Care Research and Quality (R01HS022087), the Gordon and Betty Moore Foundation and the Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13–413).
Author contributions: All authors contributed to study design and manuscript preparation and have accepted responsibility for the entire content of this submitted manuscript. All authors have approved submission.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.