Differential diagnosis: the key to reducing diagnosis error, measuring diagnosis and a mechanism to reduce healthcare costs

William Osler is credited with introducing the discipline of differential diagnosis over a hundred years ago. In essence, this is simply a trigger and structure for a methodical way of thinking.

Clinicians will debate how much of medicine is art and how much is science but interestingly “Osler’s medical art was informed and controlled by all the assistance science could give. He did not discuss diagnosis as a matter of intuition. Every scientific aid was welcome” according to the author of Osler’s biography: ‘A Life in Medicine’ [1].

There is now a good body of evidence to show the quantum of diagnostic error and why it happens. Time after time, research papers conclude that the clinicians who committed the error should have broadened their differential diagnosis or compiled and documented one in the first place [2, 3].

Why, despite this vast body of evidence in favour of differential diagnosis, is it still not obligatory? In order for diagnosis to become a mainstream quality and patient safety issue, it is vital that it can be measured and, therefore, included as a quality metric that would become part of an institution’s overall quality score. But, until all healthcare institutions have sophisticated EMR systems and recognised standards for the speed of diagnosis, appropriateness of test ordering, referrals and admissions for all diseases, measurement will not be practical. In reality, the routine measuring of diagnostic accuracy is very unlikely to happen for many years.

In spite of the many patient safety and quality of care initiatives over the last 14 years (dating back to the landmark Institute of Medicine report ‘To Err is Human’), the performance of the crucial process of diagnosis and its related functions has been relegated to the ‘too difficult to fix’ bucket primarily due to the difficulties in measurement.

With research acknowledging that the greatest predictor of diagnostic accuracy is a differential diagnosis that includes what turns out to be the correct diagnosis [4], should we not be insisting that a differential diagnosis is worked up for every patient and documented in the medical notes? With modern diagnosis decision support tools now able to help the clinician build a differential diagnosis in seconds or minutes, this would be easy, cheap and practical to implement.

The lack of a simple measure for diagnosis will always be there as an excuse for healthcare not to address this important problem. We should, therefore, be introducing a practical proxy now rather than waiting several years before the technology is in place to introduce more accurate measures.

The obligation to compile a differential could act as a vital trigger to stimulate thinking at the time of the consultation.

The disappearance of the routine use of the differential diagnosis, the increasing degree of specialisation and availability of lab tests and imaging coupled with the fee for service payment model has conspired to reduce the time and incentive to think. Doctors have been seduced by the easy availability of increasingly sophisticated tests and the illusion of precision afforded by the plethora of data. However, in spite of all the sophisticated tests available the reality is that diagnostic performance has not improved significantly and healthcare costs have risen inexorably. A recent study in JAMA looking at diagnostic accuracy showed barely any improvement contributed by lab tests and imaging after the history and physical stage [5].

The reality today is that family physicians order tests in 29% of all patient visits, ED physicians for 41% of all visits and general internists for 38% of all visits. Since only 34% of visits to primary care are for a new complaint this means that, in reality, tests are probably ordered for almost every new visit [6]. Anecdotal stories of an astonishing 30%–50% of referrals being inappropriate and hospitalists ordering 10 consults for patients who have been in the hospital for 2 days without a diagnosis show that the current practice is wasting clinicians’ time and patients’ time and money while also running the risk of delays in diagnosis. This is untenable and has to change. Tests and investigations have now become a substitute for thinking. Thinking takes time so when time is short it’s much easier and quicker to order tests and consults.

The problem clinician leaders face now is that, just as the Affordable Care Act changes the way healthcare is paid for from by volume to value or simply for looking after people and doing the right thing, a generation of doctors has grown up accustomed to doing things rather than thinking.

The requirement for a differential diagnosis would act as a trigger to start and structure thinking. It would also make doctors’ lives easier: the plethora of data from lab tests currently means that doctors have to make sense of perhaps 10 data points rather than the 2–3 at the outset based just on the clinical features. The differential diagnosis serves to filter data meaning that tests or specialist consults are only ordered to confirm a high probability diagnosis in the differential. As it engenders greater objectivity, it also serves to help refute competing hypotheses.

It remains a truism that if you listen to your patient they will tell you the diagnosis. The old adage is that 75%–80% of the diagnosis is revealed by the patient’s story and should then be confirmed by judiciously ordered tests. This adage was proven in an ED based study which found that more than 80% of newly admitted internal medicine cases could be correctly diagnosed on admission based on history and physical alone [7]. The patient is the expert on his own symptoms and, in the model described above, should be considered an active participant in helping to compile the core differential diagnosis. The patient should be encouraged to research his own diagnosis by using a symptom checker to help him articulate clearly what his most bothersome symptoms are and to contribute to a discussion on the differential diagnosis rather than remaining a passive part of the diagnostic process.

Rarely has such a complex and expensive problem had a solution that is inexpensive, simple and, curiously, has already been part of basic training for over 100 years. Differential diagnosis is a trigger for disciplined and methodical thinking that has been shown to be the most accurate indicator of diagnostic accuracy. It is a viable, cost-effective and practical proxy for the measurement of diagnosis which could make it a workable quality score. Lastly it could be a tool to filter and streamline the ordering of consults and lab tests, contributing to a significant reduction in costs and time.

The routine use of the differential diagnosis which has gradually faded over the last 50 years due to lack of clinician time and, frankly clinicians’ inability to quickly and accurately recall and synthesise vast amounts of data is now made possible by the new generation of highly sophisticated diagnosis decision support tools that can even work automatically in the background suggesting possible diagnoses when needed. With an estimated 80,000–160,000 patients in US hospitals alone (2–4 million globally) suffering death or disability each year from potentially preventable diagnostic error [8] and healthcare costs rising inexorably, how much longer can the industry stand by and ignore the problem, especially when a simple solution has been in their midst for so long?