The Toyota Production System (TPS), also called Lean, has been a useful approach for improving work process flow and reducing waste in health care , , , . The Virginia Mason Medical Center has applied TPS to reduce clinic wait times , increase nurse-patient contact time , and shorten sepsis recognition time from 8 h to 30 min . Following the adoption of the TPS, Denver Health experienced a decrease in observed to expected patient annual mortality from 0.75 to 0.5 in 2010, achieving the lowest mortality rate among US academic health systems in 2011 .
The TPS could serve as a helpful guide for reducing diagnostic waste, improving diagnostic accuracy, and improving the efficiency and effectiveness of disease management. We designed a 2-day curriculum to teach students the principles and tools of the TPS. Our goals were to encourage students to improve their diagnostic and management skills and appreciate the application of manufacturing principles to health care.
This 2-day module was designed for 3rd and 4th year students. The application of the TPS to improve patient diagnosis and management was first introduced through seven videos (most under 10 min). The course started with four videos and in early 2017 three additional videos focusing on diagnostic waste were added to the curriculum. The videos were created by the senior author (FS) who was trained in Advanced Lean at the Virginia Mason Institute. The first video describes a previously published case of vasculitis complicated by nearly fatal respiratory, cardiac, and renal failure that illustrated the consequences of diagnostic error and treatment delay  (see Figure 1). The subsequent videos showed how value stream mapping [work flow diagrams showing the steps used to provide a product or service that the customer (patient) desires] and waste elimination (waste being defined as any activity that does not improve the health and wellbeing of patients) could have prevented these complications and how these tools could be applied to common clinical problems. The students were also required to read the first two chapters of a textbook that described integrated healthcare systems and how manufacturing principles can be applied to redesign healthcare . We then assessed each student’s basic understanding of these tools and principles using 15 multiple-choice questions that were developed by the instructor. Following 1.5 days of self-study devoted to the activities described, the students participated in a 2-h seminar taught by the senior author.
The overall goal was to help students understand how waste reduction and the creation of value stream maps could serve as a useful framework for improving diagnostic accuracy, reducing diagnostic waste, and creating efficient and standardized approaches for managing common diseases. Three learning objectives were related to the TPS principles:
Describe how value is defined in health care
Describe how diagnostic waste leads to overproduction, increases processing complexity, time delays, and diagnostic errors
Apply sequential and iterative value streams for patient management.
(four additional learning objectives of the course related to clinical reasoning concepts including illness scripts, prioritized differential diagnoses, Bayes’ theorem, and high value testing; they are not discussed in detail here).
Objective 1 – Describe how value is defined in health care
To explore the concept of value students are asked two guiding questions in the seminar: “What is the definition of value?” and “How do we decide if a test or procedure is of value to our patients?” With the assistance of the instructor, students learn to define value as more than quality/cost, instead defining value as any test or procedure that improves health and wellbeing, and that an informed patient would be willing to pay for. The students are then presented with a list of different tests and procedures and asked to decide which ones they would be willing to pay for (Supplementary A).
Objective 2 – Describe how diagnostic waste leads to overproduction, increases processing complexity, time delays, and diagnostic errors
Students were introduced to the eight forms of waste identified by the TPS in the videos and in a textbook (Figure 2A). In the seminar the instructor leads a discussion on the four forms of waste most closely related to diagnosis (Figure 2B):
Overproduction – the ordering of tests and procedures that are of low value.
Time – e.g. time generated by an imaging study that can delay diagnosis while increasing the cost of care.
Processing – extra tests increase the complexity of data to be processed, reduce reliability, and increase the risk of diagnostic errors.
Defects – an error that is passed along to the next step in a process or to the patient.
During the discussion students are oriented to errors as an act that through ignorance, deficiency, or accident fails to achieve the intended outcome . The distinction between an error of omission – e.g. forgetting a step and an error of commission – e.g. incorrectly performing the task or doing the wrong thing – are also discussed . After defining errors and defects the class explores how the interpretation of large numbers of unnecessary test findings, some of which could be falsely positive (potentially leading to errors of commission), wastes time that clinicians could be devoting to obtaining a more complete history and physical examination and to more thoroughly analyzing of high yield findings and acting on them (errors of omission) . Finally, the class discusses how diagnostic error can lead to inappropriate therapy and cause patient harm that will require additional processing and time to correct the defect.
Objective 3 – Appropriately apply sequential and iterative value streams for patient management
In manufacturing the resources required to create a product or service can usually be separated into two groups. Eighty percent of the product or service requires 20% of the resources, and the other 20% demands 80% of the resources. These two products or services are managed in very different ways. This 80/20 rule or Pareto principle has also been applied in health care to clinical diagnosis distribution, medication use, laboratory testing , prioritizing quality improvement projects , and nurse management decisions .
The students are asked how to create value streams for these two populations of patients (see Figure 3). Value stream is defined as a series of steps used to provide a product or service that the customer (patient) desires. A sequential value stream for the management of a patient with pyelonephritis (Figure 4A) is presented. A patient with a history of flank pain, fever, and dysuria (Step 1) is suspected of having pyelonephritis and, based on the Infectious Diseases Society of America (IDSA) guidelines for management of pyelonephritis , a urinalysis to document pyuria, and urine and blood cultures are ordered within 30 min (Step 2). If the patient is vomiting or the patient has symptoms and signs of sepsis she receives intravenous antibiotic treatment (ceftriaxone or cefepime) within 60 min (Step 3), and when the urine culture with antibiotic sensitivities is available and the patient is afebrile, usually within 3 days, she can be switched to an oral antibiotic (Step 4) and treated for an additional 7–10 days (Step 5).
The instructor emphasizes that value streams can be continually improved to reduce waste and improve efficiency. Next the class discusses what should happen if the patient fails to improve in the expected time frame. When students suggest the need for additional diagnostic tests to exclude possible complications, the concept of a shift from a sequential to an iterative value stream is highlighted. This helps students to understand that standardized sequential value streams with time expectations enable the novice clinician to experience the normal range of therapeutic response, and to consider a different diagnosis when the response is delayed beyond the normal range (see Figure 3A). Although this exercise starts with a management problem, it emphasizes the need to revisit and reconsider diagnoses when there is a variation in the value stream.
For the other 20% of patients whose diagnosis is unclear at the time of presentation, students are introduced to a diagnostic iterative value stream approach to diagnosis (see Figure 3B). After discussion of core clinical reasoning concepts (illness scripts, pattern recognition, pre- and post-test probability, tiered differential diagnosis) students are asked to apply the diagnostic iterative value stream approach to diagnose a 19-year-old man with the acute onset of pharyngitis who has enlarged tonsils, tender anterior cervical lymph nodes, and fever (Figure 4B).
The students first apply the modified Centor score to assess the pre-test probability of Group A streptococcal (GAS) pharyngitis  and determine the score is 4 yielding a pretest probability of 52%. This intermediate pretest probability of GAS warrants the ordering of a rapid streptococcal antigen test (sensitivity 83%, specificity 99%) which results in a negative test result (iterative value stream cycle 1). Applying Bayes’ theorem, the post-test probability of GAS is 16%. The clinician chooses to treat with amoxicillin while awaiting a throat culture (iterative value stream cycle 2). On the second day of treatment the patient develops a diffuse maculopapular rash. The antibiotic is discontinued and the following day the throat culture is reported to be negative for GAS. On repeat exam he is noted to have an enlarged spleen. After some discussion the students order a heterophile antibody (iterative value stream cycle 3) given the now higher pre-test probability (90%) of acute mononucleosis. The heterophile antibody is positive confirming the diagnosis of acute Epstein-Barr virus mononucleosis.
Students learn that testing for low and very low probability diseases results in a high percentage of false-positive tests and that testing for high pre-test probability diseases results in a significant percentage of false negative tests. They conclude that these tests have low value and should not be ordered. They also discover that testing for diseases of intermediate pre-test probability results in the greatest shifts in post-test probability and therefore are of higher value , , .
Upon completion of the course students filled out a course evaluation form. We calculated the mean and standard deviation for student test scores and student evaluations. Students also provided unstructured comments. The course evaluations covered the period of January 2015 to December 2017. The anonymous evaluation process was approved by the IRB as exempt, IRB201900079.
Thirty-two students have completed the elective. In 2015 seven students took the course on the Coursera platform (https://www.coursera.org/learn/fixing-healthcare) and therefore, their knowledge assessments were not available. The remaining 25 students completed 10 multiple-choice questions designed to assess their understanding of waste and sequential and iterative value streams (Table 1 and Supplementary B) and after three instruction videos on diagnostic waste were added to the curriculum, 16 of these students also completed five multiple-choice questions that tested their understanding of diagnostic waste, illness script, tiered differential diagnosis, and the application of Bayes’ theorem (see Table 1 and Supplementary B).
Course evaluations were completed by 31 out of 32 students with an average overall satisfaction of 4.94 out of 5 (n=31) (Table 2).
Representative unstructured comments documented students’ positive attitudes towards applying TPS to clinical diagnosis and management: “I really like the concept of iterative and sequential value streams; it really helps me in approaching my patients”. “Regarding diagnostic waste, I actually do plan to use these methods as a physician”. “One part of TPS is to not overproduce. This can be immediately implemented on the wards by not ordering unnecessary tests for patients”.
Our curriculum on diagnostic waste and diagnostic error introduces medical students to a novel conceptualization of the process of diagnosis and management. The visual representation of the cycle of diagnostic waste (Figure 2B) addresses a common misconception that the ordering of more tests will lead to more accurate diagnoses. The cycle of waste can help illustrate how over-testing potentially results in time delays, increased complexity of processing, increased risk of false positive tests, and heightens rather than lowers the risk of misdiagnosis and treatment delays.
Currently many physicians misunderstand and resist standardized approaches to the delivery of healthcare like the TPS, labeling such approaches “cook-book medicine” and expressing concern that the creativity and art of medicine will be lost , . However, these physicians may fail to differentiate sequential value streams or care pathways where diagnostic certainty is present from iterative care value streams where diagnostic uncertainty is dominant. It may be useful to introduce these concepts early in physicians’ careers before they become resistant to the application of effective manufacturing systems to health care or develop the habit of over-testing in response to diagnostic uncertainty.
The concept of value streams is likely to be a new for learners and teachers. We recommend first sharing examples of straightforward diagnoses (e.g. pyelonephritis and diabetic ketoacidosis) that lend themselves to sequential value streams and standardized management pathways. Other common diseases (community-acquired pneumonia, congestive heart failure, acute myocardial infarction) also can be taught using a sequential value stream concepts to provide the novice clinician additional examples of this linear management framework.
The concept of the iterative value stream and its application to undiagnosed patients is more challenging to describe and to apply. We recommend a case-based approach beginning with less complex cases similar to our example of the 19-year-old with acute pharyngitis. Other scenarios where iterative testing is often pursued are generalized lymphadenopathy, abdominal pain, or shortness of breath. The visual representation of iterative value stream cycles encourages continual reframing of the differential diagnosis as probabilities evolve.
Our surveys and verbal feedback during the seminar sessions revealed that the concepts of waste reduction and value stream mapping were regarded by students as helpful for creating a framework for improving their clinical skills, and we were encouraged that many students intend to apply this approach in their future clinical rotations. The very positive evaluations may represent selection bias because our course was an elective. The use of multiple-choice tests largely tests knowledge, and we were unable to assess improvements in clinical skills or behavior.
We are encouraged by the students’ positive attitudes towards the application of manufacturing principles to the diagnostic and management process. We are unaware of other medical school curricula that have addressed diagnostic error and diagnostic accuracy by applying the TPS principles. Future studies are planned to assess the ultimate goals of our curriculum which are to improve the novice clinician’s diagnostic accuracy and to reduce diagnostic waste.
We would like to thank the University of Florida medical students who elected to take this course. This study is unfunded. Dr. Singh is partly supported by the VA Health Services Research and Development Service Center for Innovations in Quality, Effectiveness and Safety (CIN13-413). The authors have no conflicts of interest. This study was approved by the IRB approval number 201900079.
Southwick FS. Critically ill: a five-point plan to cure health care delivery. Carlsbad, CA: No Limit Publications Group, 2012. Google Scholar
Plsek PE. Accelerating health care transformation with lean and innovation: the Virginia Mason experience. Boca Raton, FL: CRC Press, 2014. xix, 196 pages p. Google Scholar
Gabow PA, Goodman PL. The lean prescription: powerful medicine for our ailing healthcare system. Boca Raton, FL: CRC Press, Taylor & Francis Group, 2015. xxii, 161 pages p. Google Scholar
Center VMM. Sepsis Power Hour 2018. Available at: https://www.virginiamason.org/vmps-success-stories. Accessed: 8 March 2019.
Reason JT. Human error. Cambridge, UK, New York, NY: Cambridge University Press, 1990. xv, 302 p. Google Scholar
Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med 2013;173:702–4. Web of ScienceCrossrefPubMedGoogle Scholar
Wright A, Bates DW. Distribution of problems, medications and lab results in electronic health records: the pareto principle at work. Appl Clin Inform 2010;1:32–7. CrossrefWeb of SciencePubMedGoogle Scholar
Gershengorn HB, Kocher R, Factor P. Management strategies to effect change in intensive care units: lessons from the world of business. Part I. Targeting quality improvement initiatives. Ann Am Thorac Soc 2014;11:264–9. PubMedCrossrefGoogle Scholar
Gupta K, Hooton TM, Naber KG, Wullt B, Colgan R, Miller LG, et al. International clinical practice guidelines for the treatment of acute uncomplicated cystitis and pyelonephritis in women: A 2010 update by the Infectious Diseases Society of America and the European Society for Microbiology and Infectious Diseases. Clin Infect Dis 2011;52:e103–20. CrossrefWeb of ScienceGoogle Scholar
Shulman ST, Bisno AL, Clegg HW, Gerber MA, Kaplan EL, Lee G, et al. Clinical practice guideline for the diagnosis and management of group A streptococcal pharyngitis: 2012 update by the Infectious Diseases Society of America. Clin Infect Dis 2012;55:1279–82. CrossrefGoogle Scholar
Hall MC, Kieke B, Gonzales R, Belongia EA. Spectrum bias of a rapid antigen detection test for group A beta-hemolytic streptococcal pharyngitis in a pediatric population. Pediatrics 2004;114:182–6. CrossrefGoogle Scholar
Sikkens JJ, Beekman DG, Thijs A, Bossuyt PM, Smulders YM. How much overtesting is needed to safely exclude a diagnosis? a different perspective on triage testing using bayes’ theorem. PLoS One 2016;11:e0150891. PubMedWeb of ScienceCrossrefGoogle Scholar
Cipoli DE, Martinez EZ, Castro M, Moreira AC. Clinical judgment to estimate pretest probability in the diagnosis of Cushing’s syndrome under a Bayesian perspective. Arq Bras Endocrinol Metabol 2012;56:633–7. Web of ScienceCrossrefGoogle Scholar
The online version of this article offers supplementary material (https://doi.org/10.1515/dx-2018-0081).
About the article
Published Online: 2019-03-16
Published in Print: 2019-06-26
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.