Accessible Published by De Gruyter September 27, 2014

Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review

Daniel M. Kobewka, Paul E. Ronksley, Jennifer A. McKay, Alan J. Forster and Carl van Walraven


Laboratory and radiographic tests are often ordered unnecessarily. This excess testing has financial costs and is a burden on patients. We performed a systematic review to determine the effectiveness interventions to reduce test utilization by physicians. The MEDLINE and EMBASE databases were searched for the years 1946 through to September 2013 for English articles that had themes of test utilization and cost containment or optimization. Bibliographies of included papers were scanned to identify other potentially relevant studies. Our search resulted in 3236 articles of which 109 met the inclusion criteria of having an intervention aimed at reducing test utilization with results that could be expressed as a percent reduction in test use relative to the comparator. Each intervention was categorized into one or more non-exclusive category of education, audit and feedback, system based, or incentive or penalty. A rating of study quality was also performed. The percent reductions in test use ranged from a 99.7% reduction to a 27.7% increase in test use. Each category of intervention was effective in reducing test utilization. Heterogeneity between interventions, poor study quality, and limited time horizons makes generalizations difficult and calls into question the validity of results. Very few studies measure any patient safety or quality of care outcomes affected by reduced test use. There are numerous studies that use low investment strategies to reduce test utilization with one time changes in the ordering system. These low investment strategies are the most promising for achievable and durable reductions in inappropriate test use.


Healthcare spending in has increased in both relative and absolute terms over the past several decades [1, 2]. This increase has been seen in many countries and will be unsustainable if the trend continues at the present rate [3]. Laboratory and radiographic testing is a promising target for reducing spending because many of the tests ordered by physicians are suspected to be unnecessary; in some studies, 95% of tests performed are inappropriate as judged by criteria of redundancy or their probability of adding value to patient care [4].

The perceived prevalence of inappropriate laboratory utilization has prompted many attempts to reduce test utilization. Several systematic and narrative reviews have been published on this topic [5–8]. The most recent review, published in 1998, used a behavioral framework to classify interventions and found that targeting multiple behavioral factors was more successful than targeting a single factor [8]. However, this review only dichotomized study outcomes by statistical significance, making it difficult to understand the utility of these interventions. Other reviews used categories of education, audit and feedback or other but were unable to make generalizations about which strategy is most effective because of differences within each category of intervention, a wide range of effects, and lack of a common measure [5, 6]. To date, no review has quantitatively compared the influence of various interventions on test utilization reduction. In addition, previous reviews have not examined investments required to implement these interventions. For these reasons, we performed a systematic review to determine the effectiveness of all interventions to reduce test utilization by physicians.


Data sources and searches

We performed a systematic review following a predetermined protocol in accordance with published guidelines for reporting of systematic reviews of randomized controlled trials [9]. We identified all potentially relevant articles published in English by searching Medline (1946 through September 2013) and Embase (1947 through September 2013). Searches were enhanced by systematically scanning bibliographies of identified articles and relevant review articles as well as articles deemed by PubMed to be related to the included studies. To search electronic databases, we used the strategy recommended for systematic reviews of interventional studies [10] and specified two comprehensive search themes (Online-Appendix 1). Theme 1 identified relevant terms related to laboratory utilization. Theme 2 related to optimization and cost containment. We then combined the two datasets using the Boolean operator ‘and’ and limited the intersection to human studies.

Study selection

Relevant articles were selected from the retrieved studies using a two-phase process. Abstracts were first reviewed for eligibility by three researchers (DK, PR, JM). All abstracts reporting on the effect of an intervention on laboratory utilization were selected for full text review. This initial stage was intentionally liberal; we discarded only abstracts that clearly were not reporting inventions aimed at reducing test utilization. Full text articles were then assessed by one reviewer (DK) with verification by two reviewers (PR, JM) to determine if the study met the specified inclusion criteria. Inclusion criteria included: 1) study population had to include physicians; 2) the aim of the intervention was the modification of test utilization; 3) a comparator arm (either standard care or no intervention) was needed; and 4) the study had to quantify laboratory test utilization with and without the intervention so that a percent change could be calculated. All study designs were considered within this review.

Intervention classification

Interventions used to effect physician laboratory ordering practices were categorized into one or more of the following non-exclusive categories:

  1. Educational interventions in which appropriate test ordering (including the distribution of guidelines) was taught to physicians;

  2. Audit and feedback interventions in which physicians were presented their test utilization compared to their previous utilization or peer utilization or the total costs of the tests they ordered;

  3. System-based interventions involving one-time, permanent changes to test ordering processes including: order form modifications; computer order entry systems with rules disallowing test ordering in specific circumstances; and clinical decision support systems (CDSS) in which an interactive computer system forces physicians to integrate previous knowledge about the patient and/or the medical literature into the test ordering process; and

  4. Incentive or penalty interventions in which physicians received rewards or punishments for certain test ordering practices. For each intervention, we also determined whether the targeted physicians were actively engaged during the intervention’s development [8].

Data extraction and quality assessment

For each study, we extracted information on publication year, country of origin, study design (randomized two-arm trials, before and after studies, prospective cohort), the healthcare providers targeted and the tests targeted for reduction. We recorded characteristics of the study intervention(s) along with its duration. We assessed study quality using an adapted version of the Effective Practice and Organization of Care (EPOC) Cochrane guidelines for interventional studies [11]. In particular, we assessed whether: patients and providers were similar across intervention and comparator groups; a randomized control group was used within the study, there was sufficient detail to describe the intervention, there was risk of contamination between the intervention and comparator groups, and whether a time-series analysis was performed. Lastly, we assessed whether studies measured any patient safety outcome that could be affected by reducing test utilization.

Statistical analysis

The unit of analysis was the intervention, with some studies having more than one intervention. The effectiveness of interventions was summarized using the percent relative reduction in test volume for the intervention group relative to the comparator group. This was calculated as:


Given the variability in the types of interventions, duration of interventions, targeted tests, and patient populations, we did not conduct a meta-analysis. Instead, a descriptive analysis of the relative reductions was performed using medians and inter-quartile ranges (IQR) to summarize effects across the various intervention types. We also documented when these reductions in a particular study were statistically significant. Results were stratified by categories of interventions (educational, system based, audit/feedback and incentive and penalty) and were visualized using box plots. Stratifications were then conducted by additional interventional factors within each category and by measures of study quality and statistical significance. The effect of physician involvement was also measured across interventions. Finally sensitivity analyses were conducted to assess the effect of interventions that targeted four or more tests.

In order to highlight the interventions with the best return on investment we included a narrative review of interventions with the largest reductions in test utilization as well as the most effective system-based interventions targeting four or more tests. All analyses were performed using Stata version 13.0 (Stata Corp., College Station, TX, USA).


Our search generated 3236 unique citations. Following the initial screening and full-text review, 109 unique studies that included 119 interventions were included in the systematic review (Figure 1) [12–121]. Study characteristics and brief descriptions of each intervention are shown in Table 1. Ninety-three studies (85.3%) used before-after designs, 2 (1.8%) were non-randomized prospective trials, and 14 (12.8%) were randomized controlled trials. Studies were published between 1974 and 2013 with 61 studies (56.0%) performed in North America, 27 (24.8%) in Europe, 11 (10.1%) Australasia, and 10 (9.2%) in other regions. Studies used very limited time horizons with only 15 of 109 studies (13.8%) measuring intervention effects beyond 1 year.

Figure 1 Selection of articles in systematic review.The search was performed in September of 2013.

Figure 1

Selection of articles in systematic review.

The search was performed in September of 2013.

Table 1

Study characteristics and description of interventions.

Author YearCountryHealthcare workers targetedSiteInterventionComparatorCategory of InterventionTargeted testsRelative reduction of targeted tests
Amukele 2011USAPhysicians working in surgery, hematology, and internal medicineSingle teaching hospitalThe order form was modified with some tests being bundled and others being unbundled. Physicians who were high users were educated about coagulation tests and an algorithm was distributedBefore-afterXXPT, PTT, TT and fibrinogen77.8
Archambault 2011USAAll physiciansSingle teaching hospitalESR testing was no longer performedBefore-afterXESR and CRP47.5
Attali 2006IsraelResident physiciansSingle teaching hospitalEducational lecture about excessive and inappropriate testing; tests were unbundled on the request form; a senior physician gave feedback about ordering practicesBefore-afterXXXAll blood tests34.5a
Barazzoni 2002SwitzerlandAll physicians seeing patients pre-operatively for elective procedures6 community hospitals with surgery facilitiesAll stakeholders were involved in a guideline creation process. The guidelines were then adopted as hospital policyBefore-afterXAll pre-operative tests26.1a
Bareford 1990USAAll medical staff: consultants and junior house staffSingle community hospitalMonthly usage statements were distributed to physicians. Guidelines on appropriate testing were distributed and some tests were canceled if inappropriateBefore-afterXXXCBC, ESR and PT62.1
Baricchi 2012ItalyGeneral practitioners4 primary care clinicsTesting algorithms for 7 common clinical scenarios were developed. Physicians were then educated about the algorithms and told to use themBefore-afterXXAll blood tests4.7
Barie 1995USAAll medical staff in the ICUSingle teaching hospitalTesting pathways were introduced and audit and feedback was done monthlyBefore-afterXXAll blood tests and chest radiographs34.6
Bates 1999USAAll physiciansSingle teaching hospitalDefinitions of a redundant test were created. The computer order entry system created reminders when tests were ordered that were redundantRandomized trialXElectrolytes, urinalysis, digoxin, tobramycin, aminophyline, vancomycin, and gentamycin levels, urine, sputum and stool cultures, Clostridium difficile testing, fibrin split products0.1
Bates 1997USAAll physiciansSingle teaching hospitalCosts were displayed for each test that was ordered.Before-afterXAll blood tests and the 35 most common radiologic tests4.1
Berenholtz 2001USAPhysicians working in a surgical ICUSingle teaching hospitalClinical pathways were created for various surgical problems. Appropriate laboratory testing was suggested by the pathwayBefore-afterXAll laboratories included in the pathway–6.8
Berwick 1986USA35 internistsSingle health management organizationDoctors met to discuss the use of particular tests and journal articles were distributed on the use of these tests Clinicians received individualized feedback on their rates of test ordering compared to their peers and their rank relative to their peers Physicians were ranked by the proportion of tests they ordered that were abnormal. Each physician was then given their rankBefore-afterXESR, T4, serum glucose, heterophile antibody testing, ANA, SLE preparation, RF, liver enzymes, cholesterol levels, radiographs of peripheral bones and chest, ECGs0.2
Boon-Falleur 1995BelgiumPhysicians working on liver a transplant unitPediatric liver unitA rule based decision support application decided on the next day’s laboratory tests based on previous resultsBefore-afterXAll blood tests24.9
Buckingham 1994ScotlandAll physiciansAll hospitals in a single cityJustification for tests ordered had to be given. Specifics on how this was done are not givenBefore-afterXAll biochemical blood tests1.4
Bunting 2004Canada200 physicians who ordered the most tests in 1997Single provinceFeedback on test usage was given four times over a 2-year periodRandomized controlled trialXAll blood tests7.8a
Burnett 1991AustraliaAll physicians requesting blood work from a single medical laboratorySingle private medical laboratoryAll laboratory tests were unbundled so they had to be ordered individuallyBefore-afterXAll blood tests18.8a
Calderon-Margalit 2005IsraelAll physiciansSingle teaching hospitalSeveral interventions including mandatory consultation prior to ordering, education, restriction or elimination of various testsBefore-afterXXAll blood tests19.0a
Carson 1995USAAll physiciansSingle teaching hospitalAn algorithm that automatically restricted fibrin degradation product testing and instead used D-dimer testing for suspected DICBefore-afterXD-Dimer and fibrin degradation products48.3a
Carter 2002USAPhysicians sending specimens to the labSingle teaching hospitalAll test requests were reviewed by a more senior physicianBefore-afterXAll laboratories that cost >$75 USD23.0
Chen 2003USAAll Physicians ordering selected testsSingle teaching hospitalA reminder on the half-life of various anti-epileptic drugs was shown on the computer when repeat levels were requested in a short time periodBefore-afterXXAnti epileptic drug levels23.4a
Chu 2012AustraliaMedical students and residentsSingle teaching hospitalSelected tests were restricted and required a senior staff member to sign off on them prior to being orderedBefore-afterXCoagulation tests, thyroid function tests, ESR and d-dimer7.5a
Chu 1996USAPhysicians taking care of trauma patientsSingle teaching hospitalA liberal admission algorithm with extensive testing was compared to physicians only ordering tests they thought necessaryBefore-afterXHematology, chemistry and coagulation profile, urinalysis, blood gas, blood alcohol level, type and screen68.3
Cohen 1982USAAll physicians working on the teaching units4 clinical teaching units at a single teaching hospitalThe cost of all imaging tests ordered were given to each team weeklyRandomized controlled trialXCXR, CT scans, US and liver scans12.9
The cost of all blood tests ordered were given to each team weeklyRandomized controlled trialXAll blood tests24.7a
Davidoff 1989USA24 medical interns entering a university training programSingle teaching hospitalLectures on probability and test characteristics were compared to placebo lectures on economics and cost controlRandomized controlled trialXAll blood tests16.5a
Detsky 1986CanadaAll attending physicians20 teaching hospitals in a single provinceResidents and interns went on strike for 7 days in 1980Before-afterXAll blood tests8.3a
Dickinson 1987USAAll physicians in the institutionSingle teaching hospitalThe number of unnecessary tests was measured and shown to department headsBefore-afterXAll blood tests24.5a
Dixon 1974USAInterns working on the wardsSingle teaching hospitalInterns and residents were limited to 8 tests eachBefore-afterXAll blood chemistry tests66.7a
Dowling 1989USAResidents in a family health centerSingle family health centerLectures and guidelines on appropriate testing were distributed. Audit and feedback of test usage was performed repeatedlyBefore-afterXXCBC and TSH47.5a
Durand-Zaleski 1993FranceAll physicians at the hospitalSingle teaching hospitalThe laboratory requisition was changed so that the purpose of each tumor marker was included on the requisitionBefore-afterXTumor markers23.9a
Durieux 2003FranceAll physicians at the hospitalSingle teaching hospitalThe laboratory requisition was changed so that the purpose of each tumor marker was included on the requisition. The purpose for each test, i.e., screening, follow-up or diagnosis was explained on the requisitionBefore-afterXXCEA, alpha feto-protein, CA19-956.5a
Eccles 2001England and ScotlandGeneral PractitionersMultiple primary care clinicsGuidelines on when to order lumbar spine and knee X-rays were circulated along with feedback on the number of requests to each physicianRandomized controlled trialXXKnee and lumbar spine XR40.8
Eisenberg 1977USAResident physiciansSingle community hospitalThere was an education program with a weekly lecture; results from a single audit on utility of testing were circulated along with a memorandum from the medical director asking for careful consideration of test usageBefore-afterXPTT42.1a
Emerson 2001USAAll physicians working at the medical centerSingle teaching hospitalLaboratory requisition was redesigned with some tests unbundled and cascades implemented for thyroid function and anemiaBefore-afterXAll blood tests4.7a
Everett 1985USAFirst year residentsSingle teaching hospitalResidents were taught about laboratory utilization using specific casesRandomized controlled trialXAll blood tests and microbiology–12.3
Everett 1983USAFirst year residents training in internal medicineSingle teaching hospitalAudit and feedback of costs was performed weekly, cost education news letters were distributed and costs of tests were put in patient charts. There were one-on-one meetings of faculty with residents to discuss costsRandomized controlled trialXXAll blood tests15.1a
Feldkamp 1996USAAll physicians in health centerSingle teaching hospital and 22 satellite clinicsAlgorithms were developed for thyroid function testing and implemented by the laboratoryBefore-afterXThyroid function testing21.6
Feldman 2013USAAll providers who order laboratory tests through a computerized order entry systemSingle teaching hospitalCost of each test was displayed when it was orderedRandomized controlled trialXX61 diagnostic blood tests8.6a
Finegan 2005CanadaAll physician trainees in a hospitalSingle teaching hospitalEach anesthetist selected tests to be performed preoperatively at their discretion. The comparator was surgery specific protocols that were being usedBefore-afterAll blood tests3.6
Fong 2008AustraliaAll physicians working in the emergency roomSingle teaching hospitalA clinical decision rule was used instead of physician discretionBefore-afterXCT head–27.7a
Fowkes 1986United KingdomAll residents working on the medical unitSingle teaching hospitalA guideline on testing was distributed and weekly meetings were held with medical staff to discuss their use of tests in the previous weekNon-randomized controlled trialXXAll blood tests63.4a
Friedman 2010USAPhysicians taking care of children post procedureSingle teaching hospitalA set of standard assessment and management plans were developed and implemented after certain proceduresBefore-afterXAll blood tests and imaging15.6
Froom 2012IsraelAll physiciansRegional laboratory serving a single Health Management OrganizationUrine microscopy was no longer done automatically instead the physician had to specifically request itBefore-afterXUrine for dip and microscopy99.7a
Gama 1991United KingdomGeneral internistsSingle community hospitalData on personal and peer expenditure of laboratory resources was given to physicians monthly for 12 monthsRandomized controlled trialXClinical chemistry24.6a
Gama 1992United KingdomPhysicians taking care of inpatientsSingle community hospitalData on personal and peer clinical chemistry expenditure per patient was provided to physicians monthly for 12 monthsBefore-afterXClinical chemistry and hematology27.0a
Goddard 2011IrelandAll physicians working in the ICUSingle teaching hospitalAll recurring orders for blood work were discontinued; instead each test had to be ordered as a single occurrenceBefore-afterCBC, urea, electrolytes, coagulation, LFTs, magnesium, Ca, PO4, albumin, CRP33.1
Golden 1987USAAll physiciansOutpatient department of a single teaching hospitalThe cost of tests billed to the patient was changed so that ordering individual tests was cheaper than ordering the whole panelBefore-afterXAll blood chemistry test33.5a
Gortmaker 1988USAAll physicians working in the hospitalSingle community hospitalNine staff meetings were held to discuss cost issues. Data on excess usage was sent to all doctors. All physicians reached a consensus on when each test was considered inappropriateBefore-afterXBUN, CBC, chemistry profile, CK, creatinine, electrolytes, fasting and random blood sugar, hematocrit, PT, SGOT, urinalysis14.0a
Grivell 1981AustraliaAll physicians caring for inpatientsSingle teaching hospitalPhysicians received a report every 4 weeks on the type and number of tests ordered. Physicians were also shown where they rank in comparison to their peersBefore-afterXAll blood tests0.0
Groopman 1992USAAll medical staff ordering laboratories in the ERSingle teaching hospitalTests were taken off the standard requisition and had to be written in by handBefore-afterXPT/PTT61.6a
Gross 1988USAAll physicians taking care of inpatientsSingle teaching hospitalGuidelines on when to order blood cultures were distributedBefore-afterXBlood cultures75.0a
Haschke-Becher 2009AustriaNo doctors were targeted, it was a laboratory interventionSingle teaching hospitalA decision rule was created to determine when the laboratory would process liquor diagnosticsBefore-afterXLiquor diagnostics28.6a
Hutton 2009United KingdomAll emergency room and medical unit physiciansSingle teaching hospitalRequests for CRP from the ER had to be approved by a senior physician. Junior staff were told to limit CRP use. Lastly repeat testing within 24 h was disallowedBefore-afterXXCRP85.0a
Jelinek 1990AustraliaAll physiciansSingle teaching hospitalGuidelines on when abdominal films could be ordered were circulated and all requests had to be run by a senior physicianBefore-afterXXAbdominal plain films52.2
Kelly 1998AustraliaAll physicians in the ERSingle community hospitalGuidelines on who should receive a blood culture were distributedBefore-afterXBlood cultures53.0
Kumwilaisak 2008USAAll physicians working in the surgical ICUSingle teaching hospitalGuidelines regarding when certain blood tests should be used were introduced at a staff meeting and sent out by email. A session was repeated for new residents every monthBefore-afterXAll blood tests20.8a
Larocque 1994CanadaSurgeons and surgical house staffSingle teaching hospitalGuidelines on which tests were appropriate for different medical conditions were posted on the wards and distributed as pocket cardsBefore-afterXPre-op screening tests including blood work, CXR and ECGs10.1a
Levick 2013USAClinicians ordering the BNP testA health network including 2 hospitals and several community health centersComputer order entry system displayed a warning when a repeat BNP was being orderedBefore-afterXBNP22.2a
Lewandrowski 1994USAAll physicians working at the hospitalSingle teaching hospitalA guideline on appropriate testing was developed. All requests were then reviewed by laboratory staff and the ordering physician was contacted if the request was not in accordance with the guidelineBefore-afterXXLD-isozymes99.6
Mancuso 1999USAAll physicians at the hospitalSingle specialized orthopedic surgery hospitalGuidelines were published suggesting selective test ordering preoperativelyBefore-afterXPre-operative blood work, ECG, CXR,30.0a
Martin 1980USAResidents and medical studentsSingle teaching hospitalThere was a 1 h lecture on laboratory costs. Money was given to the residents if their laboratory use decreasedNon-randomized controlled trialXXAll radiologic and blood tests5.8a
There was a 1-h lecture on laboratory costs. Senior physicians then met weekly with residents, they reviewed cases and suggested changes in test ordering practices.XX–17.8a
Marton 1985USAResidents and medical studentsSingle teaching hospitalA manual about rational test ordering was given to the traineesRandomized controlled trialXAll blood tests18.2
Feedback on laboratory usage and costs incurred was shown to trainees along with their usage compared with peersX33.3
A manual about rational test ordering was given to the trainees. They also received feedback on laboratory usage, costs incurred and their usage compared with peersXX40.9a
May 2006USAAll docs working on inpatient serviceSingle teaching hospitalDuplicate orders were deleted and all blood work orders expired in 24 h. Blood work would only be drawn every 4, 6 or 12 h. Future orders could only be for a single occurrenceBefore-afterXAll blood tests11.5a
Mehari 1997New ZealandPhysicians working in ICUSingle teaching hospitalA guideline was placed on each patients chart stating when it is appropriate to do each testBefore-afterXCBC, coagulation tests, Cr, Na, K, urea, calcium, glucose, Mg, PO4, ABGs21.2
Meng 2006CanadaAll physicians3 teaching hospitalsA policy was created regarding who is allowed to order troponin testing and how many repeats can be orderedBefore-afterXTnI2.9
Merlani 2001SwitzerlandDoctors residents and nurses working in the intensive care unitSingle teaching hospitalA guideline on when arterial blood gases can be done was circulatedBefore-afterXXABG41.5a
Meyer 2010USAGynecologic oncologists and other physiciansSingle teaching hospitalA multidisciplinary team created a guideline and then created a communication plan to disseminate itBefore-afterXPap tests31.2a
Miyakis 2006GreeceAll physicians working at the hospitalSingle teaching hospitalThe results of an audit on test use were presented along with strategies for reducing utilizationBefore-afterXX25 laboratories hematology and chemistry tests13.5a
Morris 1992USAAll physicians working at the hospitalSingle teaching hospitalOnly one stool sample for parasites was accepted by the laboratory instead of threeBefore-afterXStool for ova and parasite33.3
Mutimer 1992United KingdomAll physicians working in the liver unitLiver unit at a single teaching hospitalLaboratory tests to be ordered for the next day were determined by protocol based on patient characteristics and previous testingBefore-afterXHematology, biochemistry, immunology, microbiology and cross matching9.3a
Nardella 1995USAAnesthetists and surgeonsSingle teaching hospitalGuidelines about appropriate pre-operative testing were distributed and discussed at staff meetingsBefore-afterXCBC, PT, PTT, bleeding time, BUN, Cr, Glucose ALT AST, electrolytes61.8a
Neilson 2004USAResidents, physicians and nurse practitioners who used computer order entrySingle teaching hospitalRecurring tests had to be confirmed daily, many tests were limited to a single occurrence and previous results were displayed when ordering testsBefore-afterXAll blood tests and imaging64.2a
Nelson 1978USAAll physicians who work at the hospitalSingle community hospitalThere was a 20-min grand rounds presentation on B12 testing. Suggestions were given on when to order this testBefore-afterXB12 and folate45.3
Nightingale 1994United KingdomResidents working on the liver unitLiver unit at a single teaching hospitalA computerized protocol was created that suggested tests for the next day based on current laboratory values and patient characteristicsBefore-afterXAll blood tests23.7
Nirel 2011IsraelAll physicians working in the health systemPrimary care clinicsAn online medical record was introduced so that old laboratory tests could be accessedBefore-afterXAll blood tests3.6a
An online medical record was introduced so that old radiology tests could be accessedBefore-afterXAll radiology tests12.3
Pageler 2013USAPhysicians working in pediatric ICUSingle teaching hospitalA rule preventing repeating testing was created in the computer order entry system. It had to be overridden to order repeat testingBefore-afterXCBC, chemistry and coagulation37.7a
Pageler 2009USANot reportedSingle teaching hospitalRecurring orders for CXR were no longer allowed and instead had to be re-ordered dailyBefore-afterXChest XR10.0
Patel 2005United KingdomAll physicians working in the hospitalSingle teaching hospitalUrine was not cultured if the urinalysis was normalBefore-afterXUrine culture4.0
Pilon 1997CanadaPhysicians and nurses working in the ICUSingle teaching hospitalA guideline about when ABGs should be used was created. It was distributed on pocket cards and taught during education sessionsBefore-afterXABGs36.7a
Pop 1989NetherlandGeneral practitionersAll physicians using the same laboratory in one cityTwo times per year physicians were given a report of tests they had ordered and whether they were appropriate or not. Physicians were also informed of their redundant testsBefore-afterXBUN, Cr, liver enzymes, WBC, Hgb, ESR, glucose36.2
Power 1999AustraliaSurgical Residents in pre-op clinicSingle teaching hospitalPublished guidelines on test ordering were distributed. A maximum number of tests per patient was instituted and test ordering was reviewed by a consultantBefore-afterXXXCBC, coagulation tests, urea, LFTs, CXR, ECG37.1
Powles 2009United KingdomAll physicians in region using the university laboratory for PSA testingSingle teaching hospitalProstate-specific antigen was taken off the request form and had to be written in by handBefore-afterXPSA17.3a
Prat 2009FranceAll physicians working in ICUSingle teaching hospitalA guideline was created on when to perform each of the common blood tests. The cost of each test was also shown on the guideline. Teaching was done on how to reduce test utilizationBefore-afterXCXR and daily routine BW49.0a
Pysher 1999USAAll physicians ordering tests at this laboratorySingle teaching hospitalPredefined chemistry panels were eliminated from the requisition forms. Instead each test had to be ordered separatelyBefore-afterXAll blood chemistry tests32.7a
Ramoska 1998USAPhysicians and residents working in the emergency departmentSingle teaching hospitalGraphs of laboratory utilization and costs of laboratory tests at the hospital was displayed on a bulletin boardBefore-afterXAll blood tests17.8a
Ratnaike 1993AustraliaAll physicians working in the cardiac care unitSingle teaching hospitalGuidelines were created by the hospital and issued to the cardiology unit. No further details about the guidelines were givenBefore-afterXAll blood tests57.2
Rhyne 1979USAFamily doctorsThe family medicine program associated with one universityA chart audit of test ordering was done and the results were presented to medical staffBefore-afterXXThyroid function tests36.4a
Roberts 1993CanadaAll physicians working at the hospitalSingle teaching hospitalSerum osmolality was removed from the requisition and blood count differential was not done if white cell count was normal. A protocol was used to determine when electrolytes should be checked. Daily CXRs were no longer allowedBefore-afterXXBlood gases, serum osmolality, CBC, Na, K, glucose, CXR, and ECG18.1a
Rosenbloom 2005USAAll physicians working on selected inpatient unitsSingle teaching hospitalMagnesium could only be ordered once with no recurrence and a screen displaying old results and education on when magnesium should be checked had to be viewed before the order could be enteredBefore-afterXXMagnesium55.2a
Santos 2012BrazilAll residents and physicians working at the hospitalSingle teaching hospitalA meeting was held to discuss evidence for certain tests and the costs of the tests. Patterns of testing and appropriateness of various tests was discussedBefore-afterXCRP48.4
Seguin 2002FranceAll physicians working in ICUSingle teaching hospitalCosts for each test was shown on the order formBefore-afterXElectrolytes, CBC, ABG, LFTs, coagulation panel, CXR18.9
Shalev 2009IsraelAll primary care physicians in IsraelSingle health service provider27 tests were removed from the requisition and 2 were addedBefore-afterXAll blood test4.3a
Sorita 2011USAResidents training in medicine and surgerySingle teaching hospitalThere was a 1 h teaching session regarding when to order STAT blood tests and results of an audit were presented. The highest users of STAT tests were given individual feedbackBefore-afterXXAll stat laboratory tests21.0a
Spray 2012USAPhysicians taking care of inpatientsSingle community hospitalAn electronic protocol for electrolyte replacement and retesting was institutedBefore-afterXMagnesium, phosphorous and ionized calcium32.0a
Stafford 2003USA117 primary care providersPrimary care practices affiliated with a single teaching hospitalAudits of individual and group ECG usages patterns were mailed to physiciansBefore-afterXECG28.3
Sucov 1998USAAll physicians working in the hospitalSingle teaching hospitalGuidelines were created regarding what tests to order for which patient groups. The guideline was then rolled out with a series of educational sessions. Each department received feedback on its ordering practices but there was no individual feedbackBefore-afterXXAll blood tests22.0a
Sussman 1984USAAll physicians working in the hospitalSingle teaching hospitalAll recurring orders were eliminatedBefore-afterXCBC, electrolytes, CXR, chemistry panel, FBG, electrocardiogram, PT20.8
Thompson 1983USAAll physicians part of the health planSingle pre-paid health planA recommendation was made against screening healthy adults with blood tests and CXR. An educational campaign was then implementedBefore-afterXAll blood chemistry tests and CXR60.0a
Tierney 1987USAResidents and staff working in the clinicSingle university associated general medicine clinicAll previous results for a test were shown when using the computer order entry system. An option to cancel the current test was then availableRandomized controlled trialXAll laboratory and imaging and ECGs8.9a
Tierney 1990USAPhysicians working in the clinicSingle university associated general medicine clinicCost to the patient of ordering each tests was displayed before the order is confirmedRandomized controlled trialXAll blood tests17.6a
Toubert 2000FranceAll physicians in the hospitalSingle teaching hospitalEducational material on thyroid disease was distributed. Physicians were told that any test other than TSH required justification. Tests were still processed even if no justification was providedBefore-afterXTSH, T3, T4,39.6
van Wijk 2001NetherlandsGPs working in practices with electronic medical recordsSingle health regionComputerized decision support suggested appropriate tests based on basic clinical informationRandomized controlled trialXAll blood tests20.3a
Vardy 2005IsraelAll physicians ordering tests through the HMOSingle health management organizationLists of appropriate laboratory tests for each clinical condition were created and physicians were educated about them. Each list was then put into the computer order entry systemBefore-afterXXAll blood tests4.2
Verstappen 2003NetherlandsPhysicians – not specified5 health regions in the countryEach physician received feedback on their ordering practices regarding upper and lower abdominal pain and cardiovascular conditions compared with their peers. The physicians then met together to discuss how to change their ordering practicesRandomized controlled trialXXBiochemistry and hematology tests, ECGs, sigmoidoscopy and imaging12.0a
Each physician received feedback on their ordering practices regarding COPD, asthma, degenerative joint complaints and non-specific complaints compared with their peers. The physicians then met together to discuss how to change their ordering practicesRandomized controlled trialXXBiochemistry and hematology tests, ECGs, sigmoidoscopy and imaging8.0
Wagholikar 2011AustraliaAll physicians working in the emergency departmentSingle teaching hospitalApproval from a senior physician was required to order tests in certain circumstancesBefore-afterXCBC and ESR11.8a
Winkens 1995NetherlandsFamily physiciansSingle regional medical laboratoryIndividualized feedback was sent to each physician from a respected internal medicine physician. The feedback included alternative testing strategiesBefore-afterXCervical smear, ECG, endoscopy, allergy tests, radiographic and ultrasound tests5.5a
Winkens 1992NetherlandsFamily physiciansSingle regional medical laboratoryBiannual reports were sent on the number and rationality of laboratory testing compared to peersBefore-afterXX46 blood tests25.0a
Wones 1987USAFirst year internal medicine residentsSingle teaching hospitalThe group was simply informed that laboratory utilization was being monitoredRandomized controlled trialXAll laboratory tests–11.4
This group received individual feedback on laboratory usage and costs every 2 weeksXAll laboratory tests–13.15
This group received individual feedback on laboratory usage and costs every 2 weeks along with group data for comparisonXAll laboratory tests–7.27
Zaat 1992NetherlandsGeneral PractitionersTwo health regionsThe laboratory requisition was modified so that it only had 15 tests on it. All other tests had to be written inBefore-afterXAll hematological and chemistry tests18.2a
Zaidi 1999USAAll physicians sending samples to the hospital labSingle teaching hospitalGuidelines were created from local data and shared with all medical staff. After this all samples not meeting the criteria were rejectedBefore-afterXXStool culture30.9
Zia 2013USAAll Physicians at the hospitalSingle teaching hospitalThe test was removed from the laboratory requisition but could still be ordered if written in by handBefore-afterXSerum protein electrophoresis and immunofixation29.6

aDenotes studies in which a test of statistical significance was performed and found to be significant by the studies’ authors. A, Audit and feedback; E, Educational; I, Incentive or penalty; S, System-based.

Study quality

Study quality is summarized in Online-Appendix 2. Of the 109 studies, 14 (12.8%) had a randomized design and 37 (33.9%) provided evidence that patients were similar between groups. Time-series analysis was performed in only 11 (10.1%) studies. Interventions were described adequately to be reproducible in 88 (80.7%) studies. Test utilization was reported with the patient or the encounter as the unit of analysis in 79/109 (72.5%) studies with the rest using the physician or the group of physicians working that location as the unit of analysis.

Intervention components

Of 119 interventions, 51 (42.9%) had an educational component, 54 (45.4%) had a system-based component, 41 (34.5%) had an audit feedback component, with only one (0.8%) using incentive or penalty. Physicians were involved in the development of interventions in only 17 (14.3%) interventions. Thirty-two (26.9%) interventions were multifaceted, being classified into more than one of the four main intervention categories. A measure of patient safety or quality of care was included in 13 (11.9%) studies.

Intervention effects

The percent relative reductions in laboratory utilization ranged from a +99.7% (a reduction in the test volumes) to –27.7% (an increase in test volume). Table 2 displays the percent relative reductions in test volume for each intervention and their sub-categories. Interventions with an educational component had the highest median relative reduction in test volume at 34.5% (IQR 16.5–49.0) while audit feedback or system-based interventions had a relative reduction of 22.0% (8.6–34.6) and 22.2% (3.6–68.3), respectively (Figure 2). The single incentive and penalty intervention had a 5.8% relative reduction.

Table 2

Percent relative reductions in test utilization by intervention type.a

Number of interventionsMedian RR (IQR)Minimum RRMaximum RR
Educational interventions5134.5 (16.5–49.0)–12.399.6
 –Δ between intervention and control statistically significant3331.2 (18.1–47.5)4.785.0
 –Δ between intervention and control statistically insignificant1838.3 (8.0–53.0)–12.399.6
Exclusively education intervention2230.6 (16.5–48.5)–12.375.0
Exclusively guideline distribution1330.0 (20.8–53.0)4.799.6
System-based interventions5422.2 (3.6–68.3)0.199.7
 –Δ between intervention and control statistically significant3619.6 (10.4–36.1)3.699.7
 –Δ between intervention and control statistically insignificant1824.8 (10.0–37.1)0.177.8
Exclusively system-based intervention3721.6 (10.0–32.7)0.199.7
Exclusively clinical decision support1022.8 (9.3–23.9)4.785.0
Audit feedback interventions4122.0 (8.6–34.6)–13.199.6
 –Δ between intervention and control statistically significant2423.2 (13.8–34.5)5.563.4
 –Δ between intervention and control statistically insignificant1718.9 (0–36.2)–13.199.6
Exclusively audit and feedback intervention2018.4 (2.1–24.8)–13.136.2
Exclusively cost display1218.4 (2.1–28.8)5.599.6
Physicians targeted were involved in creating and implementing the intervention1723.0 (8.0–39.6)3.699.6
Physicians targeted were not involved in creating and implementing the intervention10322.2 (10.1–36.7)–27.799.7

aCategories of interventions are not mutually exclusive unless stated.

Figure 2 Box plots of relative reductions in laboratory utilization stratified by intervention type.1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Figure 2

Box plots of relative reductions in laboratory utilization stratified by intervention type.

1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Thirty-three (27.7%) of the interventions targeted three or fewer tests, while 86 (72.3%) targeted four or more tests. The median relative reduction for interventions that targeted three or fewer tests was 40.2% (IQR 23.7–52.8) while that targeting four or more tests was 18.8% (IQR 8.2–28.5) (Figure 3). Thirty interventions (25.2%) used multiple interventions; these multifaceted interventions had larger reductions in test use with a median relative reduction in test volume of 32.7% (IQR 15.1–47.5) versus 21.4% (IQR 9.5–33.3) for interventions that were classified into a single category. Interventions for which physicians were engaged during the intervention development had similar median relative reductions as interventions without physician input.

Figure 3 Box plots of relative reductions in laboratory utilization stratified by intervention type among interventions targeting four or more tests.1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Figure 3

Box plots of relative reductions in laboratory utilization stratified by intervention type among interventions targeting four or more tests.

1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Finally, we found important differences in the relative reductions across quality indicators (Table 3). Most notably, the effect of interventions was less if a study reported a reduction in test utilization per patient rather than reduction per institution or physician group and studies with a concurrent control group had smaller reductions. Unexpectedly there were larger relative reductions in studies that had a risk of contamination between the experimental and control groups.

Table 3

General descriptions of study quality (n=109).

Number of studiesMedian relative reduction (IQR)
Where patients similar between groups?
 Yes3719.0 (8.6–37.7)
 No7224.8 (14.0–41.5)
Were those ordering the tests similar between groups?
 Yes10323.8 (11.9–40.2)
 No611.7 (4.2–25.0)
Was there a concurrent control group?
 Yes2716.5 (5.8–27.0)
 No8224.9 (14.0–47.5)
Was the intervention described adequately enough to be replicated?
 Yes8823.5 (12.4–38.6)
 No2120.8 (4.7–36.7)
Was there a risk of contamination between experimental and control groups?
 Yes8924.9 (13.5–40.8)
 No2015.8 (4.8–29.6)
Were the results reported per patient instead of per institution or per physician?
 Yes7919.9 (9.1–35.4)
 No3033.4 (23.4–48.3)
Was a time-series analysis conducted?
 Yes1126.1 (14.0–55.2)
 No9823.5 (10.1–37.1)

Interventions with the largest effects

We found extensive heterogeneity in the components that made up each intervention, how interventions were implemented, the study setting, and the tests that were targeted for reduction. Meaningful generalizations across intervention types is therefore difficult; so the interventions with the largest relative reductions in test volume are described here (Table 1). The studies by Froom et al. and Lewandrowski et al. attained the largest relative reductions with both targeting a single test. Froom et al. changed laboratory policy so that urine microscopy was done only if specifically requested by the physician; this resulted in a 99.7% reduction in test use [53]. Lewandrowski et al. distributed guidelines stating when it was appropriate to order LD-isozymes with a laboratory physician notifying the ordering physician when the LD-isozyme request was not compliant with the guideline [69]. This combination of education and feedback led to a 99.6% reduction in LD-isozymes being ordered. The two largest relative reductions among interventions that targeted four or more tests were by Chu et al. and Dixon et al. Chu et al. used a protocol driven algorithm for admitting trauma patients and found a 68.3% reduction in the number of tests ordered per patient [33]. Dixon et al. limited the number of tests that interns could order to eight per patient per day, resulting in a 66.7% reduction in the number of tests ordered per patient [38].

The most successful interventions targeting four or more tests that were purely system based were the study by Chu et al, discussed above, and a study by Neilson et al. Neilson et al. sequentially implemented a number of system-based interventions using a computerized order entry system including: the display of previous laboratory results to physicians ordering tests; the prevention of recurring orders in various circumstances; and the unbundling of tests so that each component had to be ordered individually [82]. This package of interventions resulted in a 64.2% relative reduction in the tests targeted.


We summarized 109 studies evaluating the effect of interventions aimed at reducing test utilization by calculating relative reductions to compare interventions. We found that all intervention categories (including education, system based, audit feedback, and incentive penalty) can reduce test utilization. Even though educational interventions had the largest relative reduction in test utilization, the inter-quartile ranges were large and overlapping. The greatest difference in our stratified analysis was between interventions that targeted less than four tests, which had much larger relative reductions than interventions that targeted four or more tests. Consistent with past reviews, we found that interventions using multiple strategies were more effective [8]. Even though all intervention types seem to be effective, we found a large range of effects and much heterogeneity between studies. Many studies did not use appropriate statistical techniques and follow-up was often short. These shortcomings call into question the sustainability and generalizability of many interventions in our review.


Although interventions with an educational component had the greatest median relative reductions, their long-term sustainability is questionable. The resources needed to perpetuate educational strategies (i.e., lectures or individual teaching sessions) are significant and probably need to be continuous to maintain an effect, especially in teaching hospitals where ordering physicians are constantly changing. Even in studies where the educational strategy was low effort (i.e., the distribution of a guideline), multiple reminders and repeated meetings were necessary to maintain the change in behavior [89, 97, 106, 111]. There was only one purely educational intervention that reported results from >1 year after the intervention began but this study did not use time series analysis to determine if utilization returned to baseline during this time [70]. With all but one purely educational interventions having a time horizon of <1 year there is no evidence in our review that educational strategies have long-term sustainability. In contrast there are several examples of system-based interventions which by definition require few resources to maintain, that are shown to maintain their effect for >1 year [14, 47, 53]. The ratio of resource input to test reduction is not measured in the included studies but is an important consideration when deciding to implement a particular intervention.


Many of the interventions have questionable generalizability. There is a wide range of relative reductions among included studies due to heterogeneity at every level. There are differences in how an invention type is implemented, who is target by the intervention, and the pre-existing ordering practices among the targeted physicians. Due to the heterogeneity, similar results cannot be expected even if an intervention is perfectly replicated. Some of the large reductions (>70%) suggest that anomalous ordering is being targeted which further calls into question the generalizability of results. Lastly many of the studies target physicians in training who would likely respond differently than physicians in independent practice.


The quality of included studies was poor. As expected studies with lower quality had higher relative reductions suggesting that poor quality studies were over estimating the effects of interventions. Surprisingly, very few of the before and after studies used time-series analysis or autocorrelation to analyze results, bringing into question the validity of results.


This review has several limitations. Our search strategy used only studies published in English ignoring potentially important studies published in other languages and only searched two databases. Another limitation is that the heterogeneity between interventions makes generalizations about effective strategies difficult. Despite this weakness the relative reduction does tell us which interventions were associated with the largest reductions. This allows these successes to be examined to see if they are applicable to other settings. This review is broad in scope and therefore does not allow detailed examination of particular intervention types to find the elements that lead to success. Reviews of particular types of interventions have been done and explore these factors closely [122–124]. Lastly, our use of relative reductions leaves out absolute reduction and cost savings information that may be important to decision makers.


Our review found that research into interventions to reduce test utilization is ongoing and many strategies can be successful. There are many examples of low investment interventions that work, especially when they target tests that have a high rate of inappropriate use. Due to questionable validity and generalizability of these results, this review should be used as a reference for finding high quality studies with large relative reductions that apply to a setting of interest. Future research needs to determine how low input strategies using computerized order entry can be used to implement system changes, audit and feedback, and education. Database monitoring should be developed so that institutions know when anomalous test use is occurring and which physicians are deviating from the mean. This will allow for targeting of interventions to where they will have the greatest effect. Lastly, further studies in this field must monitor for adverse events caused by reduced testing.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Financial support: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

Corresponding author: Dr. Daniel M. Kobewka, Departments of Medicine and Epidemiology and Community Medicine, Ottawa Hospital, Civic Campus, 1053 Carling Avenue, Ottawa, ON K1Y 4E9, Canada, E-mail:


1. National Health Expenditure trends, 1975 to 2013. Ottawa, ON: Canadian Institute for Health Information, 2013. Search in Google Scholar

2. Health at a Glance 2013: OECD Indicators. Paris: OECD Publishing, 2013. Search in Google Scholar

3. Qaseem A, Alguire P, Dallas P, Feinberg LE, Fitzgerald FT, Horwitch C, et al. Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Ann Intern Med 2012;156:147–9. Search in Google Scholar

4. van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilization is? A systematic review of laboratory clinical audits. J Am Med Assoc 1998;280:9. Search in Google Scholar

5. Grossman RM. A review of physician cost-containment strategies for laboratory testing. Med Care 1983;21:783–802. Search in Google Scholar

6. Axt-Adam P, van der Wouden JC, van der Does E. Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care 1993;31:784–94. Search in Google Scholar

7. Young DW. Improving laboratory usage: a review. Postgrad Med 1988;64:283–9. Search in Google Scholar

8. Solomon DH, Hashimoto H, Daltroy L, Liang MH. Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework. J Am Med Assoc 1998;280:2020–7. Search in Google Scholar

9. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Br Med J 2009;339:b2535. Search in Google Scholar

10. Egger M, Smith GD, Altman DG. Principles of and procedures for systematic reviews. Systematic Reviews in Health Care: meta analysis in context, 2nd ed. London: BMJ Publishing Group, 2008:23–42. Search in Google Scholar

11. Cochrane Effective Practice and Organization of Care Review Group. Data collection checklist. Ottawa: Institute of Population Health, University of Ottawa, 2002. Search in Google Scholar

12. Al AM, Zidan A, Hamza AA, Al-Suwaidi R. Evaluation of laboratory investigations in accident and emergency department, Salmaniya Medical Complex. Bahrain Med Bull 2002;24:95–7. Search in Google Scholar

13. Amukele TK, Baird GS, Chandler WL. Reducing the use of coagulation test panels. Blood Coagul Fibrinolysis 2011;22:688–95. Search in Google Scholar

14. Archambault AJ, Arkin CF, Skelton T, Abel G, Gawoski J. Blood (ASH Annual Meeting Abstracts) 2011;118:Abstract 4774. Search in Google Scholar

15. Attali M, Barel Y, Somin M, Beilinson N, Shankman M, Ackerman A, et al. A cost-effective method for reducing the volume of laboratory tests in a university-associated teaching hospital. Mt Sinai J Med 2006;73:787–94. Search in Google Scholar

16. Barazzoni F, Grilli R, Amicosante AM, Brescianini S, Marca MA, Baggi M, et al. Impact of end user involvement in implementing guidelines on routine pre-operative tests. Int J Qual Health Care 2002;14:321–7. Search in Google Scholar

17. Bareford D, Hayling A. Inappropriate use of laboratory services: long term combined approach to modify request patterns. Br Med J 1990;301:1305–7. Search in Google Scholar

18. Baricchi R, Zini M, Nibali MG, Vezzosi W, Insegnante V, Manfuso C, et al. Using pathology-specific laboratory profiles in clinical pathology to reduce inappropriate test requesting: two completed audit cycles. BMC Health Serv Res 2012;12:187. Search in Google Scholar

19. Barie PS, Hydo LJ. Lessons learned: durability and progress of a program for ancillary cost reduction in surgical critical care. J Trauma 1997;43:590–6. Search in Google Scholar

20. Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma’luf N, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med 1997;157:2501–8. Search in Google Scholar

21. Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma’luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144–50. Search in Google Scholar

22. Berenholtz S, Pronovost P, Lipsett P, Dawson P, Dorman T. Assessing the effectiveness of critical pathways on reducing resource utilization in the surgical intensive care unit. Intens Care Med 2001;27:1029–36. Search in Google Scholar

23. Berwick DM, Coltin KL. Feedback reduces test use in a health maintenance organization. J Am Med Assoc 1986;255:1450–4. Search in Google Scholar

24. Boon-Falleur L, Sokal E, Peters M, Ketelslegers JM. A rule-based decision support application for laboratory investigations management. Proc Annu Symp Comput Appl Med Care 1995;314–8. Search in Google Scholar

25. Buckingham K, Russell I, Ross I, Gibson P, Paterson N. The effect of allowing clinical discretion in ordering biochemical tests. Evaluation by complementary methods. Int J Technol Assess 1994;10:695–700. Search in Google Scholar

26. Bunting PS, Van Walraven C. Effect of a controlled feedback intervention on laboratory test ordering by community physicians. Clin Chem 2004;50:321–6. Search in Google Scholar

27. Burnett L, Chesher D, Burnett JR. Optimizing the availability of ‘stat’ laboratory tests using Shewhart ‘C’ control charts. Ann Clin Biochem 2002;39:140–4. Search in Google Scholar

28. Calderon-Margalit R, Mor-Yosef S, Mayer M, Adler B, Shapira SC. An administrative intervention to improve the utilization of laboratory tests within a university hospital. Int J Qual Health Care 2005;17:243–8. Search in Google Scholar

29. Carson K, Murillo P, Runge A, Vogel DG, Baum LG. A test-ordering algorithm can reduce laboratory test use. Lab Med 1995;26:277–81. Search in Google Scholar

30. Carter E, Bennett BD. Reference test review by pathology house staff: a cost-containment strategy for the clinical laboratory. Clin Leader Manag Rev 2002;16:3–6. Search in Google Scholar

31. Chen P, Tanasijevic MJ, Schoenenberger RA, Fiskio J, Kuperman GJ, Bates DW. A computer-based intervention for improving the appropriateness of antiepileptic drug level monitoring. Am J Clin Pathol 2003;119:432–8. Search in Google Scholar

32. Chu K, Wagholikar A, Greenslade J. Reducing laboratory test ordering by limiting the range of tests that may be requested by interns and residents on pathology request forms. Emerg Med Australas 2012;24:21. Search in Google Scholar

33. Chu UB, Clevenger FW, Imami ER, Lampard SD, Frykberg ER, Tepas JJ. The impact of selective laboratory evaluation on utilization of laboratory resources and patient care in a level-I trauma center. Am J Surg 1996;172:558–3. Search in Google Scholar

34. Cohen DI, Jones P, Littenberg B, Neuhauser D. Does cost information availability reduce physician test usage? A randomized clinical trial with unexpected findings. Med Care 1982;20: 286–92. Search in Google Scholar

35. Davidoff F, Goodspeed R, Clive J. Changing test ordering behavior. A randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care 1989;27:45–58. Search in Google Scholar

36. Detsky AS, McLaughlin JR, Abrams HB, L’Abbe K, Markel FM. Do interns and residents order more tests than attending staff? Results of a house staff strike. Med Care 1986;24: 526–34. Search in Google Scholar

37. Dickinson JC. From process to policy: a generic prescription for test over-utilization in the emergency department. Fam Pract Res J 1987;7:12–21. Search in Google Scholar

38. Dixon RH, Laszlo J. Utilization of clinical chemistry services by medical house-staff. Arch Intern Med 1974;134:1064–7. Search in Google Scholar

39. Dowling PT, Alfonsi G, Brown MI, Culpepper L. An education program to reduce unnecessary laboratory tests by residents. Acad Med 1989;64:410–2. Search in Google Scholar

40. Durand-Zaleski I, Rymer JC, Roudot-Thoraval F, Revuz J, Rosa J. Reducing unnecessary laboratory use with new test request form: example of tumour markers. Lancet 1993;342:150–3. Search in Google Scholar

41. Durieux P, Ravaud P, Porcher R, Fulla Y, Manet C-S, Chaussade S. Long-term impact of a restrictive laboratory test ordering form on tumor marker prescriptions. Int J Technol Assess Health Care 2003;19:106–13. Search in Google Scholar

42. Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406–9. Search in Google Scholar

43. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ 1977;52:578–81. Search in Google Scholar

44. Emerson JF, Emerson SS. The impact of requisition design on laboratory utilization. Am J Clin Pathol 2001;116:879–84. Search in Google Scholar

45. Everett GD. Impact of supervision by medical teachers and in-patient test control programmes on the out-patient test utilization of residents. Med Educ 1985;19:138–42. Search in Google Scholar

46. Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Arch Intern Med 1983;143:942–4. Search in Google Scholar

47. Feldkamp CS, Carey JL. An algorithmic approach to thyroid function testing in a managed care setting: 3-Year experience. Am J Clin Pathol 1996;105:11–6. Search in Google Scholar

48. Feldman LS, Shihab HM, Thiemann D, Yeh HC, Ardolino M, Mandell S, et al. Impact of providing fee data on laboratory test ordering: a controlled clinical trial. JAMA Intern Med 2013;173:903–8. Search in Google Scholar

49. Finegan BA, Rashiq S, McAlister FA, O’Connor P. Selective ordering of preoperative investigations by anesthesiologists reduces the number and cost of tests. Can J Anaesth 2005;52:575–80. Search in Google Scholar

50. Fong BW. Common diagnostic tests: use and interpretation. Hawaii Med J 1989;48:7–9. Search in Google Scholar

51. Fowkes FG, Hall R, Jones JH, Scanlon MF, Elder GH, Hobbs DR, et al. Trial of strategy for reducing the use of laboratory tests. Br Med J 1986;292:883–5. Search in Google Scholar

52. Friedman KG, Rathod RH, Farias M, Graham D, Powell AJ, Fulton DR, et al. Resource utilization after introduction of a standardized clinical assessment and management plan. Congenit Heart Dis 2010;5:374–81. Search in Google Scholar

53. Froom P, Barak M. Cessation of dipstick urinalysis reflex testing and physician ordering behavior. Am J Clin Pathol 2012;137:486–9. Search in Google Scholar

54. Gama R, Nightingale PG, Broughton PM, Peters M, Bradby GV, Berg J, et al. Feedback of laboratory usage and cost data to clinicians: does it alter requesting behaviour? Ann Clin Biochem 1991;28(Pt 2):143–9. Search in Google Scholar

55. Gama R, Nightingale PG, Broughton PM, Peters M, Ratcliffe JG, Bradby GV, et al. Modifying the request behaviour of clinicians. J Clin Pathol 1992;45:248–9. Search in Google Scholar

56. Goddard K, Austin SJ. Appropriate regulation of routine laboratory testing can reduce the costs associated with patient stay in intensive care. Crit Care 2011;15:S47–8. Search in Google Scholar

57. Golden WE, Pappas AA, Lavender RC. Financial unbundling reduces outpatient laboratory use. Arch Intern Med 1987;147:1045–8. Search in Google Scholar

58. Gortmaker SL, Bickford AF, Mathewson HO, Dumbaugh K, Tirrell PC. A successful experiment to reduce unnecessary laboratory use in a community hospital. Med Care 1988;26:631–42. Search in Google Scholar

59. Grivell AR, Forgie HJ, Fraser CG, Berry MN. Effect of feedback to clinical staff of information on clinical biochemistry requesting patterns. Clin Chem 1981;27:1717–20. Search in Google Scholar

60. Groopman DS, Powers RD. Effect of ‘standard order’ deletion on emergency department coagulation profile use. Ann Emerg Med 1992;21:524–7. Search in Google Scholar

61. Gross PA, Van Antwerpen CL, Hess WA, Reilly KA. Use and abuse of blood cultures: program to limit use. Am J Infect Control 1988;16:114–7. Search in Google Scholar

62. Haschke-Becher E, Totzke U, Afazel S, Johansson T, Schwarz M, Ladurner G, et al. Clinical decision rules for the use of liquor diagnostics in hospitalized neurology patients reduced costs without affecting clinical outcomes. Int J Technol Assess Health Care 2009;25:208–13. Search in Google Scholar

63. Hutton HD, Drummond HS, Fryer AA. The rise and fall of C-reactive protein: managing demand within clinical biochemistry. Ann Clin Biochem 2009;46:155–8. Search in Google Scholar

64. Jelinek GA, Banham ND. Reducing the use of plain abdominal radiographs in an emergency department. Arch Emerg Med 1990;7:241–5. Search in Google Scholar

65. Kelly AM. Rationalising the ordering of blood cultures. Aust Health Rev 1998;21:245–50. Search in Google Scholar

66. Kumwilaisak K, Noto A, Schmidt UH, Beck CI, Crimi C, Lewandrowski K, et al. Effect of laboratory testing guidelines on the utilization of tests and order entries in a surgical intensive care unit. Crit Care Med 2008;36:2993–9. Search in Google Scholar

67. Larocque BJ, Maykut RJ. Implementation of guidelines for preoperative laboratory investigations in patients scheduled to undergo elective surgery. Can J Surg 1994;37:397–401. Search in Google Scholar

68. Levick DL, Stern G, Meyerhoefer CD, Levick A, Pucklavage D. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis 2013;13:43. Search in Google Scholar

69. Lewandrowski K, Bailey E, Dhanak E, Laposata M, Flood J. Mandatory laboratory consultation: how one hospital’s program reduced overuse of cardiac tests. Lab Med 1994;25:460–3. Search in Google Scholar

70. Mancuso CA. Impact of new guidelines on physicians’ ordering of preoperative tests. J Gen Intern Med 1999;14:166–72. Search in Google Scholar

71. Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med 1980;303:1330–6. Search in Google Scholar

72. Marton KI, Tul V, Sox HC. Modifying test-ordering behavior in the outpatient medical clinic. A controlled trial of two educational interventions. Arch Intern Med 1985;145:816–21. Search in Google Scholar

73. May TA, Clancy M, Critchfield J, Ebeling F, Enriquez A, Gallagher C, et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol 2006;126:200–6. Search in Google Scholar

74. Mehari SM, Havill JH, Montgomery C. A written guideline implementation can lead to reductions in laboratory testing in an intensive care unit. Anaesth Intensive Care 1997;25:33–7. Search in Google Scholar

75. Meng QH, Zhu S, Booth C, Stevens L, Bertsch B, Qureshi M, et al. Impact of the cardiac troponin testing algorithm on excessive and inappropriate troponin test requests. Am J Clin Pathol 2006;126:195–9. Search in Google Scholar

76. Merlani P, Garnerin P, Diby M, Ferring M, Ricou B. Quality improvement report: linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care. Br Med J 2001;323:620–4. Search in Google Scholar

77. Meyer L, Schmeler K, Wallbillich J, Urbauer D, Soliman P, Frumovitz M, et al. Clinical practice guidelines decrease unnecessary Pap tests in survivors of gynecologic malignancies. Gynecol Oncol 2011;120:S30–1. Search in Google Scholar

78. Miyakis S, Karamanof G, Liontos M, Mountokalakis TD. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med 2006;82:823–9. Search in Google Scholar

79. Morris AJ, Wilson ML, Reller LB. Application of rejection criteria for stool ovum and parasite examinations. J Clin Microbiol 1992;30:3213–6. Search in Google Scholar

80. Mutimer D, McCauley B, Nightingale P, Ryan M, Peters M, Neuberger J. Computerised protocols for laboratory investigation and their effect on use of medical time and resources. J Clin Pathol 1992;45:572–4. Search in Google Scholar

81. Nardella A, Pechet L, Snyder LM. Continuous improvement, quality control, and cost containment in clinical laboratory testing: effects of establishing and implementing guidelines for preoperative tests. Arch Pathol Lab Med 1995;119:518–22. Search in Google Scholar

82. Neilson EG, Johnson KB, Rosenbloom ST, Dupont WD, Talbert D, Giuse DA, et al. The impact of peer management on test-ordering behavior. Ann Intern Med 2004;141:196–204. Search in Google Scholar

83. Nelson III RB. Teaching technologic restraint. An evaluation of a single session. Eval Health Prof 1978;1:21–8. Search in Google Scholar

84. Nightingale PG, Peters M, Mutimer D, Neuberger JM. Effects of a computerised protocol management system on ordering of clinical tests. Qual Health Care 1994;3:23–8. Search in Google Scholar

85. Nirel N, Rosen B, Sharon A, Samuel H, Cohen AD. The impact of an integrated hospital-community medical information system on quality of care and medical service utilisation in primary-care clinics. Inform Health Soc Ca 2011;36:63–74. Search in Google Scholar

86. Pageler N, Longhurst C, Shin A, Adams E, Widen E, Barth R, et al. Using computerized provider order entry (CPOE) to promote sustained optimization of diagnostic radiologic service utilization. Crit Care Med 2009;37:A363. Search in Google Scholar

87. Pageler NM, Franzon D, Longhurst CA, Wood M, Shin AY, Adams ES, et al. Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization. Pediatr Crit Care Med 2013;14:413–9. Search in Google Scholar

88. Patel HD, Livsey SA, Swann RA, Bukhari SS. Can urine dipstick testing for urinary tract infection at point of care reduce laboratory workload? J Clin Pathol 2005;58:951–4. Search in Google Scholar

89. Pilon CS, Leathley M, London R, McLean S, Phang PT, Priestley R, et al. Practice guideline for arterial blood gas measurement in the intensive care unit decreases numbers and increases appropriateness of tests. Crit Care Med 1997;25:1308–13. Search in Google Scholar

90. Pop P, Winkens RA. A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. J R Coll Gen Pract 1989;39:507–8. Search in Google Scholar

91. Power LM, Thackray NM. Reduction of preoperative investigations with the introduction of an anaesthetist-led preoperative assessment clinic. Anaesth Intensive Care 1999;27:481–8. Search in Google Scholar

92. Powles L, Green J, Taylor E. Can redesigning a laboratory request form reduce the number of inappropriate PSA requests without compromising clinical outcome. Urology 2009;74:S148. Search in Google Scholar

93. Prat G, Lefevre M, Nowak E, Tonnelier J-M, Renault A, L’Her E, et al. Impact of clinical guidelines to improve appropriateness of laboratory tests and chest radiographs. Intens Care Med 2009;35:1047–53. Search in Google Scholar

94. Pysher TJ, Bach PR, Lowichik A, Petersen MD, Shields LH. Chemistry test ordering patterns after elimination of predefined multitest chemistry panels in a children’s hospital. Pediatr Dev Pathol 1999;2:446–53. Search in Google Scholar

95. Ramoska EA. Information sharing can reduce laboratory use by emergency physicians. Am J Emerg Med 1998;16:34–6. Search in Google Scholar

96. Ratnaike S, Hunt D, Eilermann LJ, Hazen R, Deam D. The investigation of chest pain: audit and intervention. Med J Australia 1993;159:666–71. Search in Google Scholar

97. Rhyne RL, Gehlbach SH. Effects of an educational feedback strategy on physician utilization of thyroid function panels. J Fam Practice 1979;8:1003–7. Search in Google Scholar

98. Roberts DE, Bell DD, Ostryzniuk T, Dobson K, Oppenheimer L, Martens D, et al. Eliminating needless testing in intensive care – an information-based team management approach. Crit Care Med 1993;21:1452–8. Search in Google Scholar

99. Rosenbloom ST, Chiu K-W, Byrne DW, Talbert DA, Neilson EG, Miller RA. Interventions to regulate ordering of serum magnesium levels: report of an unintended consequence of decision support. J Am Med Informat Assoc 2005;12:546–53. Search in Google Scholar

100. Santos IS, Bensenor IM, Machado JB, Fedeli LM, Lotufo PA. Intervention to reduce C-reactive protein determination requests for acute infections at an emergency department. Emerg Med J 2012;29:965–8. Search in Google Scholar

101. Seguin P, Bleichner J, Grolier J, Guillou Y, Malledant Y. Effects of price information on test ordering in an intensive care unit. Intens Care Med 2002;28:332–5. Search in Google Scholar

102. Shalev V, Chodick G, Heymann AD. Format change of a laboratory test order form affects physician behavior. Int J Med Inform 2009;78:639–44. Search in Google Scholar

103. Sorita A, Steinberg D, Leitman M, Luhan P, Burger A, Sivaprasad L. Impact of targeted individual feedback on provider patterns of stat laboratory test ordering frequency. J Hosp Med 2011;6:S135–6. Search in Google Scholar

104. Spray J, Brackbill M, Baker P. Evaluation of effect of electronic decision tree electrolyte replacement protocol on number of electrolyte labs ordered for patients in critical care. Crit Care Med 2012;40:248. Search in Google Scholar

105. Stafford RS. Feedback intervention to reduce routine electrocardiogram use in primary care. Am Heart J 2003;145:979–85. Search in Google Scholar

106. Sucov A, Bazarian JJ, deLahunta EA, Spillane L. Test ordering guidelines can alter ordering patterns in an academic emergency department. J Emerg Med 1999;17:391–7. Search in Google Scholar

107. Sussman E, Goodwin P, Rosen H. Administrative change and diagnostic test use: the effect of eliminating standing orders. Med Care 1984;22:569–72. Search in Google Scholar

108. Thompson RS, Kirz HL, Gold RA. Changes in physician behavior and cost savings associated with organizational recommendations on the use of ‘routine’ chest X rays and multichannel blood tests. Prev Med 1983;12:385–96. Search in Google Scholar

109. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results. Effects on outpatient testing. J Am Med Assoc 1988;259:1194–8. Search in Google Scholar

110. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322:1499–504. Search in Google Scholar

111. Toubert ME, Chevret S, Cassinat B, Schlageter MH, Beressi JP, Rain JD. From guidelines to hospital practice: reducing inappropriate ordering of thyroid hormone and antibody tests. Eur J Endocrinol 2000;142:605–10. Search in Google Scholar

112. van Wijk MA, van dL, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. a randomized trial. Ann Intern Med 2001;134:274–81. Search in Google Scholar

113. Vardy DA, Simon T, Limoni Y, Kuperman O, Rabzon I, Cohen A, et al. The impact of structured laboratory routines in computerized medical records in a primary care service setting. J Med Syst 2005;29:619–26. Search in Google Scholar

114. Verstappen WH, van der Weijden T, Sijbrandij J, Smeele I, Hermsen J, Grimshaw J, et al. Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. J Am Med Assoc 2003;289:2407–12. Search in Google Scholar

115. Wagholikar A, O’Dwyer J, Hansen D, Chu K. Observing effectiveness of pathology ordering controls in emergency departments. Stud Health Tech Informat 2011;168:172–8. Search in Google Scholar

116. Winkens RA, Pop P, Bugter-Maessen A, Grol RP, Kester AD, Beusmans GH, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet 1995;345:498–502. Search in Google Scholar

117. Winkens RA, Pop P, Grol RP, Kester AD, Knottnerus JA. Effect of feedback on test ordering behaviour of general practitioners. Br Med J 1992;304:1093–6. Search in Google Scholar

118. Wones RG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care 1987;25:78–82. Search in Google Scholar

119. Zaat JO. Attempts to change test ordering behavior in laboratories. Ann Clin Biochem 1992;17:8–14. Search in Google Scholar

120. Zaidi AK, Macone A, Goldmann AD. Impact of simple screening criteria on utilization of low-yield bacterial stool cultures in a Children’s Hospital. Pediatrics 1999;103: 1189–92. Search in Google Scholar

121. Zia HM, Singh G. Optimization of utilization of serum protein analysis: role of the electronic medical record in promoting consultation by pathology. Am J Clin Pathol 2013;139:793–7. Search in Google Scholar

122. Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, et al. Can computerized clinical decision support systems improve practitioners’ diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011;6:88. Search in Google Scholar

123. Buntinx F, Winkens R, Grol R, Knottnerus JA. Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract Res J 1993;10:219–28. Search in Google Scholar

124. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259. Search in Google Scholar

Supplemental Material

The online version of this article (DOI: 10.1515/cclm-2014-0778) offers supplementary material, available to authorized users.

Received: 2014-7-29
Accepted: 2014-8-25
Published Online: 2014-9-27
Published in Print: 2015-2-1

©2015 by De Gruyter