Daniel M. Kobewka , Paul E. Ronksley , Jennifer A. McKay , Alan J. Forster and Carl van Walraven

Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review

De Gruyter | Published online: September 27, 2014

Abstract

Laboratory and radiographic tests are often ordered unnecessarily. This excess testing has financial costs and is a burden on patients. We performed a systematic review to determine the effectiveness interventions to reduce test utilization by physicians. The MEDLINE and EMBASE databases were searched for the years 1946 through to September 2013 for English articles that had themes of test utilization and cost containment or optimization. Bibliographies of included papers were scanned to identify other potentially relevant studies. Our search resulted in 3236 articles of which 109 met the inclusion criteria of having an intervention aimed at reducing test utilization with results that could be expressed as a percent reduction in test use relative to the comparator. Each intervention was categorized into one or more non-exclusive category of education, audit and feedback, system based, or incentive or penalty. A rating of study quality was also performed. The percent reductions in test use ranged from a 99.7% reduction to a 27.7% increase in test use. Each category of intervention was effective in reducing test utilization. Heterogeneity between interventions, poor study quality, and limited time horizons makes generalizations difficult and calls into question the validity of results. Very few studies measure any patient safety or quality of care outcomes affected by reduced test use. There are numerous studies that use low investment strategies to reduce test utilization with one time changes in the ordering system. These low investment strategies are the most promising for achievable and durable reductions in inappropriate test use.

Introduction

Healthcare spending in has increased in both relative and absolute terms over the past several decades [1, 2]. This increase has been seen in many countries and will be unsustainable if the trend continues at the present rate [3]. Laboratory and radiographic testing is a promising target for reducing spending because many of the tests ordered by physicians are suspected to be unnecessary; in some studies, 95% of tests performed are inappropriate as judged by criteria of redundancy or their probability of adding value to patient care [4].

The perceived prevalence of inappropriate laboratory utilization has prompted many attempts to reduce test utilization. Several systematic and narrative reviews have been published on this topic [5–8]. The most recent review, published in 1998, used a behavioral framework to classify interventions and found that targeting multiple behavioral factors was more successful than targeting a single factor [8]. However, this review only dichotomized study outcomes by statistical significance, making it difficult to understand the utility of these interventions. Other reviews used categories of education, audit and feedback or other but were unable to make generalizations about which strategy is most effective because of differences within each category of intervention, a wide range of effects, and lack of a common measure [5, 6]. To date, no review has quantitatively compared the influence of various interventions on test utilization reduction. In addition, previous reviews have not examined investments required to implement these interventions. For these reasons, we performed a systematic review to determine the effectiveness of all interventions to reduce test utilization by physicians.

Methods

Data sources and searches

We performed a systematic review following a predetermined protocol in accordance with published guidelines for reporting of systematic reviews of randomized controlled trials [9]. We identified all potentially relevant articles published in English by searching Medline (1946 through September 2013) and Embase (1947 through September 2013). Searches were enhanced by systematically scanning bibliographies of identified articles and relevant review articles as well as articles deemed by PubMed to be related to the included studies. To search electronic databases, we used the strategy recommended for systematic reviews of interventional studies [10] and specified two comprehensive search themes (Online-Appendix 1). Theme 1 identified relevant terms related to laboratory utilization. Theme 2 related to optimization and cost containment. We then combined the two datasets using the Boolean operator ‘and’ and limited the intersection to human studies.

Study selection

Relevant articles were selected from the retrieved studies using a two-phase process. Abstracts were first reviewed for eligibility by three researchers (DK, PR, JM). All abstracts reporting on the effect of an intervention on laboratory utilization were selected for full text review. This initial stage was intentionally liberal; we discarded only abstracts that clearly were not reporting inventions aimed at reducing test utilization. Full text articles were then assessed by one reviewer (DK) with verification by two reviewers (PR, JM) to determine if the study met the specified inclusion criteria. Inclusion criteria included: 1) study population had to include physicians; 2) the aim of the intervention was the modification of test utilization; 3) a comparator arm (either standard care or no intervention) was needed; and 4) the study had to quantify laboratory test utilization with and without the intervention so that a percent change could be calculated. All study designs were considered within this review.

Intervention classification

Interventions used to effect physician laboratory ordering practices were categorized into one or more of the following non-exclusive categories:

  1. Educational interventions in which appropriate test ordering (including the distribution of guidelines) was taught to physicians;

  2. Audit and feedback interventions in which physicians were presented their test utilization compared to their previous utilization or peer utilization or the total costs of the tests they ordered;

  3. System-based interventions involving one-time, permanent changes to test ordering processes including: order form modifications; computer order entry systems with rules disallowing test ordering in specific circumstances; and clinical decision support systems (CDSS) in which an interactive computer system forces physicians to integrate previous knowledge about the patient and/or the medical literature into the test ordering process; and

  4. Incentive or penalty interventions in which physicians received rewards or punishments for certain test ordering practices. For each intervention, we also determined whether the targeted physicians were actively engaged during the intervention’s development [8].

Data extraction and quality assessment

For each study, we extracted information on publication year, country of origin, study design (randomized two-arm trials, before and after studies, prospective cohort), the healthcare providers targeted and the tests targeted for reduction. We recorded characteristics of the study intervention(s) along with its duration. We assessed study quality using an adapted version of the Effective Practice and Organization of Care (EPOC) Cochrane guidelines for interventional studies [11]. In particular, we assessed whether: patients and providers were similar across intervention and comparator groups; a randomized control group was used within the study, there was sufficient detail to describe the intervention, there was risk of contamination between the intervention and comparator groups, and whether a time-series analysis was performed. Lastly, we assessed whether studies measured any patient safety outcome that could be affected by reducing test utilization.

Statistical analysis

The unit of analysis was the intervention, with some studies having more than one intervention. The effectiveness of interventions was summarized using the percent relative reduction in test volume for the intervention group relative to the comparator group. This was calculated as:

(# of tests in comparator group) (# of tests in intervention group) (# of tests in comparator group)

Given the variability in the types of interventions, duration of interventions, targeted tests, and patient populations, we did not conduct a meta-analysis. Instead, a descriptive analysis of the relative reductions was performed using medians and inter-quartile ranges (IQR) to summarize effects across the various intervention types. We also documented when these reductions in a particular study were statistically significant. Results were stratified by categories of interventions (educational, system based, audit/feedback and incentive and penalty) and were visualized using box plots. Stratifications were then conducted by additional interventional factors within each category and by measures of study quality and statistical significance. The effect of physician involvement was also measured across interventions. Finally sensitivity analyses were conducted to assess the effect of interventions that targeted four or more tests.

In order to highlight the interventions with the best return on investment we included a narrative review of interventions with the largest reductions in test utilization as well as the most effective system-based interventions targeting four or more tests. All analyses were performed using Stata version 13.0 (Stata Corp., College Station, TX, USA).

Results

Our search generated 3236 unique citations. Following the initial screening and full-text review, 109 unique studies that included 119 interventions were included in the systematic review (Figure 1) [12–121]. Study characteristics and brief descriptions of each intervention are shown in Table 1. Ninety-three studies (85.3%) used before-after designs, 2 (1.8%) were non-randomized prospective trials, and 14 (12.8%) were randomized controlled trials. Studies were published between 1974 and 2013 with 61 studies (56.0%) performed in North America, 27 (24.8%) in Europe, 11 (10.1%) Australasia, and 10 (9.2%) in other regions. Studies used very limited time horizons with only 15 of 109 studies (13.8%) measuring intervention effects beyond 1 year.

Figure 1 Selection of articles in systematic review.The search was performed in September of 2013.

Figure 1

Selection of articles in systematic review.

The search was performed in September of 2013.

Table 1

Study characteristics and description of interventions.

Author Year Country Healthcare workers targeted Site Intervention Comparator Category of Intervention Targeted tests Relative reduction of targeted tests
E S A I
Amukele 2011 USA Physicians working in surgery, hematology, and internal medicine Single teaching hospital The order form was modified with some tests being bundled and others being unbundled. Physicians who were high users were educated about coagulation tests and an algorithm was distributed Before-after X X PT, PTT, TT and fibrinogen 77.8
Archambault 2011 USA All physicians Single teaching hospital ESR testing was no longer performed Before-after X ESR and CRP 47.5
Attali 2006 Israel Resident physicians Single teaching hospital Educational lecture about excessive and inappropriate testing; tests were unbundled on the request form; a senior physician gave feedback about ordering practices Before-after X X X All blood tests 34.5a
Barazzoni 2002 Switzerland All physicians seeing patients pre-operatively for elective procedures 6 community hospitals with surgery facilities All stakeholders were involved in a guideline creation process. The guidelines were then adopted as hospital policy Before-after X All pre-operative tests 26.1a
Bareford 1990 USA All medical staff: consultants and junior house staff Single community hospital Monthly usage statements were distributed to physicians. Guidelines on appropriate testing were distributed and some tests were canceled if inappropriate Before-after X X X CBC, ESR and PT 62.1
Baricchi 2012 Italy General practitioners 4 primary care clinics Testing algorithms for 7 common clinical scenarios were developed. Physicians were then educated about the algorithms and told to use them Before-after X X All blood tests 4.7
Barie 1995 USA All medical staff in the ICU Single teaching hospital Testing pathways were introduced and audit and feedback was done monthly Before-after X X All blood tests and chest radiographs 34.6
Bates 1999 USA All physicians Single teaching hospital Definitions of a redundant test were created. The computer order entry system created reminders when tests were ordered that were redundant Randomized trial X Electrolytes, urinalysis, digoxin, tobramycin, aminophyline, vancomycin, and gentamycin levels, urine, sputum and stool cultures, Clostridium difficile testing, fibrin split products 0.1
Bates 1997 USA All physicians Single teaching hospital Costs were displayed for each test that was ordered. Before-after X All blood tests and the 35 most common radiologic tests 4.1
Berenholtz 2001 USA Physicians working in a surgical ICU Single teaching hospital Clinical pathways were created for various surgical problems. Appropriate laboratory testing was suggested by the pathway Before-after X All laboratories included in the pathway –6.8
Berwick 1986 USA 35 internists Single health management organization Doctors met to discuss the use of particular tests and journal articles were distributed on the use of these tests Clinicians received individualized feedback on their rates of test ordering compared to their peers and their rank relative to their peers Physicians were ranked by the proportion of tests they ordered that were abnormal. Each physician was then given their rank Before-after X ESR, T4, serum glucose, heterophile antibody testing, ANA, SLE preparation, RF, liver enzymes, cholesterol levels, radiographs of peripheral bones and chest, ECGs 0.2
Before-after X 14.2a
Before-after X –4.1
Boon-Falleur 1995 Belgium Physicians working on liver a transplant unit Pediatric liver unit A rule based decision support application decided on the next day’s laboratory tests based on previous results Before-after X All blood tests 24.9
Buckingham 1994 Scotland All physicians All hospitals in a single city Justification for tests ordered had to be given. Specifics on how this was done are not given Before-after X All biochemical blood tests 1.4
Bunting 2004 Canada 200 physicians who ordered the most tests in 1997 Single province Feedback on test usage was given four times over a 2-year period Randomized controlled trial X All blood tests 7.8a
Burnett 1991 Australia All physicians requesting blood work from a single medical laboratory Single private medical laboratory All laboratory tests were unbundled so they had to be ordered individually Before-after X All blood tests 18.8a
Calderon-Margalit 2005 Israel All physicians Single teaching hospital Several interventions including mandatory consultation prior to ordering, education, restriction or elimination of various tests Before-after X X All blood tests 19.0a
Carson 1995 USA All physicians Single teaching hospital An algorithm that automatically restricted fibrin degradation product testing and instead used D-dimer testing for suspected DIC Before-after X D-Dimer and fibrin degradation products 48.3a
Carter 2002 USA Physicians sending specimens to the lab Single teaching hospital All test requests were reviewed by a more senior physician Before-after X All laboratories that cost >$75 USD 23.0
Chen 2003 USA All Physicians ordering selected tests Single teaching hospital A reminder on the half-life of various anti-epileptic drugs was shown on the computer when repeat levels were requested in a short time period Before-after X X Anti epileptic drug levels 23.4a
Chu 2012 Australia Medical students and residents Single teaching hospital Selected tests were restricted and required a senior staff member to sign off on them prior to being ordered Before-after X Coagulation tests, thyroid function tests, ESR and d-dimer 7.5a
Chu 1996 USA Physicians taking care of trauma patients Single teaching hospital A liberal admission algorithm with extensive testing was compared to physicians only ordering tests they thought necessary Before-after X Hematology, chemistry and coagulation profile, urinalysis, blood gas, blood alcohol level, type and screen 68.3
Cohen 1982 USA All physicians working on the teaching units 4 clinical teaching units at a single teaching hospital The cost of all imaging tests ordered were given to each team weekly Randomized controlled trial X CXR, CT scans, US and liver scans 12.9
The cost of all blood tests ordered were given to each team weekly Randomized controlled trial X All blood tests 24.7a
Davidoff 1989 USA 24 medical interns entering a university training program Single teaching hospital Lectures on probability and test characteristics were compared to placebo lectures on economics and cost control Randomized controlled trial X All blood tests 16.5a
Detsky 1986 Canada All attending physicians 20 teaching hospitals in a single province Residents and interns went on strike for 7 days in 1980 Before-after X All blood tests 8.3a
Dickinson 1987 USA All physicians in the institution Single teaching hospital The number of unnecessary tests was measured and shown to department heads Before-after X All blood tests 24.5a
Dixon 1974 USA Interns working on the wards Single teaching hospital Interns and residents were limited to 8 tests each Before-after X All blood chemistry tests 66.7a
Dowling 1989 USA Residents in a family health center Single family health center Lectures and guidelines on appropriate testing were distributed. Audit and feedback of test usage was performed repeatedly Before-after X X CBC and TSH 47.5a
Durand-Zaleski 1993 France All physicians at the hospital Single teaching hospital The laboratory requisition was changed so that the purpose of each tumor marker was included on the requisition Before-after X Tumor markers 23.9a
Durieux 2003 France All physicians at the hospital Single teaching hospital The laboratory requisition was changed so that the purpose of each tumor marker was included on the requisition. The purpose for each test, i.e., screening, follow-up or diagnosis was explained on the requisition Before-after X X CEA, alpha feto-protein, CA19-9 56.5a
Eccles 2001 England and Scotland General Practitioners Multiple primary care clinics Guidelines on when to order lumbar spine and knee X-rays were circulated along with feedback on the number of requests to each physician Randomized controlled trial X X Knee and lumbar spine XR 40.8
Eisenberg 1977 USA Resident physicians Single community hospital There was an education program with a weekly lecture; results from a single audit on utility of testing were circulated along with a memorandum from the medical director asking for careful consideration of test usage Before-after X PTT 42.1a
Emerson 2001 USA All physicians working at the medical center Single teaching hospital Laboratory requisition was redesigned with some tests unbundled and cascades implemented for thyroid function and anemia Before-after X All blood tests 4.7a
Everett 1985 USA First year residents Single teaching hospital Residents were taught about laboratory utilization using specific cases Randomized controlled trial X All blood tests and microbiology –12.3
Everett 1983 USA First year residents training in internal medicine Single teaching hospital Audit and feedback of costs was performed weekly, cost education news letters were distributed and costs of tests were put in patient charts. There were one-on-one meetings of faculty with residents to discuss costs Randomized controlled trial X X All blood tests 15.1a
Feldkamp 1996 USA All physicians in health center Single teaching hospital and 22 satellite clinics Algorithms were developed for thyroid function testing and implemented by the laboratory Before-after X Thyroid function testing 21.6
Feldman 2013 USA All providers who order laboratory tests through a computerized order entry system Single teaching hospital Cost of each test was displayed when it was ordered Randomized controlled trial X X 61 diagnostic blood tests 8.6a
Finegan 2005 Canada All physician trainees in a hospital Single teaching hospital Each anesthetist selected tests to be performed preoperatively at their discretion. The comparator was surgery specific protocols that were being used Before-after All blood tests 3.6
Fong 2008 Australia All physicians working in the emergency room Single teaching hospital A clinical decision rule was used instead of physician discretion Before-after X CT head –27.7a
Fowkes 1986 United Kingdom All residents working on the medical unit Single teaching hospital A guideline on testing was distributed and weekly meetings were held with medical staff to discuss their use of tests in the previous week Non-randomized controlled trial X X All blood tests 63.4a
Friedman 2010 USA Physicians taking care of children post procedure Single teaching hospital A set of standard assessment and management plans were developed and implemented after certain procedures Before-after X All blood tests and imaging 15.6
Froom 2012 Israel All physicians Regional laboratory serving a single Health Management Organization Urine microscopy was no longer done automatically instead the physician had to specifically request it Before-after X Urine for dip and microscopy 99.7a
Gama 1991 United Kingdom General internists Single community hospital Data on personal and peer expenditure of laboratory resources was given to physicians monthly for 12 months Randomized controlled trial X Clinical chemistry 24.6a
Gama 1992 United Kingdom Physicians taking care of inpatients Single community hospital Data on personal and peer clinical chemistry expenditure per patient was provided to physicians monthly for 12 months Before-after X Clinical chemistry and hematology 27.0a
Goddard 2011 Ireland All physicians working in the ICU Single teaching hospital All recurring orders for blood work were discontinued; instead each test had to be ordered as a single occurrence Before-after CBC, urea, electrolytes, coagulation, LFTs, magnesium, Ca, PO4, albumin, CRP 33.1
Golden 1987 USA All physicians Outpatient department of a single teaching hospital The cost of tests billed to the patient was changed so that ordering individual tests was cheaper than ordering the whole panel Before-after X All blood chemistry test 33.5a
Gortmaker 1988 USA All physicians working in the hospital Single community hospital Nine staff meetings were held to discuss cost issues. Data on excess usage was sent to all doctors. All physicians reached a consensus on when each test was considered inappropriate Before-after X BUN, CBC, chemistry profile, CK, creatinine, electrolytes, fasting and random blood sugar, hematocrit, PT, SGOT, urinalysis 14.0a
Grivell 1981 Australia All physicians caring for inpatients Single teaching hospital Physicians received a report every 4 weeks on the type and number of tests ordered. Physicians were also shown where they rank in comparison to their peers Before-after X All blood tests 0.0
Groopman 1992 USA All medical staff ordering laboratories in the ER Single teaching hospital Tests were taken off the standard requisition and had to be written in by hand Before-after X PT/PTT 61.6a
Gross 1988 USA All physicians taking care of inpatients Single teaching hospital Guidelines on when to order blood cultures were distributed Before-after X Blood cultures 75.0a
Haschke-Becher 2009 Austria No doctors were targeted, it was a laboratory intervention Single teaching hospital A decision rule was created to determine when the laboratory would process liquor diagnostics Before-after X Liquor diagnostics 28.6a
Hutton 2009 United Kingdom All emergency room and medical unit physicians Single teaching hospital Requests for CRP from the ER had to be approved by a senior physician. Junior staff were told to limit CRP use. Lastly repeat testing within 24 h was disallowed Before-after X X CRP 85.0a
Jelinek 1990 Australia All physicians Single teaching hospital Guidelines on when abdominal films could be ordered were circulated and all requests had to be run by a senior physician Before-after X X Abdominal plain films 52.2
Kelly 1998 Australia All physicians in the ER Single community hospital Guidelines on who should receive a blood culture were distributed Before-after X Blood cultures 53.0
Kumwilaisak 2008 USA All physicians working in the surgical ICU Single teaching hospital Guidelines regarding when certain blood tests should be used were introduced at a staff meeting and sent out by email. A session was repeated for new residents every month Before-after X All blood tests 20.8a
Larocque 1994 Canada Surgeons and surgical house staff Single teaching hospital Guidelines on which tests were appropriate for different medical conditions were posted on the wards and distributed as pocket cards Before-after X Pre-op screening tests including blood work, CXR and ECGs 10.1a
Levick 2013 USA Clinicians ordering the BNP test A health network including 2 hospitals and several community health centers Computer order entry system displayed a warning when a repeat BNP was being ordered Before-after X BNP 22.2a
Lewandrowski 1994 USA All physicians working at the hospital Single teaching hospital A guideline on appropriate testing was developed. All requests were then reviewed by laboratory staff and the ordering physician was contacted if the request was not in accordance with the guideline Before-after X X LD-isozymes 99.6
Mancuso 1999 USA All physicians at the hospital Single specialized orthopedic surgery hospital Guidelines were published suggesting selective test ordering preoperatively Before-after X Pre-operative blood work, ECG, CXR, 30.0a
Martin 1980 USA Residents and medical students Single teaching hospital There was a 1 h lecture on laboratory costs. Money was given to the residents if their laboratory use decreased Non-randomized controlled trial X X All radiologic and blood tests 5.8a
There was a 1-h lecture on laboratory costs. Senior physicians then met weekly with residents, they reviewed cases and suggested changes in test ordering practices. X X –17.8a
Marton 1985 USA Residents and medical students Single teaching hospital A manual about rational test ordering was given to the trainees Randomized controlled trial X All blood tests 18.2
Feedback on laboratory usage and costs incurred was shown to trainees along with their usage compared with peers X 33.3
A manual about rational test ordering was given to the trainees. They also received feedback on laboratory usage, costs incurred and their usage compared with peers X X 40.9a
May 2006 USA All docs working on inpatient service Single teaching hospital Duplicate orders were deleted and all blood work orders expired in 24 h. Blood work would only be drawn every 4, 6 or 12 h. Future orders could only be for a single occurrence Before-after X All blood tests 11.5a
Mehari 1997 New Zealand Physicians working in ICU Single teaching hospital A guideline was placed on each patients chart stating when it is appropriate to do each test Before-after X CBC, coagulation tests, Cr, Na, K, urea, calcium, glucose, Mg, PO4, ABGs 21.2
Meng 2006 Canada All physicians 3 teaching hospitals A policy was created regarding who is allowed to order troponin testing and how many repeats can be ordered Before-after X TnI 2.9
Merlani 2001 Switzerland Doctors residents and nurses working in the intensive care unit Single teaching hospital A guideline on when arterial blood gases can be done was circulated Before-after X X ABG 41.5a
Meyer 2010 USA Gynecologic oncologists and other physicians Single teaching hospital A multidisciplinary team created a guideline and then created a communication plan to disseminate it Before-after X Pap tests 31.2a
Miyakis 2006 Greece All physicians working at the hospital Single teaching hospital The results of an audit on test use were presented along with strategies for reducing utilization Before-after X X 25 laboratories hematology and chemistry tests 13.5a
Morris 1992 USA All physicians working at the hospital Single teaching hospital Only one stool sample for parasites was accepted by the laboratory instead of three Before-after X Stool for ova and parasite 33.3
Mutimer 1992 United Kingdom All physicians working in the liver unit Liver unit at a single teaching hospital Laboratory tests to be ordered for the next day were determined by protocol based on patient characteristics and previous testing Before-after X Hematology, biochemistry, immunology, microbiology and cross matching 9.3a
Nardella 1995 USA Anesthetists and surgeons Single teaching hospital Guidelines about appropriate pre-operative testing were distributed and discussed at staff meetings Before-after X CBC, PT, PTT, bleeding time, BUN, Cr, Glucose ALT AST, electrolytes 61.8a
Neilson 2004 USA Residents, physicians and nurse practitioners who used computer order entry Single teaching hospital Recurring tests had to be confirmed daily, many tests were limited to a single occurrence and previous results were displayed when ordering tests Before-after X All blood tests and imaging 64.2a
Nelson 1978 USA All physicians who work at the hospital Single community hospital There was a 20-min grand rounds presentation on B12 testing. Suggestions were given on when to order this test Before-after X B12 and folate 45.3
Nightingale 1994 United Kingdom Residents working on the liver unit Liver unit at a single teaching hospital A computerized protocol was created that suggested tests for the next day based on current laboratory values and patient characteristics Before-after X All blood tests 23.7
Nirel 2011 Israel All physicians working in the health system Primary care clinics An online medical record was introduced so that old laboratory tests could be accessed Before-after X All blood tests 3.6a
An online medical record was introduced so that old radiology tests could be accessed Before-after X All radiology tests 12.3
Pageler 2013 USA Physicians working in pediatric ICU Single teaching hospital A rule preventing repeating testing was created in the computer order entry system. It had to be overridden to order repeat testing Before-after X CBC, chemistry and coagulation 37.7a
Pageler 2009 USA Not reported Single teaching hospital Recurring orders for CXR were no longer allowed and instead had to be re-ordered daily Before-after X Chest XR 10.0
Patel 2005 United Kingdom All physicians working in the hospital Single teaching hospital Urine was not cultured if the urinalysis was normal Before-after X Urine culture 4.0
Pilon 1997 Canada Physicians and nurses working in the ICU Single teaching hospital A guideline about when ABGs should be used was created. It was distributed on pocket cards and taught during education sessions Before-after X ABGs 36.7a
Pop 1989 Netherland General practitioners All physicians using the same laboratory in one city Two times per year physicians were given a report of tests they had ordered and whether they were appropriate or not. Physicians were also informed of their redundant tests Before-after X BUN, Cr, liver enzymes, WBC, Hgb, ESR, glucose 36.2
Power 1999 Australia Surgical Residents in pre-op clinic Single teaching hospital Published guidelines on test ordering were distributed. A maximum number of tests per patient was instituted and test ordering was reviewed by a consultant Before-after X X X CBC, coagulation tests, urea, LFTs, CXR, ECG 37.1
Powles 2009 United Kingdom All physicians in region using the university laboratory for PSA testing Single teaching hospital Prostate-specific antigen was taken off the request form and had to be written in by hand Before-after X PSA 17.3a
Prat 2009 France All physicians working in ICU Single teaching hospital A guideline was created on when to perform each of the common blood tests. The cost of each test was also shown on the guideline. Teaching was done on how to reduce test utilization Before-after X CXR and daily routine BW 49.0a
Pysher 1999 USA All physicians ordering tests at this laboratory Single teaching hospital Predefined chemistry panels were eliminated from the requisition forms. Instead each test had to be ordered separately Before-after X All blood chemistry tests 32.7a
Ramoska 1998 USA Physicians and residents working in the emergency department Single teaching hospital Graphs of laboratory utilization and costs of laboratory tests at the hospital was displayed on a bulletin board Before-after X All blood tests 17.8a
Ratnaike 1993 Australia All physicians working in the cardiac care unit Single teaching hospital Guidelines were created by the hospital and issued to the cardiology unit. No further details about the guidelines were given Before-after X All blood tests 57.2
Rhyne 1979 USA Family doctors The family medicine program associated with one university A chart audit of test ordering was done and the results were presented to medical staff Before-after X X Thyroid function tests 36.4a
Roberts 1993 Canada All physicians working at the hospital Single teaching hospital Serum osmolality was removed from the requisition and blood count differential was not done if white cell count was normal. A protocol was used to determine when electrolytes should be checked. Daily CXRs were no longer allowed Before-after X X Blood gases, serum osmolality, CBC, Na, K, glucose, CXR, and ECG 18.1a
Rosenbloom 2005 USA All physicians working on selected inpatient units Single teaching hospital Magnesium could only be ordered once with no recurrence and a screen displaying old results and education on when magnesium should be checked had to be viewed before the order could be entered Before-after X X Magnesium 55.2a
Santos 2012 Brazil All residents and physicians working at the hospital Single teaching hospital A meeting was held to discuss evidence for certain tests and the costs of the tests. Patterns of testing and appropriateness of various tests was discussed Before-after X CRP 48.4
Seguin 2002 France All physicians working in ICU Single teaching hospital Costs for each test was shown on the order form Before-after X Electrolytes, CBC, ABG, LFTs, coagulation panel, CXR 18.9
Shalev 2009 Israel All primary care physicians in Israel Single health service provider 27 tests were removed from the requisition and 2 were added Before-after X All blood test 4.3a
Sorita 2011 USA Residents training in medicine and surgery Single teaching hospital There was a 1 h teaching session regarding when to order STAT blood tests and results of an audit were presented. The highest users of STAT tests were given individual feedback Before-after X X All stat laboratory tests 21.0a
Spray 2012 USA Physicians taking care of inpatients Single community hospital An electronic protocol for electrolyte replacement and retesting was instituted Before-after X Magnesium, phosphorous and ionized calcium 32.0a
Stafford 2003 USA 117 primary care providers Primary care practices affiliated with a single teaching hospital Audits of individual and group ECG usages patterns were mailed to physicians Before-after X ECG 28.3
Sucov 1998 USA All physicians working in the hospital Single teaching hospital Guidelines were created regarding what tests to order for which patient groups. The guideline was then rolled out with a series of educational sessions. Each department received feedback on its ordering practices but there was no individual feedback Before-after X X All blood tests 22.0a
Sussman 1984 USA All physicians working in the hospital Single teaching hospital All recurring orders were eliminated Before-after X CBC, electrolytes, CXR, chemistry panel, FBG, electrocardiogram, PT 20.8
Thompson 1983 USA All physicians part of the health plan Single pre-paid health plan A recommendation was made against screening healthy adults with blood tests and CXR. An educational campaign was then implemented Before-after X All blood chemistry tests and CXR 60.0a
Tierney 1987 USA Residents and staff working in the clinic Single university associated general medicine clinic All previous results for a test were shown when using the computer order entry system. An option to cancel the current test was then available Randomized controlled trial X All laboratory and imaging and ECGs 8.9a
Tierney 1990 USA Physicians working in the clinic Single university associated general medicine clinic Cost to the patient of ordering each tests was displayed before the order is confirmed Randomized controlled trial X All blood tests 17.6a
Toubert 2000 France All physicians in the hospital Single teaching hospital Educational material on thyroid disease was distributed. Physicians were told that any test other than TSH required justification. Tests were still processed even if no justification was provided Before-after X TSH, T3, T4, 39.6
van Wijk 2001 Netherlands GPs working in practices with electronic medical records Single health region Computerized decision support suggested appropriate tests based on basic clinical information Randomized controlled trial X All blood tests 20.3a
Vardy 2005 Israel All physicians ordering tests through the HMO Single health management organization Lists of appropriate laboratory tests for each clinical condition were created and physicians were educated about them. Each list was then put into the computer order entry system Before-after X X All blood tests 4.2
Verstappen 2003 Netherlands Physicians – not specified 5 health regions in the country Each physician received feedback on their ordering practices regarding upper and lower abdominal pain and cardiovascular conditions compared with their peers. The physicians then met together to discuss how to change their ordering practices Randomized controlled trial X X Biochemistry and hematology tests, ECGs, sigmoidoscopy and imaging 12.0a
Each physician received feedback on their ordering practices regarding COPD, asthma, degenerative joint complaints and non-specific complaints compared with their peers. The physicians then met together to discuss how to change their ordering practices Randomized controlled trial X X Biochemistry and hematology tests, ECGs, sigmoidoscopy and imaging 8.0
Wagholikar 2011 Australia All physicians working in the emergency department Single teaching hospital Approval from a senior physician was required to order tests in certain circumstances Before-after X CBC and ESR 11.8a
Winkens 1995 Netherlands Family physicians Single regional medical laboratory Individualized feedback was sent to each physician from a respected internal medicine physician. The feedback included alternative testing strategies Before-after X Cervical smear, ECG, endoscopy, allergy tests, radiographic and ultrasound tests 5.5a
Winkens 1992 Netherlands Family physicians Single regional medical laboratory Biannual reports were sent on the number and rationality of laboratory testing compared to peers Before-after X X 46 blood tests 25.0a
Wones 1987 USA First year internal medicine residents Single teaching hospital The group was simply informed that laboratory utilization was being monitored Randomized controlled trial X All laboratory tests –11.4
This group received individual feedback on laboratory usage and costs every 2 weeks X All laboratory tests –13.15
This group received individual feedback on laboratory usage and costs every 2 weeks along with group data for comparison X All laboratory tests –7.27
Zaat 1992 Netherlands General Practitioners Two health regions The laboratory requisition was modified so that it only had 15 tests on it. All other tests had to be written in Before-after X All hematological and chemistry tests 18.2a
Zaidi 1999 USA All physicians sending samples to the hospital lab Single teaching hospital Guidelines were created from local data and shared with all medical staff. After this all samples not meeting the criteria were rejected Before-after X X Stool culture 30.9
Zia 2013 USA All Physicians at the hospital Single teaching hospital The test was removed from the laboratory requisition but could still be ordered if written in by hand Before-after X Serum protein electrophoresis and immunofixation 29.6

aDenotes studies in which a test of statistical significance was performed and found to be significant by the studies’ authors. A, Audit and feedback; E, Educational; I, Incentive or penalty; S, System-based.

Study quality

Study quality is summarized in Online-Appendix 2. Of the 109 studies, 14 (12.8%) had a randomized design and 37 (33.9%) provided evidence that patients were similar between groups. Time-series analysis was performed in only 11 (10.1%) studies. Interventions were described adequately to be reproducible in 88 (80.7%) studies. Test utilization was reported with the patient or the encounter as the unit of analysis in 79/109 (72.5%) studies with the rest using the physician or the group of physicians working that location as the unit of analysis.

Intervention components

Of 119 interventions, 51 (42.9%) had an educational component, 54 (45.4%) had a system-based component, 41 (34.5%) had an audit feedback component, with only one (0.8%) using incentive or penalty. Physicians were involved in the development of interventions in only 17 (14.3%) interventions. Thirty-two (26.9%) interventions were multifaceted, being classified into more than one of the four main intervention categories. A measure of patient safety or quality of care was included in 13 (11.9%) studies.

Intervention effects

The percent relative reductions in laboratory utilization ranged from a +99.7% (a reduction in the test volumes) to –27.7% (an increase in test volume). Table 2 displays the percent relative reductions in test volume for each intervention and their sub-categories. Interventions with an educational component had the highest median relative reduction in test volume at 34.5% (IQR 16.5–49.0) while audit feedback or system-based interventions had a relative reduction of 22.0% (8.6–34.6) and 22.2% (3.6–68.3), respectively (Figure 2). The single incentive and penalty intervention had a 5.8% relative reduction.

Table 2

Percent relative reductions in test utilization by intervention type.a

Number of interventions Median RR (IQR) Minimum RR Maximum RR
Educational interventions 51 34.5 (16.5–49.0) –12.3 99.6
 –Δ between intervention and control statistically significant 33 31.2 (18.1–47.5) 4.7 85.0
 –Δ between intervention and control statistically insignificant 18 38.3 (8.0–53.0) –12.3 99.6
Exclusively education intervention 22 30.6 (16.5–48.5) –12.3 75.0
Exclusively guideline distribution 13 30.0 (20.8–53.0) 4.7 99.6
System-based interventions 54 22.2 (3.6–68.3) 0.1 99.7
 –Δ between intervention and control statistically significant 36 19.6 (10.4–36.1) 3.6 99.7
 –Δ between intervention and control statistically insignificant 18 24.8 (10.0–37.1) 0.1 77.8
Exclusively system-based intervention 37 21.6 (10.0–32.7) 0.1 99.7
Exclusively clinical decision support 10 22.8 (9.3–23.9) 4.7 85.0
Audit feedback interventions 41 22.0 (8.6–34.6) –13.1 99.6
 –Δ between intervention and control statistically significant 24 23.2 (13.8–34.5) 5.5 63.4
 –Δ between intervention and control statistically insignificant 17 18.9 (0–36.2) –13.1 99.6
Exclusively audit and feedback intervention 20 18.4 (2.1–24.8) –13.1 36.2
Exclusively cost display 12 18.4 (2.1–28.8) 5.5 99.6
Physicians targeted were involved in creating and implementing the intervention 17 23.0 (8.0–39.6) 3.6 99.6
Physicians targeted were not involved in creating and implementing the intervention 103 22.2 (10.1–36.7) –27.7 99.7

aCategories of interventions are not mutually exclusive unless stated.

Figure 2 Box plots of relative reductions in laboratory utilization stratified by intervention type.1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Figure 2

Box plots of relative reductions in laboratory utilization stratified by intervention type.

1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Thirty-three (27.7%) of the interventions targeted three or fewer tests, while 86 (72.3%) targeted four or more tests. The median relative reduction for interventions that targeted three or fewer tests was 40.2% (IQR 23.7–52.8) while that targeting four or more tests was 18.8% (IQR 8.2–28.5) (Figure 3). Thirty interventions (25.2%) used multiple interventions; these multifaceted interventions had larger reductions in test use with a median relative reduction in test volume of 32.7% (IQR 15.1–47.5) versus 21.4% (IQR 9.5–33.3) for interventions that were classified into a single category. Interventions for which physicians were engaged during the intervention development had similar median relative reductions as interventions without physician input.

Figure 3 Box plots of relative reductions in laboratory utilization stratified by intervention type among interventions targeting four or more tests.1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Figure 3

Box plots of relative reductions in laboratory utilization stratified by intervention type among interventions targeting four or more tests.

1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Finally, we found important differences in the relative reductions across quality indicators (Table 3). Most notably, the effect of interventions was less if a study reported a reduction in test utilization per patient rather than reduction per institution or physician group and studies with a concurrent control group had smaller reductions. Unexpectedly there were larger relative reductions in studies that had a risk of contamination between the experimental and control groups.

Table 3

General descriptions of study quality (n=109).

Number of studies Median relative reduction (IQR)
Where patients similar between groups?
 Yes 37 19.0 (8.6–37.7)
 No 72 24.8 (14.0–41.5)
Were those ordering the tests similar between groups?
 Yes 103 23.8 (11.9–40.2)
 No 6 11.7 (4.2–25.0)
Was there a concurrent control group?
 Yes 27 16.5 (5.8–27.0)
 No 82 24.9 (14.0–47.5)
Was the intervention described adequately enough to be replicated?
 Yes 88 23.5 (12.4–38.6)
 No 21 20.8 (4.7–36.7)
Was there a risk of contamination between experimental and control groups?
 Yes 89 24.9 (13.5–40.8)
 No 20 15.8 (4.8–29.6)
Were the results reported per patient instead of per institution or per physician?
 Yes 79 19.9 (9.1–35.4)
 No 30 33.4 (23.4–48.3)
Was a time-series analysis conducted?
 Yes 11 26.1 (14.0–55.2)
 No 98 23.5 (10.1–37.1)

Interventions with the largest effects

We found extensive heterogeneity in the components that made up each intervention, how interventions were implemented, the study setting, and the tests that were targeted for reduction. Meaningful generalizations across intervention types is therefore difficult; so the interventions with the largest relative reductions in test volume are described here (Table 1). The studies by Froom et al. and Lewandrowski et al. attained the largest relative reductions with both targeting a single test. Froom et al. changed laboratory policy so that urine microscopy was done only if specifically requested by the physician; this resulted in a 99.7% reduction in test use [53]. Lewandrowski et al. distributed guidelines stating when it was appropriate to order LD-isozymes with a laboratory physician notifying the ordering physician when the LD-isozyme request was not compliant with the guideline [69]. This combination of education and feedback led to a 99.6% reduction in LD-isozymes being ordered. The two largest relative reductions among interventions that targeted four or more tests were by Chu et al. and Dixon et al. Chu et al. used a protocol driven algorithm for admitting trauma patients and found a 68.3% reduction in the number of tests ordered per patient [33]. Dixon et al. limited the number of tests that interns could order to eight per patient per day, resulting in a 66.7% reduction in the number of tests ordered per patient [38].

The most successful interventions targeting four or more tests that were purely system based were the study by Chu et al, discussed above, and a study by Neilson et al. Neilson et al. sequentially implemented a number of system-based interventions using a computerized order entry system including: the display of previous laboratory results to physicians ordering tests; the prevention of recurring orders in various circumstances; and the unbundling of tests so that each component had to be ordered individually [82]. This package of interventions resulted in a 64.2% relative reduction in the tests targeted.

Discussion

We summarized 109 studies evaluating the effect of interventions aimed at reducing test utilization by calculating relative reductions to compare interventions. We found that all intervention categories (including education, system based, audit feedback, and incentive penalty) can reduce test utilization. Even though educational interventions had the largest relative reduction in test utilization, the inter-quartile ranges were large and overlapping. The greatest difference in our stratified analysis was between interventions that targeted less than four tests, which had much larger relative reductions than interventions that targeted four or more tests. Consistent with past reviews, we found that interventions using multiple strategies were more effective [8]. Even though all intervention types seem to be effective, we found a large range of effects and much heterogeneity between studies. Many studies did not use appropriate statistical techniques and follow-up was often short. These shortcomings call into question the sustainability and generalizability of many interventions in our review.

Sustainability

Although interventions with an educational component had the greatest median relative reductions, their long-term sustainability is questionable. The resources needed to perpetuate educational strategies (i.e., lectures or individual teaching sessions) are significant and probably need to be continuous to maintain an effect, especially in teaching hospitals where ordering physicians are constantly changing. Even in studies where the educational strategy was low effort (i.e., the distribution of a guideline), multiple reminders and repeated meetings were necessary to maintain the change in behavior [89, 97, 106, 111]. There was only one purely educational intervention that reported results from >1 year after the intervention began but this study did not use time series analysis to determine if utilization returned to baseline during this time [70]. With all but one purely educational interventions having a time horizon of <1 year there is no evidence in our review that educational strategies have long-term sustainability. In contrast there are several examples of system-based interventions which by definition require few resources to maintain, that are shown to maintain their effect for >1 year [14, 47, 53]. The ratio of resource input to test reduction is not measured in the included studies but is an important consideration when deciding to implement a particular intervention.

Generalizability

Many of the interventions have questionable generalizability. There is a wide range of relative reductions among included studies due to heterogeneity at every level. There are differences in how an invention type is implemented, who is target by the intervention, and the pre-existing ordering practices among the targeted physicians. Due to the heterogeneity, similar results cannot be expected even if an intervention is perfectly replicated. Some of the large reductions (>70%) suggest that anomalous ordering is being targeted which further calls into question the generalizability of results. Lastly many of the studies target physicians in training who would likely respond differently than physicians in independent practice.

Quality

The quality of included studies was poor. As expected studies with lower quality had higher relative reductions suggesting that poor quality studies were over estimating the effects of interventions. Surprisingly, very few of the before and after studies used time-series analysis or autocorrelation to analyze results, bringing into question the validity of results.

Limitations

This review has several limitations. Our search strategy used only studies published in English ignoring potentially important studies published in other languages and only searched two databases. Another limitation is that the heterogeneity between interventions makes generalizations about effective strategies difficult. Despite this weakness the relative reduction does tell us which interventions were associated with the largest reductions. This allows these successes to be examined to see if they are applicable to other settings. This review is broad in scope and therefore does not allow detailed examination of particular intervention types to find the elements that lead to success. Reviews of particular types of interventions have been done and explore these factors closely [122–124]. Lastly, our use of relative reductions leaves out absolute reduction and cost savings information that may be important to decision makers.

Conclusions

Our review found that research into interventions to reduce test utilization is ongoing and many strategies can be successful. There are many examples of low investment interventions that work, especially when they target tests that have a high rate of inappropriate use. Due to questionable validity and generalizability of these results, this review should be used as a reference for finding high quality studies with large relative reductions that apply to a setting of interest. Future research needs to determine how low input strategies using computerized order entry can be used to implement system changes, audit and feedback, and education. Database monitoring should be developed so that institutions know when anomalous test use is occurring and which physicians are deviating from the mean. This will allow for targeting of interventions to where they will have the greatest effect. Lastly, further studies in this field must monitor for adverse events caused by reduced testing.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Financial support: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References

1. National Health Expenditure trends, 1975 to 2013. Ottawa, ON: Canadian Institute for Health Information, 2013. Search in Google Scholar

2. Health at a Glance 2013: OECD Indicators. Paris: OECD Publishing, 2013. Search in Google Scholar

3. Qaseem A, Alguire P, Dallas P, Feinberg LE, Fitzgerald FT, Horwitch C, et al. Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Ann Intern Med 2012;156:147–9. Search in Google Scholar

4. van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilization is? A systematic review of laboratory clinical audits. J Am Med Assoc 1998;280:9. Search in Google Scholar

5. Grossman RM. A review of physician cost-containment strategies for laboratory testing. Med Care 1983;21:783–802. Search in Google Scholar

6. Axt-Adam P, van der Wouden JC, van der Does E. Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care 1993;31:784–94. Search in Google Scholar

7. Young DW. Improving laboratory usage: a review. Postgrad Med 1988;64:283–9. Search in Google Scholar

8. Solomon DH, Hashimoto H, Daltroy L, Liang MH. Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework. J Am Med Assoc 1998;280:2020–7. Search in Google Scholar

9. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Br Med J 2009;339:b2535. Search in Google Scholar

10. Egger M, Smith GD, Altman DG. Principles of and procedures for systematic reviews. Systematic Reviews in Health Care: meta analysis in context, 2nd ed. London: BMJ Publishing Group, 2008:23–42. Search in Google Scholar

11. Cochrane Effective Practice and Organization of Care Review Group. Data collection checklist. Ottawa: Institute of Population Health, University of Ottawa, 2002. Search in Google Scholar

12. Al AM, Zidan A, Hamza AA, Al-Suwaidi R. Evaluation of laboratory investigations in accident and emergency department, Salmaniya Medical Complex. Bahrain Med Bull 2002;24:95–7. Search in Google Scholar

13. Amukele TK, Baird GS, Chandler WL. Reducing the use of coagulation test panels. Blood Coagul Fibrinolysis 2011;22:688–95. Search in Google Scholar

14. Archambault AJ, Arkin CF, Skelton T, Abel G, Gawoski J. Blood (ASH Annual Meeting Abstracts) 2011;118:Abstract 4774. Search in Google Scholar

15. Attali M, Barel Y, Somin M, Beilinson N, Shankman M, Ackerman A, et al. A cost-effective method for reducing the volume of laboratory tests in a university-associated teaching hospital. Mt Sinai J Med 2006;73:787–94. Search in Google Scholar

16. Barazzoni F, Grilli R, Amicosante AM, Brescianini S, Marca MA, Baggi M, et al. Impact of end user involvement in implementing guidelines on routine pre-operative tests. Int J Qual Health Care 2002;14:321–7. Search in Google Scholar

17. Bareford D, Hayling A. Inappropriate use of laboratory services: long term combined approach to modify request patterns. Br Med J 1990;301:1305–7. Search in Google Scholar

18. Baricchi R, Zini M, Nibali MG, Vezzosi W, Insegnante V, Manfuso C, et al. Using pathology-specific laboratory profiles in clinical pathology to reduce inappropriate test requesting: two completed audit cycles. BMC Health Serv Res 2012;12:187. Search in Google Scholar

19. Barie PS, Hydo LJ. Lessons learned: durability and progress of a program for ancillary cost reduction in surgical critical care. J Trauma 1997;43:590–6. Search in Google Scholar

20. Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma’luf N, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med 1997;157:2501–8. Search in Google Scholar

21. Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma’luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144–50. Search in Google Scholar

22. Berenholtz S, Pronovost P, Lipsett P, Dawson P, Dorman T. Assessing the effectiveness of critical pathways on reducing resource utilization in the surgical intensive care unit. Intens Care Med 2001;27:1029–36. Search in Google Scholar

23. Berwick DM, Coltin KL. Feedback reduces test use in a health maintenance organization. J Am Med Assoc 1986;255:1450–4. Search in Google Scholar

24. Boon-Falleur L, Sokal E, Peters M, Ketelslegers JM. A rule-based decision support application for laboratory investigations management. Proc Annu Symp Comput Appl Med Care 1995;314–8. Search in Google Scholar

25. Buckingham K, Russell I, Ross I, Gibson P, Paterson N. The effect of allowing clinical discretion in ordering biochemical tests. Evaluation by complementary methods. Int J Technol Assess 1994;10:695–700. Search in Google Scholar

26. Bunting PS, Van Walraven C. Effect of a controlled feedback intervention on laboratory test ordering by community physicians. Clin Chem 2004;50:321–6. Search in Google Scholar

27. Burnett L, Chesher D, Burnett JR. Optimizing the availability of ‘stat’ laboratory tests using Shewhart ‘C’ control charts. Ann Clin Biochem 2002;39:140–4. Search in Google Scholar

28. Calderon-Margalit R, Mor-Yosef S, Mayer M, Adler B, Shapira SC. An administrative intervention to improve the utilization of laboratory tests within a university hospital. Int J Qual Health Care 2005;17:243–8. Search in Google Scholar

29. Carson K, Murillo P, Runge A, Vogel DG, Baum LG. A test-ordering algorithm can reduce laboratory test use. Lab Med 1995;26:277–81. Search in Google Scholar

30. Carter E, Bennett BD. Reference test review by pathology house staff: a cost-containment strategy for the clinical laboratory. Clin Leader Manag Rev 2002;16:3–6. Search in Google Scholar

31. Chen P, Tanasijevic MJ, Schoenenberger RA, Fiskio J, Kuperman GJ, Bates DW. A computer-based intervention for improving the appropriateness of antiepileptic drug level monitoring. Am J Clin Pathol 2003;119:432–8. Search in Google Scholar

32. Chu K, Wagholikar A, Greenslade J. Reducing laboratory test ordering by limiting the range of tests that may be requested by interns and residents on pathology request forms. Emerg Med Australas 2012;24:21. Search in Google Scholar

33. Chu UB, Clevenger FW, Imami ER, Lampard SD, Frykberg ER, Tepas JJ. The impact of selective laboratory evaluation on utilization of laboratory resources and patient care in a level-I trauma center. Am J Surg 1996;172:558–3. Search in Google Scholar

34. Cohen DI, Jones P, Littenberg B, Neuhauser D. Does cost information availability reduce physician test usage? A randomized clinical trial with unexpected findings. Med Care 1982;20: 286–92. Search in Google Scholar

35. Davidoff F, Goodspeed R, Clive J. Changing test ordering behavior. A randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care 1989;27:45–58. Search in Google Scholar

36. Detsky AS, McLaughlin JR, Abrams HB, L’Abbe K, Markel FM. Do interns and residents order more tests than attending staff? Results of a house staff strike. Med Care 1986;24: 526–34. Search in Google Scholar

37. Dickinson JC. From process to policy: a generic prescription for test over-utilization in the emergency department. Fam Pract Res J 1987;7:12–21. Search in Google Scholar

38. Dixon RH, Laszlo J. Utilization of clinical chemistry services by medical house-staff. Arch Intern Med 1974;134:1064–7. Search in Google Scholar

39. Dowling PT, Alfonsi G, Brown MI, Culpepper L. An education program to reduce unnecessary laboratory tests by residents. Acad Med 1989;64:410–2. Search in Google Scholar

40. Durand-Zaleski I, Rymer JC, Roudot-Thoraval F, Revuz J, Rosa J. Reducing unnecessary laboratory use with new test request form: example of tumour markers. Lancet 1993;342:150–3. Search in Google Scholar

41. Durieux P, Ravaud P, Porcher R, Fulla Y, Manet C-S, Chaussade S. Long-term impact of a restrictive laboratory test ordering form on tumor marker prescriptions. Int J Technol Assess Health Care 2003;19:106–13. Search in Google Scholar

42. Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406–9. Search in Google Scholar

43. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ 1977;52:578–81. Search in Google Scholar

44. Emerson JF, Emerson SS. The impact of requisition design on laboratory utilization. Am J Clin Pathol 2001;116:879–84. Search in Google Scholar

45. Everett GD. Impact of supervision by medical teachers and in-patient test control programmes on the out-patient test utilization of residents. Med Educ 1985;19:138–42. Search in Google Scholar

46. Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Arch Intern Med 1983;143:942–4. Search in Google Scholar

47. Feldkamp CS, Carey JL. An algorithmic approach to thyroid function testing in a managed care setting: 3-Year experience. Am J Clin Pathol 1996;105:11–6. Search in Google Scholar

48. Feldman LS, Shihab HM, Thiemann D, Yeh HC, Ardolino M, Mandell S, et al. Impact of providing fee data on laboratory test ordering: a controlled clinical trial. JAMA Intern Med 2013;173:903–8. Search in Google Scholar

49. Finegan BA, Rashiq S, McAlister FA, O’Connor P. Selective ordering of preoperative investigations by anesthesiologists reduces the number and cost of tests. Can J Anaesth 2005;52:575–80. Search in Google Scholar

50. Fong BW. Common diagnostic tests: use and interpretation. Hawaii Med J 1989;48:7–9. Search in Google Scholar

51. Fowkes FG, Hall R, Jones JH, Scanlon MF, Elder GH, Hobbs DR, et al. Trial of strategy for reducing the use of laboratory tests. Br Med J 1986;292:883–5. Search in Google Scholar

52. Friedman KG, Rathod RH, Farias M, Graham D, Powell AJ, Fulton DR, et al. Resource utilization after introduction of a standardized clinical assessment and management plan. Congenit Heart Dis 2010;5:374–81. Search in Google Scholar

53. Froom P, Barak M. Cessation of dipstick urinalysis reflex testing and physician ordering behavior. Am J Clin Pathol 2012;137:486–9. Search in Google Scholar

54. Gama R, Nightingale PG, Broughton PM, Peters M, Bradby GV, Berg J, et al. Feedback of laboratory usage and cost data to clinicians: does it alter requesting behaviour? Ann Clin Biochem 1991;28(Pt 2):143–9. Search in Google Scholar

55. Gama R, Nightingale PG, Broughton PM, Peters M, Ratcliffe JG, Bradby GV, et al. Modifying the request behaviour of clinicians. J Clin Pathol 1992;45:248–9. Search in Google Scholar

56. Goddard K, Austin SJ. Appropriate regulation of routine laboratory testing can reduce the costs associated with patient stay in intensive care. Crit Care 2011;15:S47–8. Search in Google Scholar

57. Golden WE, Pappas AA, Lavender RC. Financial unbundling reduces outpatient laboratory use. Arch Intern Med 1987;147:1045–8. Search in Google Scholar

58. Gortmaker SL, Bickford AF, Mathewson HO, Dumbaugh K, Tirrell PC. A successful experiment to reduce unnecessary laboratory use in a community hospital. Med Care 1988;26:631–42. Search in Google Scholar

59. Grivell AR, Forgie HJ, Fraser CG, Berry MN. Effect of feedback to clinical staff of information on clinical biochemistry requesting patterns. Clin Chem 1981;27:1717–20. Search in Google Scholar

60. Groopman DS, Powers RD. Effect of ‘standard order’ deletion on emergency department coagulation profile use. Ann Emerg Med 1992;21:524–7. Search in Google Scholar

61. Gross PA, Van Antwerpen CL, Hess WA, Reilly KA. Use and abuse of blood cultures: program to limit use. Am J Infect Control 1988;16:114–7. Search in Google Scholar

62. Haschke-Becher E, Totzke U, Afazel S, Johansson T, Schwarz M, Ladurner G, et al. Clinical decision rules for the use of liquor diagnostics in hospitalized neurology patients reduced costs without affecting clinical outcomes. Int J Technol Assess Health Care 2009;25:208–13. Search in Google Scholar

63. Hutton HD, Drummond HS, Fryer AA. The rise and fall of C-reactive protein: managing demand within clinical biochemistry. Ann Clin Biochem 2009;46:155–8. Search in Google Scholar

64. Jelinek GA, Banham ND. Reducing the use of plain abdominal radiographs in an emergency department. Arch Emerg Med 1990;7:241–5. Search in Google Scholar

65. Kelly AM. Rationalising the ordering of blood cultures. Aust Health Rev 1998;21:245–50. Search in Google Scholar

66. Kumwilaisak K, Noto A, Schmidt UH, Beck CI, Crimi C, Lewandrowski K, et al. Effect of laboratory testing guidelines on the utilization of tests and order entries in a surgical intensive care unit. Crit Care Med 2008;36:2993–9. Search in Google Scholar

67. Larocque BJ, Maykut RJ. Implementation of guidelines for preoperative laboratory investigations in patients scheduled to undergo elective surgery. Can J Surg 1994;37:397–401. Search in Google Scholar

68. Levick DL, Stern G, Meyerhoefer CD, Levick A, Pucklavage D. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis 2013;13:43. Search in Google Scholar

69. Lewandrowski K, Bailey E, Dhanak E, Laposata M, Flood J. Mandatory laboratory consultation: how one hospital’s program reduced overuse of cardiac tests. Lab Med 1994;25:460–3. Search in Google Scholar

70. Mancuso CA. Impact of new guidelines on physicians’ ordering of preoperative tests. J Gen Intern Med 1999;14:166–72. Search in Google Scholar

71. Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med 1980;303:1330–6. Search in Google Scholar

72. Marton KI, Tul V, Sox HC. Modifying test-ordering behavior in the outpatient medical clinic. A controlled trial of two educational interventions. Arch Intern Med 1985;145:816–21. Search in Google Scholar

73. May TA, Clancy M, Critchfield J, Ebeling F, Enriquez A, Gallagher C, et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol 2006;126:200–6. Search in Google Scholar

74. Mehari SM, Havill JH, Montgomery C. A written guideline implementation can lead to reductions in laboratory testing in an intensive care unit. Anaesth Intensive Care 1997;25:33–7. Search in Google Scholar

75. Meng QH, Zhu S, Booth C, Stevens L, Bertsch B, Qureshi M, et al. Impact of the cardiac troponin testing algorithm on excessive and inappropriate troponin test requests. Am J Clin Pathol 2006;126:195–9. Search in Google Scholar

76. Merlani P, Garnerin P, Diby M, Ferring M, Ricou B. Quality improvement report: linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care. Br Med J 2001;323:620–4. Search in Google Scholar

77. Meyer L, Schmeler K, Wallbillich J, Urbauer D, Soliman P, Frumovitz M, et al. Clinical practice guidelines decrease unnecessary Pap tests in survivors of gynecologic malignancies. Gynecol Oncol 2011;120:S30–1. Search in Google Scholar

78. Miyakis S, Karamanof G, Liontos M, Mountokalakis TD. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med 2006;82:823–9. Search in Google Scholar

79. Morris AJ, Wilson ML, Reller LB. Application of rejection criteria for stool ovum and parasite examinations. J Clin Microbiol 1992;30:3213–6. Search in Google Scholar

80. Mutimer D, McCauley B, Nightingale P, Ryan M, Peters M, Neuberger J. Computerised protocols for laboratory investigation and their effect on use of medical time and resources. J Clin Pathol 1992;45:572–4. Search in Google Scholar

81. Nardella A, Pechet L, Snyder LM. Continuous improvement, quality control, and cost containment in clinical laboratory testing: effects of establishing and implementing guidelines for preoperative tests. Arch Pathol Lab Med 1995;119:518–22. Search in Google Scholar

82. Neilson EG, Johnson KB, Rosenbloom ST, Dupont WD, Talbert D, Giuse DA, et al. The impact of peer management on test-ordering behavior. Ann Intern Med 2004;141:196–204. Search in Google Scholar

83. Nelson III RB. Teaching technologic restraint. An evaluation of a single session. Eval Health Prof 1978;1:21–8. Search in Google Scholar

84. Nightingale PG, Peters M, Mutimer D, Neuberger JM. Effects of a computerised protocol management system on ordering of clinical tests. Qual Health Care 1994;3:23–8. Search in Google Scholar

85. Nirel N, Rosen B, Sharon A, Samuel H, Cohen AD. The impact of an integrated hospital-community medical information system on quality of care and medical service utilisation in primary-care clinics. Inform Health Soc Ca 2011;36:63–74. Search in Google Scholar

86. Pageler N, Longhurst C, Shin A, Adams E, Widen E, Barth R, et al. Using computerized provider order entry (CPOE) to promote sustained optimization of diagnostic radiologic service utilization. Crit Care Med 2009;37:A363. Search in Google Scholar

87. Pageler NM, Franzon D, Longhurst CA, Wood M, Shin AY, Adams ES, et al. Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization. Pediatr Crit Care Med 2013;14:413–9. Search in Google Scholar

88. Patel HD, Livsey SA, Swann RA, Bukhari SS. Can urine dipstick testing for urinary tract infection at point of care reduce laboratory workload? J Clin Pathol 2005;58:951–4. Search in Google Scholar

89. Pilon CS, Leathley M, London R, McLean S, Phang PT, Priestley R, et al. Practice guideline for arterial blood gas measurement in the intensive care unit decreases numbers and increases appropriateness of tests. Crit Care Med 1997;25:1308–13. Search in Google Scholar

90. Pop P, Winkens RA. A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. J R Coll Gen Pract 1989;39:507–8. Search in Google Scholar

91. Power LM, Thackray NM. Reduction of preoperative investigations with the introduction of an anaesthetist-led preoperative assessment clinic. Anaesth Intensive Care 1999;27:481–8. Search in Google Scholar

92. Powles L, Green J, Taylor E. Can redesigning a laboratory request form reduce the number of inappropriate PSA requests without compromising clinical outcome. Urology 2009;74:S148. Search in Google Scholar

93. Prat G, Lefevre M, Nowak E, Tonnelier J-M, Renault A, L’Her E, et al. Impact of clinical guidelines to improve appropriateness of laboratory tests and chest radiographs. Intens Care Med 2009;35:1047–53. Search in Google Scholar

94. Pysher TJ, Bach PR, Lowichik A, Petersen MD, Shields LH. Chemistry test ordering patterns after elimination of predefined multitest chemistry panels in a children’s hospital. Pediatr Dev Pathol 1999;2:446–53. Search in Google Scholar

95. Ramoska EA. Information sharing can reduce laboratory use by emergency physicians. Am J Emerg Med 1998;16:34–6. Search in Google Scholar

96. Ratnaike S, Hunt D, Eilermann LJ, Hazen R, Deam D. The investigation of chest pain: audit and intervention. Med J Australia 1993;159:666–71. Search in Google Scholar

97. Rhyne RL, Gehlbach SH. Effects of an educational feedback strategy on physician utilization of thyroid function panels. J Fam Practice 1979;8:1003–7. Search in Google Scholar

98. Roberts DE, Bell DD, Ostryzniuk T, Dobson K, Oppenheimer L, Martens D, et al. Eliminating needless testing in intensive care – an information-based team management approach. Crit Care Med 1993;21:1452–8. Search in Google Scholar

99. Rosenbloom ST, Chiu K-W, Byrne DW, Talbert DA, Neilson EG, Miller RA. Interventions to regulate ordering of serum magnesium levels: report of an unintended consequence of decision support. J Am Med Informat Assoc 2005;12:546–53. Search in Google Scholar

100. Santos IS, Bensenor IM, Machado JB, Fedeli LM, Lotufo PA. Intervention to reduce C-reactive protein determination requests for acute infections at an emergency department. Emerg Med J 2012;29:965–8. Search in Google Scholar

101. Seguin P, Bleichner J, Grolier J, Guillou Y, Malledant Y. Effects of price information on test ordering in an intensive care unit. Intens Care Med 2002;28:332–5. Search in Google Scholar

102. Shalev V, Chodick G, Heymann AD. Format change of a laboratory test order form affects physician behavior. Int J Med Inform 2009;78:639–44. Search in Google Scholar

103. Sorita A, Steinberg D, Leitman M, Luhan P, Burger A, Sivaprasad L. Impact of targeted individual feedback on provider patterns of stat laboratory test ordering frequency. J Hosp Med 2011;6:S135–6. Search in Google Scholar

104. Spray J, Brackbill M, Baker P. Evaluation of effect of electronic decision tree electrolyte replacement protocol on number of electrolyte labs ordered for patients in critical care. Crit Care Med 2012;40:248. Search in Google Scholar

105. Stafford RS. Feedback intervention to reduce routine electrocardiogram use in primary care. Am Heart J 2003;145:979–85. Search in Google Scholar

106. Sucov A, Bazarian JJ, deLahunta EA, Spillane L. Test ordering guidelines can alter ordering patterns in an academic emergency department. J Emerg Med 1999;17:391–7. Search in Google Scholar

107. Sussman E, Goodwin P, Rosen H. Administrative change and diagnostic test use: the effect of eliminating standing orders. Med Care 1984;22:569–72. Search in Google Scholar

108. Thompson RS, Kirz HL, Gold RA. Changes in physician behavior and cost savings associated with organizational recommendations on the use of ‘routine’ chest X rays and multichannel blood tests. Prev Med 1983;12:385–96. Search in Google Scholar

109. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results. Effects on outpatient testing. J Am Med Assoc 1988;259:1194–8. Search in Google Scholar

110. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322:1499–504. Search in Google Scholar

111. Toubert ME, Chevret S, Cassinat B, Schlageter MH, Beressi JP, Rain JD. From guidelines to hospital practice: reducing inappropriate ordering of thyroid hormone and antibody tests. Eur J Endocrinol 2000;142:605–10. Search in Google Scholar

112. van Wijk MA, van dL, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. a randomized trial. Ann Intern Med 2001;134:274–81. Search in Google Scholar

113. Vardy DA, Simon T, Limoni Y, Kuperman O, Rabzon I, Cohen A, et al. The impact of structured laboratory routines in computerized medical records in a primary care service setting. J Med Syst 2005;29:619–26. Search in Google Scholar

114. Verstappen WH, van der Weijden T, Sijbrandij J, Smeele I, Hermsen J, Grimshaw J, et al. Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. J Am Med Assoc 2003;289:2407–12. Search in Google Scholar

115. Wagholikar A, O’Dwyer J, Hansen D, Chu K. Observing effectiveness of pathology ordering controls in emergency departments. Stud Health Tech Informat 2011;168:172–8. Search in Google Scholar

116. Winkens RA, Pop P, Bugter-Maessen A, Grol RP, Kester AD, Beusmans GH, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet 1995;345:498–502. Search in Google Scholar

117. Winkens RA, Pop P, Grol RP, Kester AD, Knottnerus JA. Effect of feedback on test ordering behaviour of general practitioners. Br Med J 1992;304:1093–6. Search in Google Scholar

118. Wones RG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care 1987;25:78–82. Search in Google Scholar

119. Zaat JO. Attempts to change test ordering behavior in laboratories. Ann Clin Biochem 1992;17:8–14. Search in Google Scholar

120. Zaidi AK, Macone A, Goldmann AD. Impact of simple screening criteria on utilization of low-yield bacterial stool cultures in a Children’s Hospital. Pediatrics 1999;103: 1189–92. Search in Google Scholar

121. Zia HM, Singh G. Optimization of utilization of serum protein analysis: role of the electronic medical record in promoting consultation by pathology. Am J Clin Pathol 2013;139:793–7. Search in Google Scholar

122. Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, et al. Can computerized clinical decision support systems improve practitioners’ diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011;6:88. Search in Google Scholar

123. Buntinx F, Winkens R, Grol R, Knottnerus JA. Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract Res J 1993;10:219–28. Search in Google Scholar

124. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259. Search in Google Scholar

Supplemental Material

The online version of this article (DOI: 10.1515/cclm-2014-0778) offers supplementary material, available to authorized users.

Received: 2014-7-29
Accepted: 2014-8-25
Published Online: 2014-9-27
Published in Print: 2015-2-1

©2015 by De Gruyter