Since its founding in 1946, the College of American Pathologists (CAP) has maintained the highest standards for laboratory medicine through education, evaluation, and certification. One form of External Quality Assurance – proficiency testing (PT) – has been the centerpiece of that mission. The program ensures standardized testing across clinical laboratories through the use of unknown challenges.
Laboratories compare test results of unknown specimens to monitor quality on an ongoing basis. Over 500 medical and scientific experts organized into 25 scientific resource committees oversee the CAP PT program which includes more than 600 tests performed by over 22,000 laboratories in 108 countries. There are currently 14,723 urine PT subscribers worldwide. What began as a glucose testing survey now includes next generation sequencing. It is the most comprehensive laboratory peer-review comparison program in the world and successful completion of proficiency testing is a requirement for laboratory licensure in the US.
Although this article is focused on urine sediment external quality assurance, it is important to review the origin of such testing in the US. The steps toward laboratory excellence were gradual and proved to be a mixture of dedication, foresight, and regulatory oversight.
History of proficiency testing in the US
In 1946, responding to reports that Philadelphia inter-hospital laboratory testing showed very poor correlation, F. William Sunderman, MD, PhD, a founding member of the CAP Board of Governors, conducted a Pennsylvania statewide anonymous laboratory testing survey . The results were distressing. Serum glucose testing, e.g. showed only 58% correct responses and almost 10% of all testing results were deemed “totally unacceptable”. Lack of proper training, inadequate equipment, cramped laboratory conditions, and poor communication between the pathologists and staff were cited as reasons for the poor performance – issues that still affect laboratories today.
The CAP followed up the Pennsylvania study with two national surveys in 1947 and 1948 and the results were even more unfavorable . This led Dr. Sunderman and a group of pathologists to create a laboratory surveillance program in several states. The surveys were intended to assess the general state of laboratory testing and identify areas for improvement. Although not a formal PT program, there was a definite improvement in inter-laboratory precision from earlier assessments, demonstrating the value of comparison testing .
These early efforts to improve laboratory testing were formalized by the CAP in 1964 with the creation of the very first national comprehensive inter-laboratory comparison program. Unknown testing samples were divided into three kits covering chemistry, hematology, blood bank, urinalysis, bacteriology, parasitology, and serology. Testing included serum bilirubin, cholesterol, blood smear evaluation, and stool culture. Urine was tested for ketones, protein, and reducing substances. Kodachrome transparency slides of urine sediment were added the following year with identifications of WBC clump, RBC cast, urothelial cell and granular cast. Beginning in 1971, transparencies of urinary sediment became a regular part of the testing regimen. Gradually, the program matured into a true external quality assurance activity.
US regulatory landscape
All the concerns regarding accurate laboratory testing did not escape the notice of the US Federal Government. Laboratories in the US experienced considerable negative press. There were unsubstantiated claims by the Federal Communicable Disease Center and New York State Health Department officials that up to 25% of laboratory tests were inaccurate . This led to legislation called the “Partnership for Health Amendments of 1967”, eventually known as the Federal Clinical Laboratory Improvement Act of 1967 (CLIA). Proficiency testing was required for all regulated analytes, resulting in a dramatic expansion in proficiency testing. The final rules implementing CLIA were published in 1992. These regulations replaced the Medicare, Medicaid, and CLIA 1967 standards with a single set of requirements that applies to almost all laboratories testing human specimens.
CLIA divides analytes into three groups: regulated, unregulated, and waived.
Regulated analytes include most blood tests; hemoglobin and glucose are examples. The government keeps a list of these regulated tests and requires enrollment in a Centers for Medicare and Medicaid Services (CMS) approved proficiency testing program. There must be five PT challenges evaluated three times per year. The exception is mycobacteriology which requires five challenges two times per year. The count of five challenges per testing event allowed the imposition of a percentage grading system. The pass/fail criterion of 80% made it easier to monitor laboratory performance.
At the other end of the regulatory testing spectrum are so-called waived tests which are considered by the government to be so simple that errors in testing are nearly impossible. Waived tests are listed in the CLIA regulations. Examples include urine pregnancy tests by visual color comparison, blood glucose by glucose monitoring devices cleared by the Food and Drug Administration (FDA) for home use and dipstick or tablet reagent urinalysis (non-automated tests for bilirubin, glucose, hemoglobin, ketone, leukocytes, nitrite, pH, protein, specific gravity, and urobilinogen). Sites performing only waived testing must have a CLIA certificate and follow the manufacturer’s instructions but other CLIA requirements, such as proficiency testing, do not apply although many laboratories still subscribe to PT programs to help maintain quality. The initial listing of “waived” tests was quite limited but additional legislation has now expanded this list to include any tests available “over the counter”.
Unregulated analytes are all other laboratory tests that are not in the waived or regulated category. Interestingly, testing on body fluids falls into the unregulated category. That includes urine sediment evaluation. Unregulated analytes have fewer challenges – usually two each year – and fewer samples per challenge. CAP urine sediment surveys consist of two mailings containing three or four images each.
Participants in organized PT programs analyze the specimens and report results. The answers are aggregated into peer groups so that participants are compared to laboratories using similar methodology. All responses are monitored by the CAP accreditation program. There are no sanctions if a laboratory has an isolated unsatisfactory PT event for a specific analyte or testing event, such as a set of morphologic identifications in peripheral blood and urine sediment. If a laboratory has unsatisfactory performance (<80% score in most cases) for the same analyte or testing event in two consecutive or two out of three challenges, the laboratory is cited for unsuccessful performance.
CLIA requires that laboratories test PT specimens in the same manner as patient specimens. This means that the same laboratory staff who routinely perform patient testing should analyze the PT specimens using routine test methods. There can be no repeat testing and averaging of results. Laboratories are prohibited from comparing their results with another laboratory (even another laboratory in the same practice) and are prohibited from sending PT specimens to a reference laboratory prior to submitting the results to the accrediting agency. Violations can result in loss of licensure .
Managing the proficiency testing programs
Initially, all activities concerning proficiency testing were overseen by the Standards Committee of the CAP Board of Governors. As the various PT programs expanded, there was a need for specific scientific expertise in a particular discipline of laboratory medicine. This resulted in the creation of Resource Committees that manage test development .
There are currently over 500 members and 25 Resource Committees involved in CAP proficiency testing. Appointments to committees are for 3–6 years. Members are volunteers – primarily MD or MD-PhD pathologists who are recognized as leaders in their field. In a few instances, PhD scientists are also committee members. Industry is not represented; there is a vigorous attempt not to show instrument bias. These practicing laboratory professionals work with collaborators from other specialty medical societies and the CAP professional staff to keep the PT program current and relevant. This scientific expertise is a key distinguishing feature of the CAP’s PT program.
The role of each committee is to define and monitor new technologies and to contribute to the development, launch, maintenance and enhancement of proficiency testing programs. Members provide input to accreditation requirements of best practices and help design continuing medical education, as noted below.
Urine sediment proficiency testing
Today, the CAP offers four urine sediment proficiency test products tailored to the needs of different laboratories. One is specifically designed for automated urinalysis instruments.
As noted earlier, urine sediment evaluation is not an official US regulated test. Nevertheless, all CAP accredited laboratories are required to enroll in a proficiency testing program for manual urine sediment identification. The CAP monitors each laboratory’s performance and may require it to stop performing the test until any issues are resolved. The only PT enrollment exception is if the laboratory has a certificate of Provider-Performed Microscopy Procedures (PPMP) which permits a laboratory to perform waived tests and a limited list of moderate complexity tests. Testing is performed by physicians, nurse practitioners, nurse midwives, dentists, or physician assistants rather than laboratory technologists. Such designated tests require the use of a microscope and the specimen must be labile. Urine sediment examination certainly applies. Although formal PT program enrollment is not required, the laboratory must verify the accuracy of results at least twice a year. To do that, laboratories with a PPMP certificate may enroll in a formal PT program.
Oversight of the urine sediment proficiency challenges is handled by the Hematology and Clinical Microscopy Committee (HCMRC), which meets three times a year. In most instances, urine sediment images are first photographed by members of the HCMRC and then reviewed by the entire committee to select the best examples for upcoming challenges.
The list of possible urine sediment identifications are shown in Table 1. This list is reviewed each year and modified as appropriate. The images were originally captured as 35 mm transparency slides which were duplicated and sent to participants along with case histories and pertinent laboratory data. Later, printed color photographs and web-based images were introduced.
Before sending urine sediment photographs to participants, they are reviewed by a cohort of 20 or 30 referees chosen from laboratories participating in the cell identification portion of the survey who have demonstrated perfect performance over the previous years. Referee results are reviewed by the scientific committee and assigned “good” or “acceptable” answers. In most cases, only those photomicrographs that achieve referee consensus in identification are used for graded cell identification. The urine sediment images are scored if there is ≥80% consensus of either referee or participant laboratories. Additional images may be selected that are either considered to be difficult or unusual. These are ungraded challenges that are included for their educational value.
In some instances, referees are not used. In this case, only the participant scores define a good or acceptable response. This method eliminates the need to manage referees but is more likely to result in non-consensus or ungraded identifications.
Laboratories who do not obtain at least 80% correct responses must analyze the reasons for the failure, report the results, and initiate corrective action that is reviewed by the CAP laboratory inspection team. The majority of errors are either clerical or technical in nature. Over the years, there has been a progressive decline in the number of errors. This supports the belief that education and regulatory oversight are major contributors to improved proficiency testing performance. By extension, then, it is believed that patient care has also improved.
Proficiency testing programs focus on the analytic phase of laboratory testing (Table 2), but a high percentage of erroneously reported test results originate in either the pre-analytic (46%–68%) or post-analytic (18%–47%) phases of testing [5, 6]. Laboratory involvement in institutional quality management programs is focused on identifying these errors and is mandated by the CAP for accreditation . The CAP’s Q-Probes program is designed to help address these issues.
Over the years, hundreds of photographs have been used for urine sediment proficiency testing. This image databank is a unique resource because it represents the consensus of many individuals and laboratories regarding the identity of urine sediment constituents. Review of participant and referee responses help to identify which morphologic variants illustrate unambiguous examples of the various urine formed elements and which are more difficult to classify.
Reviewing the participant performance data since the program began, a few points can be made (Table 2). Among casts, almost all identifications averaged better than the 80% target and were often 90% or better. Rare examples of a fatty cast and a hemoglobin cast had slightly poorer results. Identification of cells was also at least 80% correct in most cases but certain photographs of macrophages, transitional epithelial cells and renal tubular epithelial cells had scores in the 15%–63% range. For crystals, the most challenging were drugs, uric acid variants, and one example of an oxalate crystal. Miscellaneous identifications were correct at the required 80% level except for rare images of bacteria, starch and fibers.
Education in proficiency testing
Although cell identification serves a regulatory function, the CAP is primarily focused on the educational value of proficiency testing. The need for continuing education became clear as the surveys program gained acceptance. Initially, workshops were held discussing chemistry and hematology testing . Over the years, educational discussions became part of the PT program itself.
In addition to the grading document, each laboratory receives a Participant Summary Report that lists results from all participants for each analyte grouped by methodology. The report contains critiques discussing disease pathophysiology and key morphologic features. In some cases, the write-ups are quite detailed and expansive. This teaching component is what helps to set the CAP’s program apart from all others. The participant summaries formed the basis for the Color Atlas of Urinary Sediment published by the CAP in 2010 . There is also an online glossary of terms that help to standardize the morphologic identifications.
Value of proficiency testing
The surveys have provided valuable data mining opportunities. Resource Committees or their individual members have published numerous scientific articles documenting improved laboratory performance.
The Proficiency Testing Exception Survey (PTES) was initiated by the CAP in 1994 and requires under-performing laboratories to analyze the reasons for repeated failures (defined as two or more failures in three testing events for any analyte), report their results, and initiate a corrective action plan that will be reviewed during the next scheduled inspection. Since the implementation of government regulations, there has been a progressive decline in the number of PTES events for the Urinalysis surveys, suggesting that regulatory concerns and education are major contributors to improved performance on proficiency testing challenges .
This is gratifying but the true value of proficiency testing lies in its benefit to patient care. Does getting the “right” answer contribute to improved patient outcomes? Is the expense and time worth it? This is difficult to prove directly. Laboratory tests are ordered to assist in the clinical evaluation of a patient’s signs and symptoms and help to establish or exclude disease. Proficiency testing goes a long way toward ensuring that the result is reported to the correct care-giver and reflects the actual concentration of the analyte under consideration in the right patient.
For urinalysis, proper sediment evaluation provides important diagnostic information regarding urinary tract disease, particularly bladder neoplasms, upper and lower urinary tract infections, acute tubular necrosis, allergic nephritis, nephrotic syndrome, crush syndrome, urate nephropathy, cystinuria, and renal transplant rejection [8,9,10].
Correct identification of urine sediment is just one piece of the diagnostic puzzle. It is recognized that a more comprehensive approach to laboratory test interpretation is needed . This is both a challenge and an opportunity for the laboratory to be actively involved with the medical staff to ensure that results are optimally applied to expedite patient diagnosis and improve outcomes.
Urine proficiency testing issues
Figure 1 shows the frequency distribution of PT results. The graph reflects both the rarity of some formed elements in the urine sediment but also the CAP Committee’s desire to emphasize identifications that are encountered in most practice settings – that is the starting point for good laboratory practice. The most commonly used identifications are leukocytes, erythrocytes and granular casts. The least common are drug crystals. This difference also reflects the difficulty is obtaining good photographs of crystals.
The CAP resource committee has tried to centralize image capture and has also tested various transport media in an attempt to preserve urine containing unusual sediment findings. The idea was to have interesting and challenging cases sent to a single pathologist who would be the photography expert. The preservative fluid was never fully successful and the idea was abandoned. In the end, urine sediment proficiency testing requires multiple dedicated clinical pathologists or technologists who are willing to take the time to photograph interesting elements.
Most laboratories are not equipped with proper photography equipment to capture urine sediment. Those that do, primarily use bright field photography. Phase contrast microscopy, championed by Giovanni Fogazzi, MD, and others , is commonly employed in European laboratories, and is becoming more common in the US. It requires an inexpensive phase condenser and objective lens and the accompanying training.
The available database of images needs to be updated. Photographs of crystals mostly reflect antibiotics and radiologic contrast agents no longer in use. The increasing emphasis on urinary cytology to screen for malignancies will require laboratorians to become familiar with the appearance of malignant cells in the unstained specimens encountered in the clinical laboratory .
Slides in the database should also reflect the use of alternative imaging techniques for cell identification – bright field, polarized light, phase contrast, and interference contrast microscopy (Nomarski imaging). As automated instruments incorporating high-resolution photography become more widely available, challenging cases can be more easily captured and cataloged.
The CAP resource committee is always willing to accept contributions from other hospitals. After all, the goal is to further the identification of elements in the urinary sediment that trigger treatment decisions.
How urine sediment images are delivered to participants must evolve. Currently, all proficiency testing challenges are evaluated using color still images in printed form or viewed on a computer monitor. Participants select answers from a master list of identifications. Other testing methods need to be explored. Whole slide scans have been successfully deployed for the CAP’s virtual peripheral blood smear proficiency testing program but typical wet mounts for urinary sediment evaluation are unstained and subject to distortion by Brownian motion. One solution may be video. The CAP is working with the University of Utah and other vendors to pilot video clips of urine sediment. These innovative delivery methods create a testing environment that more closely mimics the real life workflow in urinalysis laboratories. This has always been the goal of proficiency testing – make the assessment as close as possible to how a real patient sample is handled.
The authors would like to acknowledge the assistance of Carol Colasacco, SCT(ASCP), MLIS, AHIP, Medical Librarian at the College of American Pathologists, who helped gather appropriate references. Douglas Murphy MT(ASCP) and Caryn Tursky at the CAP also provided valuable support.
Lusky K. PT referral raising red flags with CMS. CAP Today 2008;22:March Google Scholar
Wallin O, Söderberg J, Van Guelpen B, Stenlund H, Grankvist K, Brulin C. Preanalytical venous blood sampling practices demand improvement – a survey of test request management, test-tube labeling, and information search procedures. Clin Chim Acta 2008;391:91–7. CrossrefGoogle Scholar
College of American Pathologists, Commission on Laboratory Accreditation (revision 9/27/2007) Laboratory General Checklist. Northfield, IL, USA: College of American Pathologists, 2007. Google Scholar
Haber MH, Galagan K, Blomberg D, Glassy EF, Ward PC, editors. Color atlas of urinary sediment; an illustrated field guide based on proficiency testing. Chicago: CAP Press, 2010. Google Scholar
Van den Bruel A, Cleemput I, Aertgeerts B, Ramaekers D, Buntinx F. The evaluation of diagnostic tests: evidence on technical and diagnostic accuracy, impact on patient outcome and cost effectiveness is needed. J Clin Epidemiol 2007;60:1116–22. CrossrefPubMedWeb of ScienceGoogle Scholar
Fogazzi GB. The urinary sediment. An integrated view, 3rd ed. Milano: Elsevier, 2010. Google Scholar
Fogazzi GB, Pallotti F, Garigali G. Atypical/malignant urothelial cells in routine urinary sediment: worth knowing and reporting. Clin Chim Acta 2015;439:107–11. CrossrefWeb of SciencePubMedGoogle Scholar
About the article
Published Online: 2015-08-11
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Financial support: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.