Multi-site performance evaluation and Sigma metrics of 20 assays on the Atellica chemistry and immunoassay analyzers

Tommaso Fasano 1 , Jose Luis Bedini 2 , Pierre-Antoine Fle 3 , Malik Jlaiel 3 , Karl Hubbert 4 , Harish Datta 4 , Valerie Chicha-Cattoir 5 , Hayfa Mansour 6 , Aurea Mira 2 , Bernardino González de la Presa 2 , Nayra Rico 2 , Maria Sanz de Pedro 7 , Jorge Diaz-Garzon 7 , Antonio Buño Soto 7 , Luigi Vecchia 1  and Katell Peoc’h 5
  • 1 Azienda Unità Sanitaria Locale – IRCCS, Reggio Emilia, Italy
  • 2 Hospital Clínic, Barcelona, Spain
  • 3 Laboratoire de Biologie Médicale Bioesterel, Mandelieu la Napoule, France
  • 4 James Cook University Hospital, Middlesbrough, UK
  • 5 Hôpital Beaujon, APHP, HUPNVS, Clichy, France
  • 6 Hôpital Bichat, APHP, HUPNVS, Paris, France
  • 7 Hospital Universitario La Paz, Madrid, Spain
Tommaso Fasano, Jose Luis Bedini, Pierre-Antoine Fle, Malik Jlaiel, Karl Hubbert, Harish Datta, Valerie Chicha-Cattoir, Hayfa Mansour, Aurea Mira, Bernardino González de la Presa, Nayra Rico, Maria Sanz de Pedro, Jorge Diaz-Garzon, Antonio Buño Soto, Luigi Vecchia and Katell Peoc’h

Abstract

Background

The Atellica Solution comprises chemistry (CH) and immunoassay (IM) analyzers. Recently, six early adopter clinical laboratories across Europe evaluated the analytical performance of 20 CH and IM assays. To measure analytical performance quality, Sigma metrics were calculated for individual-site and pooled-site results.

Methods

Precision, detection capability, linearity, and method comparison studies were performed according to Clinical Laboratory Standards Institute protocols. Global Sigma metrics across sites were calculated from pooled data at the medical decision level using total allowable error (TEa) goals from CLIA for CH assays, and TEa goals from RiliBÄK for IM assays; and, the equation:

Sigma metrics=%TEa–%bias/%CV.

A pooled %CV was calculated by combining the imprecision obtained from individual sites. Bias calculations were performed against the ADVIA Chemistry system or ADVIA Centaur system using Deming regression analysis (Passing-Bablok regression for electrolytes) on the pooled-site data. The 103 individual-site Sigma metric calculations used individual-site imprecision and pooled-bias.

Results

The limits of blank and detection results agreed with the manufacturer’s claims. Most assays were linear across the assay range tested. Pooled Sigma metrics were good or better (>4 Sigma) for 18 of 20 assays; and, acceptable for urea nitrogen (3.1) and sodium (3.9), the latter values attributable to higher imprecision at one of five sites.

Conclusions

Sigma metrics for data generated across multiple real-world sites evaluating the Atellica Solution demonstrated good or better performance of greater than 4 Sigma for 18 of 20 assays tested. Overall, results verified the manufacturer’s claims that methods were fit for use in clinical laboratories.

    • Supplementary material
  • 1.

    Rohr UP, Binder C, Dieterle T, Giusti F, Messina CG, Toerien E, et al. The value of in vitro diagnostic testing in medical practice: a status report. PLoS One 2016;11:e0149856.

    • Crossref
    • Export Citation
  • 2.

    Topic E, Nikolac N, Panteghini M, Theodorsson E, Salvagno GL, Miler M, et al. How to assess the quality of your analytical method? Clin Chem Lab Med 2015;53:1707–18.

    • PubMed
    • Export Citation
  • 3.

    Nichols JH. Verification of method performance for clinical laboratories. Adv Clin Chem 2009;47:121–37.

    • Crossref
    • PubMed
    • Export Citation
  • 4.

    ISO 15189, Medical laboratories – requirements for quality and competence (ISO 15189:2012, Edition 3; Corrected version 2014-08-15).

  • 5.

    Westgard S, Bayat H, Westgard JO. Analytical Sigma metrics: a review of Six Sigma implementation tools for medical laboratories. Biochem Med (Zagreb) 2018;28:020502.

    • Crossref
    • PubMed
    • Export Citation
  • 6.

    Aarsand AK, Roraas T, Fernandez-Calle P, Ricos C, Diaz-Garzon J, Jonker N, et al. The biological variation data critical appraisal checklist: a standard for evaluating studies on biological variation. Clin Chem 2018;64:501–14.

    • Crossref
    • PubMed
    • Export Citation
  • 7.

    Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A,Jimenez CV, et al. Current databases on biological variation: pros, cons and progress. Scand J Clin Lab Invest 1999;59:491–500.

    • Crossref
    • PubMed
    • Export Citation
  • 8.

    U.S. Centers for Medicare & Medicaid Services (CMS). Medicare, Medicaid, and CLIA programs: laboratory requirements relating to quality systems and certain personnel qualifications. Final Rule. Fed Regist 2003;16:3650–714.

  • 9.

    College of American Pathologists (CAP). Proficiency Testing Programs. http://www.cap.org/web/home/lab/proficiency-testing. Accessed: 11 Jun 2019.

  • 10.

    German Medical Association on Quality Assurance in Medical Laboratory Examinations (Rili-BAEK). Revision of the “Guideline of the German Medical Asociation on Quality Assurance in Mdical Laboratory Examinations Rili-BAEK” (unauthorized translation). Lab Med 2015;39:26–69.

  • 11.

    Siemens Healthcare Diagnostics. Atellica® CH Albumin (Alb) BCG Assay instructions for use. 11110153_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 12.

    Siemens Healthcare Diagnostics. Atellica® CH Alanine Aminotransferase (ALT) Assay instructions for use. 11110168_EN Rev. 03. Tarrytown, NY: Siemens, 2017.

  • 13.

    Siemens Healthcare Diagnostics. Atellica® CH Aspartate Aminotransferase (AST) Assay instructions for use. 11110170_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 14.

    Siemens Healthcare Diagnostics. Atellica® CH Calcium (Ca) Assay instructions for use. 11110158_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 15.

    Siemens Healthcare Diagnostics. Atellica® CH A-LYTE Integrated Multisensor (IMT Na K Cl) Assay instructions for use. 11109447_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 16.

    Siemens Healthcare Diagnostics. Atellica® CH Creatinine_2 (Crea_2) Assay instructions for use. 11110159_EN Rev. 03. Tarrytown, NY: Siemens, 2017.

  • 17.

    Siemens Healthcare Diagnostics. Atellica® CH Glucose Hexokinase_3 (GluH_3) Assay instructions for use. 11110155_EN Rev. 02. Tarrytown, NY: Siemens, 2017.

  • 18.

    Siemens Healthcare Diagnostics. Atellica® CH Total Bilirubin_2 (TBil_2) Assay instructions for use. 11110149_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 19.

    Siemens Healthcare Diagnostics. Atellica® CH Total Protein II (TP) Assay instructions for use. 11110167_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 20.

    Siemens Healthcare Diagnostics. Atellica® CH Triglycerides (Concentrated) (Trig) Assay instructions for use. 11110154_EN Rev. 02. Tarrytown, NY: Siemens, 2017.

  • 21.

    Siemens Healthcare Diagnostics. Atellica® CH Urea Nitrogen (UN_c) Assay instructions for use. 11110156_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 22.

    Siemens Healthcare Diagnostics. Atellica® IM Enhanced Estradiol (eE2) Assay instructions for use. 10995342_EN Rev. 03. Tarrytown, NY: Siemens, 2017.

  • 23.

    Siemens Healthcare Diagnostics. Atellica® IM Ferritin (Fer) Assay instructions for use. 10995343_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 24.

    Siemens Healthcare Diagnostics. Atellica® IM Progesterone (PRGE) Assay instructions for use. 10995415_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 25.

    Siemens Healthcare Diagnostics. Atellica® IM Prostate-Specific Antigen (PSA) Assay instructions for use. 10995416_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 26.

    Siemens Healthcare Diagnostics. Atellica® IM Total hCG (ThCG) Assay instructions for use. 10995416_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 27.

    Siemens Healthcare Diagnostics. Atellica® IM Thyroid Stimulating Hormone 3-Ultra (TSH3-UL) Assay instructions for use. 10995434_EN Rev. 01. Tarrytown, NY: Siemens, 2017.

  • 28.

    Siemens Healthcare Diagnostics. Atellica® IM Testosterone II (TSTII) Assay instructions for use. 10995435_EN Rev. 03. Tarrytown, NY: Siemens, 2018.

  • 29.

    Law SJ, Miller T, Piran U, Klukas C, Chang S, Unger J. Novel poly-substituted aryl acridinium esters and their use in immunoassay. J Biolumin Chemilumin 1989;4:88–98.

    • Crossref
    • PubMed
    • Export Citation
  • 30.

    Lee JH, Rho JE, Rho TH, Newby JG. Advent of innovative chemiluminescent enzyme immunoassay. Biosens Bioelectron 2010;26:377–82.

    • Crossref
    • PubMed
    • Export Citation
  • 31.

    Natrajan A, Wen D. A comparison of chemiluminescent acridinium dimethylphenyl ester labels with different conjugation sites. Org Biomol Chem 2015;13:2622–33.

    • Crossref
    • PubMed
    • Export Citation
  • 32.

    Clinical and Laboratory Standards Institute. User Verification of Performance for Precision and Trueness; Approved Guideline – Third Edition. Wayne, PA: Clinical and Laboratory Standards Institute, 2014. CLSI Document EP15-A3.

  • 33.

    Clinical and Laboratory Standards Institute. Measurement Procedure Comparison and Bias Estimation Using Patient Samples; Approved guideline – Third Edition. Wayne, PA: Clinical and Laboratory Standards Institute, 2013. CLSI Document EP09-A3.

  • 34.

    Taher J, Cosme J, Renley BA, Daghfal DJ, Yip PM. A novel Sigma metric encompasses global multi-site performance of 18 assays on the Abbott Alinity system. Clin Biochem 2019;63:106–12.

    • Crossref
    • PubMed
    • Export Citation
  • 35.

    Clinical and Laboratory Standards Institute. Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline-Second Edition. Wayne, PA: Clinical and Laboratory Standards Institute, 2012. CLSI document EP17-A2.

  • 36.

    Clinical and Laboratory Standards Institute. Evaluation of the Linearity of Quantitative Measurement Procedures; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute, 2003. CLSI Document EP06-A.

  • 37.

    Clinical and Laboratory Standards Institute. Evaluation of Precision of Quantitative Measurement Procedures; Approved Guideline – Third Edition. Wayne, PA: Clinical and Laboratory Standards Institute, 2014. CLSI Document EP05-A3.

  • 38.

    Passing H, Bablok. A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, Part I. J Clin Chem Clin Biochem 1983;21:709–20.

    • PubMed
    • Export Citation
  • 39.

    Chesher D. Evaluating assay precision. Clin Biochem Rev 2008;29 Suppl 1:S23–6.

    • PubMed
    • Export Citation
  • 40.

    Ricos C, Alvarez V, Minchinela J, Fernandez-Calle P, Perich C, Boned B, et al. Biologic variation approach to daily laboratory. Clin Lab Med 2017;37:47–56.

    • Crossref
    • PubMed
    • Export Citation
  • 41.

    CLIA & Quality, Quality Requirements. Desirable biological variation database specifications. https://www.westgard.com/biodatabase1.htm. Last updated 2014. Accessed: 23 Jul 2018.

  • 42.

    Perich C, Minchinela J, Ricos C, Fernandez-Calle P, Alvarez V, Domenech MV, et al. Biological variation database: structure and criteria used for generation and update. Clin Chem Lab Med 2015;53:299–305.

    • PubMed
    • Export Citation
  • 43.

    Aarsand AK, Diaz-Garzon J, Fernandez-Calle P, Guerra E, Locatelli M, Bartlett WA, et al. The EuBIVAS: within- and between-subject biological variation data for electrolytes, lipids, urea, uric acid, total protein, total bilirubin, direct bilirubin, and glucose. Clin Chem 2018;64:1380–93.

    • Crossref
    • PubMed
    • Export Citation
Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

Clinical Chemistry and Laboratory Medicine ( CCLM) publishes articles on novel teaching and training methods applicable to laboratory medicine. CCLM welcomes contributions on the progress in fundamental and applied research and cutting-edge clinical laboratory medicine. It is one of the leading journals in the field, with an impact factor of over three. CCLM is the official journal of nine national clinical societies and associated with EFLM.

Search