Sigma metrics of Alinity ci system – a study on thirty-nine clinical chemistry and immunoassay parameters

Abstract Objectives Sigma metrics in an invaluable and inexpensive tool used in laboratories to monitor analytical quality of the assays. Alinity ci platform is a relatively recent analytical system launched by Abbott Diagnostics, and as such performance studies on it are few. We have calculated sigma metrics of 39 clinical chemistry and immunoassay analytes on two Alinity ci systems. Methods Sigma metrics were calculated using results of method validation studies. Coefficient of variation (CV) was calculated according to CLSI EP 15 guidelines. Bias was calculated using three different methods i.e., proficiency testing material, alternate method comparison with existent analyzers and linearity experiment. Total allowable error limits were kept similar to or less than the ones used in reference studies. Results All analytes except blood urea nitrogen (BUN) demonstrated greater than six sigma value across one or more levels and methods. No analyte amongst clinical chemistry and immunoassays was at below three sigma class. Amongst electrolytes, sodium was below three sigma class at two levels by proficiency testing method, although it was above four sigma class by other two methods. Sigma levels obtained were comparable to those reported in previously published studies. Conclusions Acceptable sigma metrics were achieved for all clinical chemistry, immunoassays and electrolytes on Alinity ci. Sigma metrics is an objective and well established cost effective tool to tailor internal quality control practices. This study determines sigma metrics for a wide range of high throughput assays. Long term assay performance needs to be monitored.


Introduction
Sigma metric analysis is a well-established tool for quality assessment in industry which is now being increasingly used in clinical laboratories [1]. It is based on the concept of defects per million opportunities and aims to minimize errors in processes [2]. In the clinical laboratory, the internal and external quality control statistics are often used to calculate measurable values for sigma calculations based on the precision and accuracy of the assay. Clinical chemists and laboratorians have used various approaches to assess and monitor their laboratory performance based on this technique. The precision is generally calculated by the coefficient of variation generated by the daily analysis of the different internal quality control levels used for the assays. The accuracy is studied by calculating the bias, usually from the peer group values on Proficiency Testing schemes, although other approaches have also been used [3,4]. Another variable in the calculation of Sigma-metrics is the total allowable error (TEα) limit chosen. There are different sources available which have proposed various TEα for analytes based on optimum analytical performance [5][6][7][8], clinical guidelines [9] or biological variation [10]. A summary of some of these available sources is given in Supplementary Table S1.
The major advantage of sigma-metrics is that it provides a quantifiable and measurable tool against which assay performance can be objectively assessed and monitored. Six sigma and above are taken as world class standard, in which there are just under four defects per million opportunities. The minimum acceptable standard for a clinical laboratory assay is a three-sigma level. This approximates to 67,000 defects per million opportunities, below which the assays are not deemed clinically safe for patient testing [11][12][13].
Another important benefit of calculating the sigma metrics of the laboratory assays is that as we objectively measure quality of the assays, we can tailor the internal quality control (IQC) frequencies accordingly. An assay with high precision and accuracy, giving high sigma values requires fewer internal quality controls per run, more samples per run and fewer Westgard rules to adhere to. On the other hand, assays with higher imprecision or bias require more internal quality controls, smaller runs with fewer samples and more Westgard rules to follow. This is the concept advocated by Westgard in the newer Westgard Sigma Rules [14,15] introduced. Thus this contributes to significant savings in terms of labor and cost for the well-established assays with a more focused approach for the troubled ones. Cost is a major constraint not only in the under developed regions of the world, but also for commercial laboratories elsewhere. The economy crunch which has started to engulf the various countries, especially so in this pandemic era, has made it apparent that organizations need to come up with cost-effective approaches in their running operations. Sigma metric is one such approach.
In this study, we have used three different methods to calculate sigma metrics of Alinity ci series in which bias has been calculated from the proficiency testing material data, method correlation and linearity experiment data generated during method validation studies. We have calculated sigma metrics of the assays performed on Alinity ci during initial validations, and this is part of an ongoing continuous exercise aimed at tailoring the use of internal quality controls.

Materials and methods
This study was not related to either human or animal use and, therefore, was exempted from approval by the Institutional Review Board. It was conducted on 39 parameters across two Alinity c and two Alinity i analyzers during the months of June to September, 2018. Two sets of Alinity ci series on a3600 track were installed with pre and post analytics as part of total laboratory automation of the core clinical chemistry laboratory, and the data was generated during the method validation exercises performed on these analysers.
Alinity analytical platform was launched by Abbott Diagnostics in 2017. Alinity c is a totally automated clinical chemistry analyzer with indirect ion sensitive electrodes on integrated chip technology (ICT) for electrolytes, while Alinity i system is an immunoassay analyzer using paramagnetic microparticles and chemiluminescent detection based on Acridinium labeled conjugate. The clinical chemistry and electrolyte parameters included albumin, alkaline phosphatase (ALKP), alanine aminotransferase (ALT), amylase, aspartate aminotransferase (AST), blood urea nitrogen (BUN), calcium, chloride, cholesterol, CO 2 , creatinine phosphokinase (CPK), creatinine, direct bilirubin, gammaglutamyl transferase (GGT), glucose, high density lipoprotein cholesterol (HDL-C) iron, lactate dehydrogenase (LDH), low density lipoprotein cholesterol (LDL-C), magnesium, phosphorus, potassium, sodium, total bilirubin, total protein, triglycerides, and uric acid, while the immunoassay parameters were alfa-fetoprotein (AFP) beta-human chorionic gonadotrophin (β-HCG), free tri-iodothyronine (FT 3 ), free thyroxine (FT 4 ), ferritin, follicle stimulating hormone (FSH), luteinizing hormone (LH), prolactin, thyroid stimulating hormone (TSH), total prostate specific antigen (PSA), and vitamins B12 and D. Sigma metrics of all analytes was calculated on both sets of analyzers, and the average was considered for the study purpose. Sigma metrics was calculated by the following formula: Σσ = (TEα -Bias)/CV (where Σσ is Sigma metric, TEα is the total allowable error and CV is the coefficient of variation). A. Total allowable error: All values for TEα were taken from Clinical and Laboratory Improvement Act (CLIA), College of American Pathologists (CAP) and National Cholesterol Education Program (NCEP), except for vitamin D for which TEα was taken from a reference study on biological variations (Table S1) [10]. TEα were kept similar or lower than the ones used in comparable studies [16,17]. B. Precision: Three levels of commercial quality controls were run in five replicates for five days on both Alinity ci instruments according to CLSI EP 15 guidelines [18] and coefficient of variation (CV%) was calculated. C. Bias: Bias was calculated by three different methods:

Proficiency testing
Proficiency testing material of seven to ten levels obtained from College of American Pathologists were previously analyzed and stored at −40°C. These were run on both Alinity c and i analyzers in three replicates. Results were compared to Abbott Architect peer group mean as Alinity was till the time of the study not included in CAP surveys. Bias was calculated as [(target mean − observed mean/target mean) × 100]. Cumulative bias was obtained by taking an average of all the biases.

Linearity experiment
Linearity studies were performed on all assays by running three replicates across seven levels using either commercial linearity material (clinical chemistry and electrolytes) or patient samples (immunoassay), except for analytes with multipoint calibrators. Bias was calculated from the difference between the expected and mean of observed values as given above. Bias and sigma metrics calculated from the linearity experiment are shown in Supplementary Table S2.

Results
Concentration levels, precision, coefficient of variations and bias of these analytes are shown in Table 1. As can be seen, precision of all analytes are equal or less than 5.0%, while variability exists in the biases by different methods. Precision of clinical chemistry parameters and electrolytes is higher than immunoassays. Detailed sigma metric analysis is given in Table 2. In clinical chemistry, all parameters except BUN, demonstrated greater than six sigma values in one or more levels while BUN, CO 2 , creatinine, phosphorous and triglycerides gave values of less than Six Sigma in one or more levels and methods. Amongst immunoassays ferritin, prolactin and PSA demonstrated greater than six sigma values across all levels and methods, while other parameters demonstrated more than six sigma values at one or more levels across the different methods. No parameter in clinical chemistry or immunoassays was below three sigma level. Within electrolytes, potassium stood at greater than six sigma class across all levels and methods, while sodium was the only parameter to fall at below three sigma value at two levels in the proficiency testing method ( Table 2).
Sigma metrics achieved in this study are comparable with other studies on Alinity as evident in Table 3. The Method Decision Charts (MEDx) made using Matplotlib package in Python3.6 at three concentration levels by the three different methods are displayed in Figure 1 and Figure S1.

Discussion
Sigma metrics analysis has emerged as an easy to use and objective tool for monitoring and "quantitating" the quality of test assays. It can be calculated easily from routine quality control values generated in the laboratories at no additional expense. Moreover, it can be implemented as a tool to tailor internal quality control policy and to reduce the daily frequency and number of controls, serving as a cost reducing activity. In the current financial crisis gripping the world where laboratories are struggling to balance the quality monitoring measures with the expenses entailed, application of sigma metric based QC rules finds a new dimension in their utility.
Our initial validation studies on Alinity ci (Abbott Diagnostics, Abbott Gmbh & Co. KG, Wiesbaden, Germany) was very promising and demonstrated high sigma metrics for majority of the assays. All three methods performed well, but the results were lower with the proficiency testing method. This could be because the PT samples were stored at −40°C before use. To overcome this challenge, sample integrity was first confirmed by running the samples on the existent analyzers and comparing the results to the participant summary results. They were then repeated on Alinity ci within 2 h. Although we used proficiency testing, alternate method comparison and linearity studies to generate sigma metrics, we recommend proficiency testing as a more standardized tool to compare one's method with other laboratories around the globe on a continuous basis. Alternate method comparison is useful during the initial assessment phase only and at times difficult to adopt in immunoassays, while linearity studies are more useful for measuring range and calibration verification studies than for bias estimation.
A major determinant of the sigma metric calculation is the choice of TEα [19]. The more expansive the limit, the better sigma would be achieved, giving a false sense of security and over estimation of the assay quality. The use of stringent limits reduces the sigma levels and gives a better and truer picture of the assay quality. We have tried to use similar to or lower TEαs than the comparative studies on Alinty ci. Westgard [17] and Tahir et al. [16] have defined a sequence whereby they have used lowest realistically possible error limits of the assays. We have kept the same limits in order to compare sigma scale achieved by us with the initial analyses done within factory settings or in the field by company researchers. In certain assays such as direct bilirubin (20 vs. 44.5%), triglycerides (15 vs. 25%) and prolactin (20 vs 29.4%), we have chosen lower allowable error limits than the ones selected in the reference studies, yet achieved six sigma levels. The total allowable goals put forward by CLIA in 2019 are a step to raise the bar even more and we intend to use these in our further assessments.
Clinical chemistry analytes generally demonstrated better sigma levels than immune assay parameters. This was because of the higher precision inherent in these assays compared to immunoassays. The variability in the sigma levels across different methods was due to the variation in the biases generated, as the precision used in the calculations was constant, being derived from the internal quality controls data. > ≥- < . ALB, albumin; ALKP, alkaline phosphatase; ALT, alanine aminotransferase; AMY, amylase; AST, aspartate aminotransferase; Ca, calcium; CHOL, cholesterol; CO  , carbon dioxide; CPK, creatinine phosphokinase; CREA, creatinine; DB, direct bilirubin; GGT, gamma glutamyl transferase; GLU, glucose; HDL, high density lipoprotein; LDH, lactate dehydrogenase; LDL, low density lipoprotein; Mg, magnesium; P, phosphorus; Total PSA, total prostatic specific antigen; TB, total bilirubin; TG, triglycerides; TP, total protein; UA, uric acid; AFP, alpha fetoprotein; βHCG, beta human chorionic gonadotrophin; FER, ferritin; FSH, follicle stimulating hormone; FT, free triiodothyronine; FT, free thyroxine; LH, luteinizing hormone; TSH, thyroid stimulating hormone; VIT B, vitamin B; VIT D, vitamin D; Cl, chloride; K, potassium; Na, sodium; CLIA  was the source of total allowable errors for various analytes     There were few limitations to the study. Firstly, the proficiency testing results were compared to Architect peer group due to the fact that Alinity CAP participation results were not existent at the time of the study. Secondly, the study was performed over a short span of time with EP 15 for generating the precision data and by using a single lot of reagents, calibrators and controls. The experience now after going live with these assays is a different story with real time peer group analysis in CAP surveys, and varied lots of reagents, calibrators, controls, environmental conditions and operators.

Conclusions
Alinity ci series generated acceptable sigma metrics for all routine clinical chemistry and immunoassay parameters. Sigma metric calculation is a practical tool for assessing the quality of an analyte based on its precision and accuracy which can help the laboratories to modify and taper the IQC frequencies. It enables laboratories to work in a more cost effective manner without compromising on patient safety. The present study was conducted using different total allowable errors limits. However, with the introduction of the revised and restricted 2019 CLIA total allowable error goals, we propose that laboratories should universally adopt these to calculate their sigma metrics.