Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Clinical Chemistry and Laboratory Medicine (CCLM)

Published in Association with the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM)

Editor-in-Chief: Plebani, Mario

Ed. by Gillery, Philippe / Greaves, Ronda / Lackner, Karl J. / Lippi, Giuseppe / Melichar, Bohuslav / Payne, Deborah A. / Schlattmann, Peter


IMPACT FACTOR 2018: 3.638

CiteScore 2018: 2.44

SCImago Journal Rank (SJR) 2018: 1.191
Source Normalized Impact per Paper (SNIP) 2018: 1.205

Online
ISSN
1437-4331
See all formats and pricing
More options …
Volume 57, Issue 10

Issues

Trueness assessment of HbA1c routine assays: are processed EQA materials up to the job?

Vincent Delatour / Noémie Clouet-Foraison
  • Laboratoire National de Métrologie et d’Essais (LNE), Paris, France
  • University Hospital of Reims, Laboratory of Biochemistry, Reims, France
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Stéphane Jaisson / Patricia Kaiser / Philippe Gillery
Published Online: 2019-05-14 | DOI: https://doi.org/10.1515/cclm-2019-0219

Abstract

Background

With the worldwide increase of diabetes mellitus prevalence, ensuring that HbA1c assays are accurate is essential. External quality assessment (EQA) programs enable laboratories to verify that analytical methods perform according to the manufacturers’ specifications. However, assessing trueness requires commutable materials, a property that is rarely characterized for EQA materials.

Methods

The difference in bias approach was used to assess commutability of 26 processed quality control materials for 17 of the most frequently used HbA1c assays. Involved assays included immuno-assays, enzymatic assays, affinity, ion-exchange HPLC boronate affinity HPLC and capillary electrophoresis. The measurements were performed at manufacturers or expert laboratories. Assay trueness was additionally assessed against the IFCC reference measurement procedure using fresh clinical specimens that were distributed to 450 medical laboratories.

Results

Commutability of processed EQA materials was highly heterogeneous and globally insufficient to rigorously assess the trueness of HbA1c assays. Using fresh clinical specimens, mean bias was −0.13 mmol/mol for low HbA1c (34 mmol/mol), between +1.0 and +1.3 mmol/mol for intermediate HbA1c (49 and 58 mmol/mol) and +1.2 mmol/mol for elevated HbA1c (90 mmol/mol).

Conclusions

This study demonstrates that due to insufficient commutability, most processed EQA materials are unsuitable to assess trueness of HbA1c assays and agreement between the different assays. These materials can only provide information on comparability of individual laboratory results with its peers and on assay precision. Using fresh whole blood samples, this study additionally shows that most HbA1c assays are fairly accurate and meet the total allowable error quality target of 5 mmol/mol.

This article offers supplementary material which is provided at the end of the article.

Keywords: accuracy; accuracy-based program; commutability; comparability; external quality assessment schemes; glycated hemoglobin; HbA1c

Introduction

Diabetes mellitus is the seventh major cause of mortality worldwide [1] and is characterized by hyperglycemia. Hyperglycemia favors non-enzymatic glycation, a slow process that affects all proteins in the blood and tissues and alters their conformation and function, ultimately participating in micro- and macrovascular complications [2], [3]. Hemoglobin is particularly prone to non-enzymatic glycation and HbA1c, the major form of glycated hemoglobin, is a well-known biomarker of diabetes mellitus [4], [5]. As it reflects the mean glycemia over erythrocyte lifetime (approx. 120 days), HbA1c testing is widely used to ensure treatment adequacy and observance of patients with diabetes. Since 2009, HbA1c is also recommended as a diagnostic marker of diabetes [6], [7].

Given the prevalence of diabetes and the role of HbA1c measurements in patient care, evaluating the trueness of HbA1c assays is of prime importance. In addition to internal quality controls, participation in external quality assessments schemes (EQAS) helps to verify on a recurring basis that laboratory results meet quality requirements [8]. As a measure of accuracy, a pair-wise comparison of individual laboratory results with the consensus mean of its peers or all participants, is very often used. It has been recommended that, if a reference measurement procedure (RMP) exists, method trueness should be estimated against the target value determined by the RMP [8]. However, a prerequisite to evaluate trueness is that materials must be commutable for the concerned assay [9], [10], [11], [12].

Commutability is defined as the equivalence of mathematical relationships between the results obtained with different measurement procedures for a reference material or an EQA material and for representative samples from healthy and diseased individuals [13]. It thus reflects the ability of the material to mimic clinical specimens (CSs). Although guidelines to produce commutable materials exist for cholesterol in serum [14], following CLSI C-37A guidelines does not per se guarantee the commutability of the material for other parameters and matrices. In the case of HbA1c, EQA materials generally consist in lyophilized hemolysates or stabilized whole blood. As commutability is a property of a material for a given measurement procedure, it cannot be predicted, and a material found commutable for one assay may not be commutable for another one.

This article presents the results of a commutability study performed on 26 processed quality control materials for 17 of the most frequently used HbA1c assays and the impact of materials commutability on trueness assessment and between-methods comparability [15]. Trueness of frequently used HbA1c assays against the IFCC RMP is additionally presented.

Materials and methods

Commutability study

Commutability of processed EQA materials was assessed against a set of CSs consisting of fresh whole blood. Twenty-four diabetic and non-diabetic donors were selected to achieve a wide range of HbA1c concentrations so as to bracket the HbA1c concentrations of the EQA materials under investigation. CSs were collected at Solomon Park Research Laboratories (Kirkland, WA, USA) in EDTA tubes (Becton Dickinson, NJ, USA). Blood collection was performed in agreement with human subject protection laws and following IRB approved protocol. Freshly collected CSs were divided in two parts. A first half was kept fresh and stored at +4 °C until analysis. The second half was frozen immediately after collection and stored at −80 °C. HbA1c concentrations of the 24 CSs ranged from 30 mmol/mol (4.9% National Glycohemoglobin Standardization Program; NGSP) to 120 mmol/mol (13.1% NGSP), with a mean concentration of 55 mmol/mol (7.2% NGSP). Fresh CSs were collected from 23rd to 26th May 2017 and were shipped to participating laboratories on the 27th, in insulated boxes, on ice packs and under controlled temperature conditions (+4 °C). They were delivered on 30th May to European laboratories and on 31st May to US laboratories. Samples were assayed between 31st May and 2nd June. Concentrations of HbS, HbA2, HbC, HbE and HbF were measured in all single donations (Supplementary Data S1) [16]. Elevated concentrations of HbS were detected in CSs 012 and 013 and these samples were excluded from data analysis.

EQA materials in which commutability was evaluated consisted of frozen stabilized whole blood or lyophilized whole blood. EQA materials were obtained from different EQA providers that did not all agree to disclose from which manufacturers samples were sourced but it can be expected that the different materials were produced using different manufacturing procedures. According to the information obtained from some EQA providers, there were sometimes different lots from the same producer, but this information could not be obtained for all the materials involved in this study. It should be noted that none of the EQA materials have indications for use as a trueness control or to evaluate accuracy.

Commutability of the 26 EQA materials was assessed for 17 HbA1c assays: immunoassays from Roche (Cobas C513), Beckman Coulter (DxC700AU B00389) and Siemens (DCA Vantage), enzymatic assay from Abbott (Architect C-4000), ion-exchange HPLC assays from Bio-Rad (D10, D100, Variant II NU and Variant II Turbo), Tosoh (G8, G11 and Gx) and Arkray (HA-8180T), affinity assay from Abbott (Abbott Afinion AS-100), boronate affinity HPLC assay from Trinity Biotech (Premier Hb9210) and capillary electrophoresis assays from Sebia (Capillarys 2, Capillarys 3 and Minicap). Detailed information on analyzers, reagent lots, calibrators and laboratories involved in the commutability study are provided in Supplementary Data S2. All samples (EQA materials, fresh CSs and frozen CSs) were analyzed in triplicates by all participants, altogether in a unique sequence. Measurements were performed in manufacturers’ quality control or R&D laboratories, except for Arkray, Trinity Biotech, Abbott and Siemens (measurements performed in expert laboratories are listed in Supplementary Data S2). All measurements were performed according to manufacturers’ instructions. To ensure that methods performed according to specifications, the calibration of each system was verified by measuring quality control (QC) materials provided by assay manufacturers 5 times during the analytical sequence.

Trueness assessment

Trueness was assessed using four trueness control materials consisting of single donations of fresh whole blood. Four donors were selected in order to obtain four materials at three concentration levels: low HbA1c (34 mmol/mol, i.e. 5.3% NGSP), medium HbA1c close to the clinical decision limit (49 and 58 mmol/mol, i.e. 6.6% and 7.4% NGSP, respectively) and high HbA1c (90 mmol/mol, i.e. 10.3% NGSP). Samples were collected at Queen Beatrix Hospital (Winterswijk, The Netherlands) in EDTA tubes and stored at +4 °C until shipment.

Three campaigns were organized: January (one sample), March (two samples) and June 2017 (one sample). Each time, blood was drawn from volunteers on Monday and stored at +4 °C until shipping to study participants on Tuesday. Participants performed and reported measurements no later than Friday. Homogeneity and stability of the materials were tested according to the requirements of ISO 17043 and ISO 13528 standards.

For each campaign, samples were analyzed once in approximately 450 medical laboratories participating to the Instand e.V. EQA scheme [17]. Involved assays were partitioned in 11 peer groups including different combinations of analyzers and reagents for which calibration was identical: Abbott Architect, Abbott-Afinion System, Beckman AU, Beckman Unicel DxC, Bio-Rad Variant II, Menarini, Roche, Sebia Capillarys, Siemens DCA 2000, Siemens Dimension+ and Tosoh.

Reference method value assignment

All samples (EQA materials, trueness verifiers, fresh and frozen CSs) were analyzed with the LC-MS IFCC RMP for HbA1c quantification [18], [19]. Measurements were conducted at the Instand reference laboratory (Düsseldorf, Germany), a member of the IFCC reference network for HbA1c that is accredited according to ISO 17025 and ISO 15195 standards for HbA1c measurement. Assays were performed in two duplicates (two independent digestions, four values). Accuracy of the IFCC RMP was verified using QC materials obtained from the IFCC network. No significant deviation from the target value of the IFCC reference laboratory network was observed (see Supplementary Data S3).

Data analysis

HbA1c results reported in NGSP units in % were converted into IFCC units in mmol/mol using the master equation NGSP=[0.09148×IFCC]+2.152 [20].

Commutability of the 26 EQA materials was assessed according to the difference in bias approach described in the IFCC recommendations on commutability assessment [21], [22]. Briefly, this approach consists in calculating, for each assay, the difference between the bias of the assay against the IFCC RMP for the EQA material under investigation, and the mean bias against the IFCC RMP of the same assay for a panel of fresh CSs. This bias difference is then compared with a commutability acceptance criterion that corresponds to the maximum bias allowed to meet medical needs [23]. The results were categorized as commutable, inconclusive and non-commutable [21], [22]. In this study, HbA1c concentrations were ln-transformed and a fixed acceptance criterion of 6% was used [22].

The effect of freeze-thawing was evaluated by calculating the mean relative difference between HbA1c concentrations measured in the 22 fresh CSs and the corresponding frozen CSs.

Trueness was assessed by calculating the mean relative bias of the different peer groups against the IFCC RMP for each of the four trueness verifiers. Between-method imprecision was additionally evaluated by calculating the relative standard deviation across HbA1c concentrations measured with the 17 routine assays for the 26 EQA materials, as well as the fresh and frozen CSs involved in the commutability study. The mean of all the RSDs was then reported. Assay performance was additionally characterized using the model described by Weykamp et al. that consists in combining the biological variation [24] and the sigma metrics models [25]. All the research meets the ethical guidelines and comply with the World Medical Association Declaration of Helsinki regarding ethical conduct of research involving human subjects.

Results

Commutability assessment of 26 EQA materials

Commutability results of the 26 processed EQA materials for the 17 HbA1c assays are presented in Figure 1 (see also Supplementary Data S4A). It shows that commutability of EQA materials is highly heterogeneous. Only a few EQA materials were commutable for most assays and some materials were found non-commutable for most HbA1c assays, suggesting that commutability of processed EQA materials cannot be assumed a priori. Results additionally show that commutability of EQA materials was globally better for some assays than for others (see Supplementary Data S4B). For some assays like Abbott enzymatic, Siemens DCA Vantage and Bio-Rad Variant II NU, very few of the 26 EQA materials were found to be commutable. On the contrary, a larger fraction of EQA materials was found to be commutable for Tosoh, Arkray HA-8180T, Sebia Capillarys 3 and Roche Cobas C-513 assays. None of the EQA materials investigated were found to be commutable for the Abbott Afinion assay as it does not allow the measurement of coagulated or hemolyzed samples. As comparison point, the 22 fresh and frozen CSs were first examined as if they were processed materials. Altogether, the 22 fresh CSs were commutable in 87% of the pairwise comparisons and 13% of the statistical analyses were inconclusive (see Supplementary Data S4C). Altogether, the 22 frozen CSs were found to be commutable in 67% of the pairwise comparisons, non-commutable in 12% of them and 21% of statistical assessments were inconclusive (Supplementary Data S4D).

Commutability of 26 processed EQA materials and 22 fresh and frozen clinical specimens. The different HbA1c assays (columns) and EQA materials (rows) are listed. C stands for commutable; NC for non-commutable and I for inconclusive.
Figure 1:

Commutability of 26 processed EQA materials and 22 fresh and frozen clinical specimens.

The different HbA1c assays (columns) and EQA materials (rows) are listed. C stands for commutable; NC for non-commutable and I for inconclusive.

Impact of EQA materials commutability on between-method imprecision

As shown in Figure 2, the between-method imprecision was better with fresh and frozen CSs (between-methods RSD of 3.2±1.1% and 3.9±1.4%, respectively) than with processed EQA materials (RSD of 8.0±5.8%). This means that the non-commutability of processed EQA materials leads to underestimating how well the different HbA1c assays agree with each other. In addition, the agreement between HbA1c assays greatly varies across the different EQA materials, confirming that the heterogeneity in commutability levels results in highly discrepant and underestimated between-methods agreement [10]. This observation confirms that processed materials are not adequate to evaluate comparability between methods.

Between-method imprecision calculated across 17 HbA1c assays for the 22 fresh and frozen CSs and for the 26 HbA1c EQA materials investigated in the commutability study. Between-method relative standard deviation (RSDs) was calculated for the 22 fresh CSs (diamonds), the 22 frozen CSs (circles) and the 26 processed EQA materials including lyophilized and liquid materials (triangles). Error bars correspond to the SD calculated on between-methods RSDs within each sample category, i.e. fresh CSs, frozen CSs and processed EQA materials.
Figure 2:

Between-method imprecision calculated across 17 HbA1c assays for the 22 fresh and frozen CSs and for the 26 HbA1c EQA materials investigated in the commutability study.

Between-method relative standard deviation (RSDs) was calculated for the 22 fresh CSs (diamonds), the 22 frozen CSs (circles) and the 26 processed EQA materials including lyophilized and liquid materials (triangles). Error bars correspond to the SD calculated on between-methods RSDs within each sample category, i.e. fresh CSs, frozen CSs and processed EQA materials.

Impact of EQA materials commutability on trueness assessment

Relative deviations of results provided by the different assays against the IFCC RMP are detailed in Figure 3 depending on the commutability of the control materials. Results show that non-commutability of EQA materials greatly affects the trueness assessment of HbA1c assays. For most assays, a larger deviation against the IFCC RMP was almost always observed when non-commutable EQA materials were used. Interestingly, for some assays like Siemens DCA Vantage, Bio-Rad D100 and Sebia, non-commutability of EQA materials consistently resulted in a positive shift in bias, i.e. in overestimating bias. For example, for the Sebia Capillarys 3 assay, using non-commutable materials resulted in observing a mean bias of +17.1% while a bias of only +1.8% was observed when using commutable materials. On the contrary, for other assays like the Abbott enzymatic, non-commutability resulted in a negative shift in bias (+3.0% bias with commutable materials and −17.8% with non-commutable materials). These results show that non-commutability of EQA materials skews trueness assessment differently depending on the assay. Even though some assays are more affected than others by the matrix effects, using non-commutable materials consistently leads to measuring an erroneously large bias.

Impact of EQA materials commutability on trueness assessment of HbA1c assays. Relative deviation of results provided by 8 HbA1c assays against the IFCC RMP measured with EQA materials found commutable (white circles), inconclusive (gray dots) and non-commutable (black dots).
Figure 3:

Impact of EQA materials commutability on trueness assessment of HbA1c assays.

Relative deviation of results provided by 8 HbA1c assays against the IFCC RMP measured with EQA materials found commutable (white circles), inconclusive (gray dots) and non-commutable (black dots).

The fact that results obtained using frozen CSs are globally close to those obtained with fresh CSs was confirmed and was supported by the absence of the freeze-thawing effect. The mean relative difference between HbA1c concentrations measured in fresh and frozen CSs with eight assays and the IFCC RMP was −0.0±0.9% (Supplementary Data S5). It is noteworthy that most of the frozen CSs received by Sebia and Bio-Rad were partly or fully degraded during shipment. Analysis of the electrophoretic and chromatographic profiles showed that these samples were compromised and the corresponding data were excluded from the study, implying that the freeze-thawing effect could not be evaluated for Sebia and Bio-Rad assays.

Trueness assessment of HbA1c assays

Trueness assessment of HbA1c assays was performed via the Instand accuracy-based EQA program and the results are presented in Figure 4. The mean bias against the IFCC RMP across the 11 peer groups was −0.1 mmol/mol for the low HbA1c sample (34 mmol/mol, i.e. 5.3% NGSP), +1.3 mmol/mol and +1.0 mmol/mol, for the two medium HbA1c samples, respectively (49 and 58 mmol/mol, i.e. 6.6% and 7.4% NGSP) and +1.2 mmol/mol for the high HbA1c sample (90 mmol/mol, i.e. 10.3% NGSP). The mean bias across the four trueness verifiers ranged from −0.53 mmol/mol for the Siemens Dimension+ peer group to +2.15 mmol/mol for the Siemens DCA 2000 peer group. The TAE quality target of 5 mmol/mol (0.46% NGSP) suggested by Weykamp et al. was met for all methods except for the Siemens DCA 2000 peer group for the high trueness verifier (mean bias of +5.2 mmol/mol) and for the Siemens Dimension+ peer group for the high trueness verifier (mean bias of −5.1 mmol/mol) (Figure 4).

Trueness assessment of 11 HbA1c assays against the IFCC LC-MS RMP for HbA1c for four single donations consisting in fresh whole blood. The mean bias of each method is presented against the results obtained with the IFCC RMP for four fresh single donations consisting in fresh whole blood with HbA1c concentrations of 34 mmol/mol (i.e. 5.3% NGSP) (white bars), 49 mmol/mol (i.e. 6.6% NGSP) (light gray bars), 58 mmol/mol (i.e. 7.4% NGSP) (dark gray bars) and 90 mmol/mol (i.e. 10.3% NGSP) (black bars). Error bars correspond to the standard deviation associated to the mean absolute bias across the different laboratories of each given peer group, expressed in mmol/mol.
Figure 4:

Trueness assessment of 11 HbA1c assays against the IFCC LC-MS RMP for HbA1c for four single donations consisting in fresh whole blood.

The mean bias of each method is presented against the results obtained with the IFCC RMP for four fresh single donations consisting in fresh whole blood with HbA1c concentrations of 34 mmol/mol (i.e. 5.3% NGSP) (white bars), 49 mmol/mol (i.e. 6.6% NGSP) (light gray bars), 58 mmol/mol (i.e. 7.4% NGSP) (dark gray bars) and 90 mmol/mol (i.e. 10.3% NGSP) (black bars). Error bars correspond to the standard deviation associated to the mean absolute bias across the different laboratories of each given peer group, expressed in mmol/mol.

Assay performance

Each assay’s performance estimated using the model proposed by Weykamp et al. are presented in Figure 5. Using the Sigma Metrics model, four assays met the 4σ criterion (Abbott-Afinion System by affinity, Siemens Dimension+ by immunoassay, Menarini HA8160 and Sebia assays by capillary electrophoresis). All other assays met the 2σ criterion (TAE<5 mmol/mol). Using the biological variation model, four assays did not meet the minimum performance criterion (Beckman AU B00389, Beckman UniCel DxC, Siemens DCA 2000 immunoassays and Bio-Rad Variant II assays by ion-exchange HPLC) while the other assays met the minimum performance criterion. The Abbott-Afinion System was the only assay to meet the desirable performance criterion. No method met the optimal performance criterion.

Performance evaluation of 11 HbA1c assays with four trueness verifiers consisting in fresh whole blood and using the biological variation and sigma-metrics models for quality targets. The mean RSD calculated on triplicate analysis of the four trueness verifiers is represented on the x-axis for the 11 peer groups. The absolute bias is represented on the y-axis. Solid lines represent the performance criteria for a 5 mmol/mol (0.46% NGSP) total allowable error with 2σ, 4σ and 6σ acceptable ranges. Areas represent the biological variation model quality targets: minimum (Min.), desirable (Des.), optimum (Opt.). The different methods principles are represented as follows: immuno-nephelometry by empty circles, ion-exchange HPLC by diamonds, enzymatic assays by a black triangle, capillary electrophoresis by a gray square and IFCC RMP by a black star. Peer groups are identified by letters: (A) Beckman AU B00389, (B) Beckman UniCel DxC Synchron, (C) Roche Cobas C513, (D) Siemens DCA 2000, (E) Siemens Dimension+, (F) Abbott enzymatic, (G) Bio-Rad Variant II, (H) Menarini HA 8160, (I) Tosoh, (J) Abbott–Afinion System, (K) Sebia.
Figure 5:

Performance evaluation of 11 HbA1c assays with four trueness verifiers consisting in fresh whole blood and using the biological variation and sigma-metrics models for quality targets.

The mean RSD calculated on triplicate analysis of the four trueness verifiers is represented on the x-axis for the 11 peer groups. The absolute bias is represented on the y-axis. Solid lines represent the performance criteria for a 5 mmol/mol (0.46% NGSP) total allowable error with 2σ, 4σ and 6σ acceptable ranges. Areas represent the biological variation model quality targets: minimum (Min.), desirable (Des.), optimum (Opt.). The different methods principles are represented as follows: immuno-nephelometry by empty circles, ion-exchange HPLC by diamonds, enzymatic assays by a black triangle, capillary electrophoresis by a gray square and IFCC RMP by a black star. Peer groups are identified by letters: (A) Beckman AU B00389, (B) Beckman UniCel DxC Synchron, (C) Roche Cobas C513, (D) Siemens DCA 2000, (E) Siemens Dimension+, (F) Abbott enzymatic, (G) Bio-Rad Variant II, (H) Menarini HA 8160, (I) Tosoh, (J) Abbott–Afinion System, (K) Sebia.

Performance of individual laboratories

While the evaluation of assays performance according to the sigma metrics model shown in Figure 5 describes global performance of all laboratories from a given peer group, the Youden plot shown in Figure 6 provides further insights on the performance of individual laboratories. The results show that 86% of the laboratories met the 6% acceptance criterion for bias (data not shown). Ninety-nine percent and 86% of laboratories met the 5 mmol/mol acceptance criterion for TAE at low and high HbA1c concentration, respectively. For the high control (90 mmol/mol, i.e. 10.3% NGSP), only 2% of the participants reported results below the acceptance limit while 11% exceeded the upper acceptance limit. This result confirms that accuracy of HbA1c assays is globally better for low and medium HbA1c concentrations than for high concentrations.

Youden plot for the performance evaluation of individual laboratories for the measurement of HbA1c concentration in fresh whole blood clinical specimens. The Youden plot shows results from the 438 medical laboratories (white circles) that measured the low trueness verifier (x-axis) and the high trueness verifier (y-axis). Results obtained with the IFCC RMP are represented by black square. The solid rectangle corresponds to the 6% acceptance criteria for bias. The doted rectangle corresponds to the 5 mmol/mol acceptance criteria for TAE.
Figure 6:

Youden plot for the performance evaluation of individual laboratories for the measurement of HbA1c concentration in fresh whole blood clinical specimens.

The Youden plot shows results from the 438 medical laboratories (white circles) that measured the low trueness verifier (x-axis) and the high trueness verifier (y-axis). Results obtained with the IFCC RMP are represented by black square. The solid rectangle corresponds to the 6% acceptance criteria for bias. The doted rectangle corresponds to the 5 mmol/mol acceptance criteria for TAE.

Discussion

In this study, we assessed the commutability of 26 processed EQA materials for 17 of the most frequently used HbA1c assays. Our results show that the materials’ commutability was highly heterogeneous and generally low. Although some of the processed materials were shown to be more commutable than others, it was not possible to identify any unique features for the more commutable materials. Additional work is needed to gather extensive information on the materials’ properties, composition and manufacturing process so as to better understand what the parameters affecting commutability are. This would make it possible to select the most appropriate manufacturing process leading to the highest commutability levels and ultimately be able to predict the commutability of materials.

Although the 26 processed EQA materials involved in this study were used in various EQA schemes and were expected to be representative of the usual behavior of other similar EQA materials, the conclusions of this study may not be extended to other processed EQA materials. It should be noted that none of the EQA providers who participated in the study claimed, nor were anticipated to have, full commutability across all platforms. The fact that fresh CSs were not all found to be commutable when examined as if they were processed materials was unexpected and could be due to samples aging and/or degradation during shipment. This could cause either matrix effects or an alteration of samples homogeneity that would affect measurements precision and result in a higher rate of inconclusive statistical assessments. Still, our study clearly demonstrates that the commutability of EQA materials is very heterogeneous and cannot be taken for granted without conducting a formal commutability assessment.

In addition to selecting the study materials, a critical step in assessing commutability with the difference in bias approach is the choice of an appropriate commutability acceptance criterion [22]. If this criterion is too wide, it could be concluded that a material is suitable to assess trueness and/or between-methods comparability, when it might not be. On the contrary, a criterion that would be too stringent would result in a high rate of inconclusive results and the inability to make a conclusion [12], [23]. A consensus in commutability assessment of a trueness verifier is that it should correspond to a fraction of the TAE [21], [22]. Recently, Weykamp et al. recommended a 5 mmol/mol (0.46% NGSP) TAE for a sample of 50 mmol/mol (6.7% NGSP) HbA1c [25]. However, the use of this criterion in our study resulted in finding almost all EQA materials to be commutable (data not shown), even though strong discrepancies could be observed between different materials in terms of bias and between-method agreement, suggesting that this criterion was not sufficiently stringent. Besides, in France, a 7% TAE is used in the national mandatory EQA program [15] and the NGSP and the College of American Pathologists (CAP) both recommend a 6% TAE for HbA1c concentrations between 31 and 86 mmol/mol (5.0% and 10.0% NGSP) [26]. Similarly, in Germany, a change of TAE from 18% to 8% has been suggested for the national EQAS [27]. As the non-commutability of an EQA material is supposed to consume only a reasonable part of the TAE, a 6% criterion appeared a reasonable choice. Although it can be considered very stringent, only 12% of the statistical analyses were inconclusive and the fresh CSs were never found to be non-commutable.

In a recent study, Liu et al. used both the CLSI EP30-A guidelines [28] and the difference in bias approach to characterize the commutability of three certified reference materials consisting of frozen hemolyzed whole blood [28]. Using a similar study design and acceptance criteria, the authors found that the three materials were commutable for the main HbA1c assays except one. This confirms that producing materials that are commutable for all assays remains challenging and that commutability should be assessed for all assays for which trueness is evaluated, as was demonstrated in our study.

Due to the insufficient commutability of the processed EQA materials involved in our study, assay trueness could not be evaluated with these materials. Although the 22 fresh CSs used to evaluate commutability of the processed materials were also measured with the IFCC RMP, measurements were conducted in only one laboratory at the manufacturer sites (except for Abbott, Siemens, Trinity Biotech and Arkray assays). In order to better reflect “real” assay performances, it was decided to evaluate the trueness of HbA1c assays using fresh whole blood samples that were assayed in a large number of medical laboratories even though these did not necessarily use the same assays as those involved in the commutability study (e.g. Siemens DCA Vantage and DCA 2000). Biases observed during this study were in good agreement with those reported in previously published studies. Indeed, in a study involving 526 laboratories, Kaiser et al. reported a mean +1.3 mmol/mol bias for a sample at 71 mmol/mol (8.6% NGSP) [27]. Similarly, Weykamp et al. in a study involving the 3277 laboratories of the CAP 2014 survey, reported a mean bias of +1.3 mmol/mol (0.11% NGSP) for a material at 48 mmol/mol HbA1c (6.5% NGSP) [25]. More recently, the EurA1c Trial Group reported a +0.2 mmol/mol bias over 1517 laboratories using a fresh whole blood material and a −0.5 mmol/mol bias across 649 laboratories using a lyophilized material [29]. While biases reported in our study and in the EurA1c trial are consistent, it should be noted that in our study, peer groups gathered different analyzers and reagents, potentially shifting up the between-laboratory imprecision component. Despite this, the EurA1c Trial still showed a larger proportion of assays not meeting the 2σ performance criterion, supporting the fact that efforts are needed to further reduce between-laboratory variation [29]. Results of the EurA1c study also confirm our findings that for some assays, bias estimated using lyophilized materials were substantially larger than with fresh samples, especially the Abbott enzymatic assay [29].

Conclusions

Although processed EQA materials can still be used to evaluate assay precision and compare individual laboratory results with its peers, our study shows that the commutability of processed EQA materials is highly heterogeneous and generally insufficient to properly estimate between-method agreement and trueness of HbA1c assays. As commutability cannot be assumed or predicted, accuracy-based EQA programs should preferably rely on processed materials of proven commutability or fresh whole blood.

Even though it proved to be feasible, the use of fresh whole blood in the context of large EQA schemes remains challenging as it requires large blood volumes, ethical authorizations and optimal logistics due to the limited stability of materials [27], [28], [29], [30]. Using frozen materials could be a viable option as it would reduce the risks associated with the stability of samples. However, it should be kept in mind that frozen samples are not commutable for all assays. Especially, more and more POCT instruments are coming onto the market and many of these instruments can only work with fresh whole blood.

Finally, this study demonstrates the overall good analytical performances of HbA1c assays. Even though the accuracy is still perfectible at elevated HbA1c concentrations, most HbA1c assays meet the quality target of 5 mmol/mol TAE and demonstrate good comparability when evaluated with fresh blood samples or commutable materials.

Acknowledgments

We thank EQA providers and manufacturers of quality control materials having shared the materials which commutability was assessed in this study.

References

  • 1.

    World Health Organization. Global report on diabetes. 2016. Google Scholar

  • 2.

    Bunn HF. Nonenzymatic glycosylation of protein: relevance to diabetes. Am J Med 1981;70:325–30. CrossrefPubMedGoogle Scholar

  • 3.

    Stratton IM, Adler AI, Neil HA, Matthews DR, Manley SE, Cull CA, et al. Association of glycaemia with macrovascular and microvasular complications of type 2 diabetes (UKPDS 35): prospective observational study. Br Med J 2000;321:405–12. CrossrefGoogle Scholar

  • 4.

    The Diabetes Control and Complications Trial Research Group (DCCT). The relationship of glycemic exposure (HbA1c) to the risk of development and progression of retinopathy in the diabetes control and complications trial. Diabetes 1995;44:968–83. PubMedGoogle Scholar

  • 5.

    Peterson KP, Pavlovich JG, Goldstein D, Little R, England J, Peterson CM, et al. What is hemoglobin A1c? An analysis of glycated hemoglobins by electrospray ionization mass spectrometry. Clin Chem 1998;44:1951–8. CrossrefPubMedGoogle Scholar

  • 6.

    American Diabetes Association. Classification and diagnosis of diabetes. Diabetes Care 2017;40 (Suppl. 1):S11–24. PubMedGoogle Scholar

  • 7.

    Gillery P. A history of HbA1c through Clinical Chemistry and Laboratory Medicine. Clin Chem Lab Med 2013;51:65–74. PubMedGoogle Scholar

  • 8.

    Miller WG, Jones GR, Horowitz GL, Weykamp C. Proficiency testing/external quality assessment: current challenges and future directions. Clin Chem 2011;57:1670–80. Web of ScienceCrossrefPubMedGoogle Scholar

  • 9.

    Miller WG. Specimen materials, target values and commutability for external quality assessment (proficiency testing) schemes. Clin Chim Acta 2003;327:25–37. PubMedCrossrefGoogle Scholar

  • 10.

    Miller WG, Myers GL. Commutability still matters. Clin Chem 2013;59:1291–3. Web of ScienceCrossrefPubMedGoogle Scholar

  • 11.

    Vesper HW, Miller WG, Myers GL. Reference materials and commutability. Clin Biochem Rev 2007;28:139–47. PubMedGoogle Scholar

  • 12.

    Delatour V, Liu Q, Vesper HW. Commutability assessment of external quality assessment materials with the difference in bias approach: are acceptance criteria based on medical requirements too strict? Clin Chem 2016;62:1670–1. PubMedWeb of ScienceCrossrefGoogle Scholar

  • 13.

    Joint Committee for Guides in Metrology (JCGM). JCGM 200: 2012 International vocabulary of metrology – basic and general concepts and associated terms (VIM) – 3rd edition. 2012. Google Scholar

  • 14.

    Myers GL, Eckfeldt JH, Greenberg N, Levine JB, Miller WG, WiebeDA. CLSI C37-A: National Committee for Clinical Laboratory Standards. Preparation and validation of commutable frozen human serum pools as secondary reference materials for cholesterol measurements procedures; approved guideline. Wayne, PA: NCCLS Document C37-A 1999 NCCLS, 1999. Google Scholar

  • 15.

    Agence nationale de sécurité du médicament. Annales du contrôle national de qualité des analyses de biologie médicale – 15BIO1 – HbA1c. 2015. Google Scholar

  • 16.

    Little Randie R, Roberts William L. A review of variant hemoglobins interfering with hemoglobin A1c measurement. J Diabetes Sci Technol 2009;3:446–51. PubMedCrossrefGoogle Scholar

  • 17.

    INSTAND e.V. – Ringversuchsprogramm [Internet]. [cited 2019 April 29]. Available from: https://www.instand-ev.de/en/eqas/eqa-program.html

  • 18.

    Kaiser P, Akerboom T, Molnar P, Reinauer H. Modified HPLC-electrospray ionization/mass spectrometry method for HbA1c based on IFCC reference measurement procedure. Clin Chem 2008;54:1018–22. PubMedCrossrefGoogle Scholar

  • 19.

    Jeppsson J, Kobold U, Barr J, Hoelzel W, Hoshino T, Miedema K, et al. Approved IFCC reference method for the measurement of HbA1c in human blood. Clin Chem Lab Med 2002;35:78–89. Google Scholar

  • 20.

    Hoelzel W, Weykamp C, Jeppsson J-O, Miedema K, Barr JR, Goodall I, et al. IFCC Reference system for measurement of Hemoglobin A1c in human blood and the National Standardization Schemes in the unites States, Japan, and Sweden: a method-comparison study. Clin Chem 2004;50:166–74. CrossrefGoogle Scholar

  • 21.

    Miller WG, Schimmel H, Rej R, Greenberg N, Ceriotti F, Burns C, et al. IFCC working group recommendations for assessing commutability. Part 1: general experimental design. Clin Chem 2018;64:447–54. CrossrefWeb of ScienceGoogle Scholar

  • 22.

    Nilsson G, Budd JR, Greenberg N, Delatour V, Rej R, Panteghini M, et al. IFCC Working Group Recommendations for Assessing Commutability. Part 2: using the difference in bias between a reference material and clinical samples. Clin Chem 2018;64:455–64. Web of ScienceCrossrefGoogle Scholar

  • 23.

    Korzun WJ, Nilsson G, Bachmann LM, Myers GL, Sakurabayashi I, Nakajima K, et al. Difference in bias approach for commutability assessment: application to frozen pools of human serum measured by 8 direct methods for HDL and LDL cholesterol. Clin Chem 2015;61:1107–13. CrossrefPubMedWeb of ScienceGoogle Scholar

  • 24.

    Desirable Biological Variation Database Specifications – Westgard [Internet]. [cited 2019 April 29]. Available from: https://www.westgard.com/biodatabase1.htm

  • 25.

    Weykamp C, John G, Gillery P, English E, Ji L, Lenters-Westra E, et al. Investigation of 2 models to set and evaluate quality targets for HbA1c: biological variation and sigma-metrics. Clin Chem 2015;61:752–9. CrossrefPubMedGoogle Scholar

  • 26.

    Little RR, Rohlfing CL. Assessing quality from an accuracy-based HbA1c proficiency survey. Clin Chem Lab Med 2016;54:e75–6. PubMedGoogle Scholar

  • 27.

    Kaiser P, Spannagl M, Van Campenhout C, Lenga Y, Siebelder C, Weykamp C. HbA1c: EQA in Germany, Belgium and the Netherlands using fresh whole blood samples with target values assigned with the IFCC reference system. Clin Chem Lab Med 2016;54:1769–75. PubMedGoogle Scholar

  • 28.

    Liu H, Wong L, Yong S, Liu Q, Teo TL, Lee TK, et al. Commutable whole blood reference materials for hemoglobin A1c validated on multiple clinical analyzers. Clin Chem Lab Med 2019;57:648–58. PubMedCrossrefGoogle Scholar

  • 29.

    Weykamp C, John WG, English E, Erasmus RT, Sacks DB, on behalf of the EurA1c Trial Group. EurA1c: the European HbA1c trial to investigate the performance of HbA1c assays in 2166 laboratories across 17 countries and 24 manufacturers by use of the IFCC model for quality targets. Clin Chem 2018;64:1183–92. CrossrefWeb of SciencePubMedGoogle Scholar

  • 30.

    Mosca A, Paleari R, Carobene A, Weykamp C, Ceriotti F. Performance of glycated hemoglobin (HbA1c) methods evaluated with EQAS studies using fresh blood samples: still space for improvements. Clin Chim Acta 2015;451:305–9. CrossrefPubMedGoogle Scholar

Supplementary Material

The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2019-0219).

About the article

Corresponding author: Dr. Vincent Delatour, Laboratoire National de Métrologie et d’Essais (LNE), 1 rue Gaston Boissier, 75724 Paris Cedex 15, France, Phone: +33 140 434 075

aVincent Delatour and Noémie Clouet-Foraison contributed equally to the writing of this article.


Received: 2019-02-25

Accepted: 2019-04-07

Published Online: 2019-05-14

Published in Print: 2019-09-25


Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Research funding: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.


Citation Information: Clinical Chemistry and Laboratory Medicine (CCLM), Volume 57, Issue 10, Pages 1623–1631, ISSN (Online) 1437-4331, ISSN (Print) 1434-6621, DOI: https://doi.org/10.1515/cclm-2019-0219.

Export Citation

©2019 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Supplementary Article Materials

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Vincent Delatour, Noémie Clouet-Foraison, Stéphane Jaisson, Patricia Kaiser, and Philippe Gillery
Clinical Chemistry, 2020, Volume 66, Number 2, Page 390
[2]
Christoph Gerke, Monika Buchholz, Holger Müller, Reinhard Meusinger, Matthias Grimmler, and Erwin Metzmann
Analytical and Bioanalytical Chemistry, 2019

Comments (0)

Please log in or register to comment.
Log in