Diabetes mellitus is a global health issue which results in significant morbidity, mortality and economic burden, with an estimated one in 11 adults having the disease . Essential to tackling this global epidemic is early identification and effective management of the disease. Crucial to these objectives is high quality, standardised, reliable haemoglobin A1c (HbA1c) testing, without which it would be impossible to set accurate diagnostic thresholds or beneficial treatment targets.
The mission statement for the IFCC Education and Management Division (EMD) highlights their ambition to ‘provide the healthcare community with education relevant to Clinical Chemistry and Laboratory Medicine, directed at scientific, management and clinical issues’ . The division is home to a number of committees, each with a specific clinical or analytical area of focus, together they work collectively towards supporting the education and knowledge exchange of laboratory personnel, clinicians, patients and policy makers in order to have significant impact on the way in which diagnostic testing is perceived and used to improve health.
The Committee for Education in the Use of Biomarkers in Diabetes (C-EUBD) sits within the EMD and is focussed on education strategies for global implementation of HbA1c standardisation and other biomarkers associated with diabetes care which is essential to ensure the ongoing clinical utility of these markers . Implementation of standardisation of a diagnostic test requires key components such as a reference measurement procedure (RMP), reference materials and a reference laboratory network. Alongside these, appropriate clinical reference intervals/decision limits are needed, with defined quality targets for the analyte. Internal quality control (IQC) and external quality assessment (EQA) are needed to monitor that the quality of the results produced, meets the quality targets that have been set . The C-EUBD and predecessor committees have developed and implemented each of these key components to satisfy the needs of standardisation of HbA1c , , . However, adoption, knowledge and understanding of the process remains a global issue with a number of barriers to truly successful standardisation remaining.
In order to contextualise the barriers to effective implementation of personalised medicine Horgan et al. undertook a systematic review of policy and literature to define key areas of focus . The key themes identified were scientific, operational, economic and legislative barriers, which are also applicable to the implementation of the standardisation of diagnostic tests.
In scientific terms, the analytical standardisation and the role of HbA1c in diagnostic testing and monitoring of people with diabetes is well established. However, specific treatment targets and the most appropriate diagnostic threshold are topics that are still widely debated, with not all countries adopting the test for diagnosis or using the 48 mmol/mol (6.5%) threshold advised . Analytically, high quality equipment is available for the measurement of HbA1c however, the extent to which this is available or utilised on a global level is unclear, with some countries using re-badged or locally manufactured instruments in which the link to the IFCC RMP has not been demonstrated. The IFCC C-EUBD continues to work with expanding numbers of manufacturers to ensure calibration to the RMP through the network of reference laboratories . The committee is also engaged in studies which will address clinical issues, reviewing the evidence for the diagnostic thresholds and working towards defining personalised targets for HbA1c monitoring.
In operational terms, information delivery, education and training, empowerment of patients and health professionals and introduction of regulations are all barriers to effective implementation of standardised HbA1c testing. The provision of clinical decision support systems, education around interpretation of results and their communication to patients, and improving health literacy to adjust public expectations and knowledge around diabetes testing are all needed to overcome these barriers. The IFCC Task Force on Implementation of HbA1c Standardisation (TF-HbA1c) started to address these issues with lecture series and review articles to support understanding and knowledge around the analytical process of standardisation. The current C-EUBD is taking this further with workshops funded by UK research councils and industry partners aimed at improving knowledge and understanding in health professionals in areas such as result interpretation, analytical performance of both main laboratory and POC instruments, the value of EQA programmes and expanded roles for HbA1c in other disease areas , .
Economic barriers to the effective use of HbA1c testing are a particular hindrance in low and lower middle income countries where the test cost is simply too high to be affordable with many insurance companies not covering the cost of testing, keeping demand low. The low demand, in turn, maintains a high test cost, resulting in a self-perpetuating cycle that will only be broken if either test demand increases or test cost falls or both . The C-EUBD, is working with industry partners to facilitate pilot studies to improve access to testing in low resource setting such as sub-Saharan Africa and parts of the Asia-Pacific region, in turn this will generate wider implementation of testing in these areas.
Regulatory and guideline barriers are multi-faceted issues. Different countries or regions have different regulatory guidance, for example, in Europe legislation dictates that manufacturers must demonstrate that their instruments are traceable to higher-order references such as the IFCC-RMP for HbA1c . However, whilst the consensus statement of key clinical and scientific bodies associated with diabetes care advises the same practice, it is not mandatory on a global level. Amongst the scientific and clinical communities there is a high level of engagement with the concept of implementation of HbA1c standardisation, however in order to effect a change it is the government and non-government organisations (NGOs) who commission healthcare provision that need to be engaged. The C-EUBD is building collaborative links with such organisations to ensure a multi-stakeholder approach is taken to addressing the barriers to implementation of HbA1c standardisation.
For the implementation of standardisation to be a true success all stakeholders need to work together to ensure that patient care remains consistent and optimal. One such example of the way in which the C-EUBD aims to engage all stakeholders to tackle barriers in each of the main themes above, is through a collaborative multi-centre trial in China, which we describe in this paper.
A barrier to the introduction of HbA1c for diagnosis in China is a lack of evidence to demonstrate that the quality targets for HbA1c can be met across the country, at the individual laboratory level or across a network of laboratories which may form an EQA scheme. The aims of this study were to assess the analytical performance of 30 clinical centres across China using the international quality standards  and to assess the logistical issues associated with creating a quality network. Key stakeholders include, industry partners (Trinity Biotech, Bray, Ireland), the Chinese Diabetes Society (CDS), the IFCC C-EUBD and leading clinical centres across the country.
Materials and methods
This large study was conducted in 30 laboratories across China. Centres were chosen to gain a representative geographical and population based sample. Participating centres are listed in Table 1, in alphabetical order. In order to maintain confidentiality for each centre this does not represent the centre numbers used in the data analysis.
Knowledge exchange workshops
Workshops were developed and held in Beijing and Shanghai with clinical and laboratory staff from each of the participating centres as well as national key opinion leaders able to meet, sponsored as part of the study.
Each laboratory utilised a Premier Hb9210 automated benchtop HbA1c analyser (Trinity Biotech, Bray, Ireland) which uses boronate affinity high performance liquid chromatography (HPLC) for measurement of HbA1c. The performance of the analyser in a multicentre setting has previously been shown to be adequate . The Hb9210 system was calibrated using a two-point calibration as per the manufacturer’s instructions. Calibrator values were assigned by the manufacturer using secondary calibration material supplied by the IFCC Standardisation Network as recommended in the consensus statement .
Sample preparation and storage
Each laboratory was supplied with a set of 50 (25 duplicates) HbA1c samples for the trueness study and 20 samples for the imprecision study. The set of 50 samples was generated by mixing pooled samples, collected from multiple donors; a high pool (78 mmol/mol [9.3% NGSP]) and low pool (31.2 mmol/mol [5% NGSP]) were mixed in varying increments to achieve a range of HbA1c values. Donor samples (700 mL) were collected into di-potassium EDTA containing tubes. Before mixing the volume of the plasma in the sample with the lowest haemoglobin concentration was adjusted to ensure that the haemoglobin concentrations of the high and low donations were equivalent. Aliquots of 0.25 mL of each of the 50 samples were individually labelled in cryo-vials with red caps to help differentiate from the imprecision samples, and packaged into labelled boxes before being shipped to participating laboratories. Samples were prepared on the day of collection and prior to shipment samples were stored at −80 °C and shipped to participating centres on dry ice.
Samples for the imprecision studies were prepared as above with low sample values of (37.6 mmol/mol, [5.6% NGSP] yellow caps) and high of (59.0 mmol/mol, [7.5% NGSP] blue caps). Samples were shipped to participating laboratories on dry ice.
All samples were tested and found negative for Hepatitis B Surface Antigen (HBsAg), antibody to hepatitis C (HCV), antibody to HIV and HIV antigen.
Target value assignment
The target values of all of the samples were assigned in duplicate with each of six IFCC calibrated routine methods Tosoh G8; ARKRAY/Menarini HA 8180V, Sebia Capillarys 2 Flex Piercing, Roche Tina Quant Gen 3 on Cobas c513 and Trinity Biotech Premier Hb9210 (two instruments) at the European Reference Laboratory, Queen Beatrix Hospital, Winterswijk and Isala, Zwolle, The Netherlands.
Imprecision was assessed using a Clinical and Laboratory Standards Institute (CLSI) EP-5  based protocol with low and high samples measured twice a day, in duplicate, for 20 working days.
Each laboratory received 50 samples (25 duplicates) which were analysed in a single batch on two separate days . The laboratories were blinded to the values of the samples prior to analysis. Bias was assessed by plotting the measured vs. expected results and a simple linear regression line calculated. The bias at a target value of 50 mmol/mol (6.7% NGSP) was calculated and further used in the quality target analysis.
The assessment of performance for each laboratory, both individually and as a group, was performed using the IFCC TF-HbA1c, guidance on the use of sigma metrics for quality targets . In the laboratory sigma-metrics is a quality management strategy that provides a universal benchmark for process performances. Analytical characteristics (bias and imprecision) in the form of total allowable error (TAE) are placed within a framework of clinical requirements. The TAE was set at 5 mmol/mol (0.46% NGSP) with a minimum sigma value of 2 to pass the evaluation criteria.
Statistical analyses were performed using Microsoft® Excel 2010 (Microsoft Corporation) and EP-Evaluator Release 11 (Data Innovations) .
Laboratories were instructed to undertake a structured familiarisation study of the analyser in order to ensure that they were efficient and competent with the use of the instrument before commencement of the main study protocol. Essentially the familiarisation protocol assessed imprecision and bias in a 7-day protocol. The results of the familiarisation study are not included in the main evaluated data. Once the familiarisation study was complete the participants requested the delivery of the main study samples for analysis. A detailed handbook for both the familiarisation and main studies was compiled and translated into Chinese to ensure the protocol was consistently performed.
Ethical approval for the study was granted by the University of East Anglia Faculty of Medicine and Health Sciences Research Ethics Committee (ref: 20152016-21).
Engagement from participating centres, IFCC committee members, key opinion leaders from China and key policy makers such as representatives of the CDS was achieved.
The results of the EP-5 analysis for each individual laboratory are detailed in Table 2. Coefficient of variations (CV) values for the low samples ranged from 1.1 to 2.6% and CV values for the high samples ranged from 0.9 to 2.7%, well within the 3% target often cited for CV calculated using mmol/mol units , . At the low level the mean HbA1c for all laboratories was 37.59 mmol/mol (5.6% NGSP) at a target value of 37.6 mmol/mol (5.6% NGSP). At the high level the mean for all laboratories was 57.91 mmol/mol (7.4% NGSP) at a target value of 59.0 mmol/mol (7.5% NGSP), thus an acceptable difference of 1.09 mmol/mol.
All of the mean values were analysed for outliers using the Q-Test. With a critical value for Q of 0.23 the values for the low and high CVs of centre 10 and the high CV for centre 23 were excluded from subsequent analysis, despite CV values within a range that would normally be acceptable.
At the low level the mean within lab CV was 1.86% and the between lab CV was calculated as the mean of the SDs divided by the target value for the samples×100 (0.97/37.6×100%) giving a value of 2.6%.
At the high level the mean within lab CV was 1.49% and the between lab CV was 1.20/57.8×100%=2.1%. It is widely recognised that the ratio of the between/within lab CV is commonly approximately 1.5. Is this study the ratio is 2.6/1.86=1.4 and at the high level 2.1/1.49=1.4. The higher the ratio the less control a manufacturer has on batches of reagents/calibrators, however, this ratio is considered acceptable.
The results of the comparison study (against IFCC targeted values) are shown in Table 3. The individual laboratory CVs were calculated from the 25 duplicate samples, with the regression lines and bias at 50 mmol/mol (6.7%) calculated for each centre. The performance of all laboratories as a group was also assessed for imprecision and trueness. Supplementary Table 1 details the target values and bias for all laboratories for the calculation of group performance data.
When the IFCC model for quality targets was applied at a level of 2 σ each of the participating centres was deemed to have achieved a satisfactory performance level. Figure 1 demonstrates the within laboratory performance, with each point representing an individual centre. It can be seen that there is a wide performance range with some laboratories performing significantly better than the TAE of 5 mmol/mol at the 2 σ level. Additional coloured bands of bronze, silver and gold, represent reducing TAE levels of 3, 2 and 1 mmol/mol TAE, respectively.
The performance of all participating centres as a group is shown in Figure 2. The bias of all laboratories was calculated from the target value minus the mean of all laboratories (see Supplementary Table 1). The between laboratory CV was calculated and plotted against the between laboratory bias. The samples were divided into low, medium and high values as shown in Figure 2. Overall 48 out of the 50 samples met or exceeded 2 σ and were deemed to have passed the quality standards. Statistically it can be expected that one out of 20 results will not be within the target limits (thus two out of 50 is acceptable).
There was little difference in performance between samples with low and high HbA1c concentrations. The between laboratory CV is slightly lower in the samples with the high HbA1c levels. Bias is also slightly lower in samples with the high HbA1c however, differences were small. It can be concluded that the project succeeded in achieving standardisation across 30 laboratories in China using a single HbA1c analyser.
Using a multi-stakeholder approach the aim of this study was to assess the barriers to generating a network of laboratories, across a large country, where each centre is capable of producing HbA1c values that are aligned to international reference measurement values and the analytical performance at each laboratory meets published international quality criteria. This is important as it is a major requirement to enable the roll out of HbA1c for diagnosis of type 2 diabetes mellitus (T2DM) in accordance with World Health Organization (WHO) guidance. The aim was also to generate a network of laboratories with between laboratory analytical agreement which is good enough to support multicentre trials in the future, providing an essential service for population based studies. Finally, the study was successful in forging strong partnerships and collaborations with key stakeholders involved in the commissioning and delivery of diabetes care and is the first multi-centre study of its kind both in terms of scale and scope.
The study utilised fresh frozen whole blood samples with target values traceable to the IFCC RMP and each of the 30 centres received identical samples and instruments, which minimised uncertainty in the estimation of trueness. Each individual laboratory demonstrated performance at or above the 2 σ level with a TAE of 5 mmol/mol. In order to stimulate further improvement in quality a reduced TAE may be used, with 14 laboratories performing at a TAE of 3 mmol/mol or less and six at 2 mmol/mol TAE or less. Each individual laboratory has demonstrated the ability to produce accurate and precise results and the enhanced performance shown by some demonstrates that there is already a stimulus towards ever improving quality. The use of a single method type may be seen as a limitation of this study as it is preferable to assess quality using a range of methods which represent the ‘real-world’ situation. However, the decision was made to utilise a single method type in this study to reduce the potential for uncertainty, as it was the process as a whole that was being assessed.
When the results are viewed as a group rather than individual centres then the key questions to be addressed are; can HbA1c be measured at any laboratory and generate the correct result? And does this performance apply at all HbA1c levels? Even when outliers were considered, all centres passed the IFCC criteria therefore it is possible to conclude that HbA1c can be measured correctly at any/all of the centres. Secondly, the performance of the group at normal, medium and high levels showed that 48 out of the 50 samples also passed the IFCC criteria with results tightly clustered, demonstrating good performance at all HbA1c levels.
The WHO statement on the use of HbA1c for diagnosis states that a cut off of 6.5% (48 mmol/mol) should be used but only if the method used is aligned to the international reference values and that stringent quality assurance tests are in place .
The second requirement of the WHO statement is that stringent quality assurance tests are in place, to demonstrate the accuracy of the measurement system being used. Where national EQA schemes or proficiency testing (PT) schemes are available and are fit for purpose, engagement with these may enable the use of HbA1c for diagnosis.
Overall, this unique, large-scale study has shown that it is possible to generate a network of laboratories across China that provides a standardised measurement of HbA1c values which meet the IFCC criteria for analytical performance and are aligned to the international reference values .
This in turn fulfils the first element of the WHO criteria for the use of HbA1c for the diagnosis of T2DM. This network could also be used to form the basis of a national quality scheme thus meeting the second element of the WHO criteria in addition to supporting high quality results for the monitoring of patients already diagnosed.
Importantly this joint project between CDS, clinical centres, the IFCC and industry has demonstrated that it is possible to engage multiple stakeholders in a common purpose to improve diabetes testing across China. This engagement has led to the formation of a new partnership between the IFCC C-EUBD and the CDS who together will devise guidance on diabetes testing and support change across China in future years. This has been an immensely successful project demonstrating the value and impact of the IFCC EMD and its committees.
This project would not have been possible without considerable support from technologists and physicians at all of the participating centres. These are listed in Supplementary Table 2 in order to give complete recognition to all involved. The authors would like to thank Trinity Biotech (Ireland) and Primus Medical Shanghai (China) for contribution of the instruments and reagents for this study.
Federation ID. IDF Diabetes Atlas, 8th ed. Brussels: Belgium International Diabetes Federation, 2017. Google Scholar
IFCC. IFCC – Education and Management Division (EMD) 2018 [cited 2018]. http://www.ifcc.org/ifcc-education- division/. Accessed: June 2018.
IFCC. Education in the Use of Biomarkers in Diabetes (C-EUBD) 2018. http://www.ifcc.org/ifcc-education-division/emd-committees/c-eubd/. Accessed: June 2018.
Weykamp C, John G, Gillery P, English E, Ji L, Lenters-Westra E, et al. Investigation of 2 models to set and evaluate quality targets for hb a1c: biological variation and sigma-metrics. Clin Chem 2015;61:752–9. CrossrefPubMedGoogle Scholar
Horgan D, Jansen M, Leyens L, Lal JA, Sudbrak R, Hackenitz E, et al. An index of barriers for the implementation of personalised medicine and pharmacogenomics in Europe. Public Health Genomics 2014;17:287–98. PubMedWeb of ScienceCrossrefGoogle Scholar
Organization WH. Use of Glycated Haemoglobin (HbA1c) in the Diagnosis of Diabetes Mellitus (abbreviated Report of a WHO Consultation). 2011. Google Scholar
Medicine IFoCCaL. IFCC HbA1c Network. http://www.ifcchba1c.net/network/approved?page=1.
Kaiser P, Spannagl M, van Campenhout C, Lenga Y, Siebelder C, Weykamp C. HbA1c: EQA in Germany, Belgium and the Netherlands using fresh whole blood samples with target values assigned with the IFCC reference system. Clin Chem Lab Med 2016;54:1769–75. PubMedGoogle Scholar
Weykamp C, Siebelder C. Evaluation of performance of laboratories and manufacturers within the framework of the IFCC model for quality targets of HbA1c. J Diabetes Sci Technol 2018;12:747–52. PubMedCrossrefGoogle Scholar
Atun R, Davies JI, Gale EA, Barnighausen T, Beran D, Kengne AP, et al. Diabetes in sub-Saharan Africa: from clinical care to health policy. Lancet Diabetes Endocrinol 2017;5:622–67. Web of ScienceCrossrefPubMedGoogle Scholar
Standardization IOf. ISO/TC 212 Clinical laboratory testing and in vitro diagnostic test systems. https://www.iso.org/committee/54916.html.
John WG, Little R, Sacks DB, Weykamp C, Lenters-Westra E, Hornsby T, et al. Multicentre evaluation of the Premier Hb9210 HbA1c analyser. Clin Chem Lab Med 2015;53:319–27. PubMedWeb of ScienceGoogle Scholar
Institute CaLS. Evaluation of Precision Performance of Clinical Chemistry Devices; Approved Guideline. NCCLS Document EP05-A. PA: CLSI Wayne, 1992. Google Scholar
Institute CaLS. Method Comparison and Bias Estimation using Patient Samples; Approved Guideline. CLSI Document EP09-A PA: CLSI Wayne, 1995. Google Scholar
Innovations D. EP Evaluator. https://www.datainnovations.com/products/ep-evaluator.
Goodall I, Colman PG, Schneider HG, McLean M, Barker G. Desirable performance standards for HbA(1c) analysis – precision, accuracy and standardisation: consensus statement of the Australasian Association of Clinical Biochemists (AACB), the Australian Diabetes Society (ADS), the Royal College of Pathologists of Australasia (RCPA), Endocrine Society of Australia (ESA), and the Australian Diabetes Educators Association (ADEA). Clin Chem Lab Med 2007;45:1083–97. PubMedGoogle Scholar
Weykamp CW, Mosca A, Gillery P, Panteghini M. The analytical goals for hemoglobin A(1c) measurement in IFCC units and National Glycohemoglobin Standardization Program Units are different. Clin Chem 2011;57:1204–6. Web of SciencePubMedCrossrefGoogle Scholar
The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2018-0434).
About the article
Published Online: 2018-07-25
Published in Print: 2018-12-19
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
Citation Information: Clinical Chemistry and Laboratory Medicine (CCLM), Volume 57, Issue 2, Pages 288–295, ISSN (Online) 1437-4331, ISSN (Print) 1434-6621, DOI: https://doi.org/10.1515/cclm-2018-0434.
©2018 Emma English et al., published by De Gruyter, Berlin/Boston. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0