Point-of-care testing in primary healthcare: a scoring system to determine the frequency of performing internal quality control

Objectives: Internal quality control (IQC) plays an important role in quality assurance in laboratory medicine. How-ever, there is no universal consensus or guideline on when and how IQC should be analyzed on point-of-care testing (POCT) devices. The aim of this study was to develop a scoring system to determine how often IQC should be analyzedinprimary healthcareonthevarious POCT devices. Methods: Based on a systematic literature review and a thorough process involving the whole Noklus, a nation-wide POC organization, a scoring system for when to analyze IQC was developed. Four factors were considered to significantly impact IQC frequency: The importance of the analyte in diagnosing and monitoring patients, type of POCT device, user-friendliness, and number of patient samples. For each POCT device, the first three factors were given a score, and the sum of the scores determined the general recommended IQC frequency. The number of patient samples determined whether and how to adjust these frequencies in each individual general practice. Results: The scoring system was applied to 17 analytes and 134 different POCT devices (153 analyte-device com-binations). Most of the devices analyzing high-risk analytes (71 out of 74) obtained daily or weekly IQC frequency. For example, all blood-cell counters and all glucose meters should undergo IQC daily and weekly, respectively. Conclusions: This study presents a consensus-based scoring system for differentiated and device-specific recommendations for IQC frequency on POCT devices in primary healthcare. The scoring system can easily be adopted to other local environments and is easy to use.


Introduction
Physicians in primary healthcare base many of their clinical decisions on point-of-care testing (POCT) results. The ISO 22870:2016 defines POCT as "testing that is performed near or at the site of a patient with the result leading to possible change in the care of the patient" [1]. POCT aims to provide test results more quickly, which induces more expeditious clinical decisions than if analyses are performed in large medical laboratories. Despite ongoing improvements in technology and operational simplicity, POCT has many potential limitations [2]. Therefore, to ensure accurate and reliable test results, POCT guidelines describe comprehensive plans for equipment selection and maintenance, staff training, reporting results, external quality assurance (EQA), and internal quality control (IQC) [1]. However, publications are few and evidence scarce for determining when IQC should be used in POCT [3,4].
IQC is analysis of control samples containing analytes of a known concentration to monitor the accuracy and precision of the analytical process over time. In contrast to IQC performed in large medical laboratories, analysis in primary healthcare is commonly handled by non-laboratory personnel on single-analyte devices and with a low number of samples. Another difference is that opposite analysis in large medical laboratories where most errors occur in the preanalytical phase, most errors in POCT occur in the analytical phase [5]. Therefore, the IQC procedures from large medical laboratories cannot be adopted, but must be specially designed to the POCT environment.
The ISO 15189:2012 states that quality control materials should be periodically examined with a frequency that is based on the stability of the procedure and the risk of harm to the patient from an erroneous result [6], while the ISO 22870:2016 states that the IQC frequency should be specified for each device [1]. In a study from our group, it was found that analysing internal quality control once a week for hemoglobin, C-reactive protein (CRP), and glucose was associated with better analytical quality [7]. Various POCT guidelines suggest that IQC should be performed before a new batch of reagents or tests are being used, after unexpected test results, and after maintenance and instrument repair [8][9][10][11][12][13]. These guidelines also propose that IQC should be analyzed at least monthly [9,10], weekly [8], daily [9,12,13], or at the frequency proposed by the manufacturer [9,11,12,14]. However, these recommendations are neither universal nor device specific, meaning that each practice should determine the appropriate IQC frequency for the POCT devices available to them. This process can be challenging [15,16]. Various studies have proposed different procedures for determining the IQC frequency and have applied a pragmatic approach that considers the risk of harm to the patient based on the importance of the analyte in diagnosing and monitoring patients, the complexity and user-friendliness of the POCT devices, and the number of patient samples [3,[16][17][18][19]. To our knowledge, however, previous studies have not combined all these factors into a single scoring system for determining the IQC frequency.
The aim of this study was to develop a scoring system for determining the IQC frequency for POCT devices in primary healthcare.

Working group
For almost three decades, the Norwegian Organization for Quality Improvement of laboratory Examinations (Noklus) has focused on POCT quality in primary healthcare laboratories [20]. Noklus is a nonprofit organization with more than 3,600 voluntary participants. Noklus organizes EQA programs for all analytes analyzed in primary healtcare. As a part of these EQA programs, Noklus requests the primary healthcare laboratories to report data on several quality indicators, two of them being IQC frequency and number of patient samples. Laboratory advisers located in 22 different hospitals around the country provide face-to-face guidance for primary healthcare laboratories such as general practice (GP) offices, nursing homes, home care and others, based on their broad knowledge of POCT [20].
In December 2018, Noklus established a working group to revise the IQC guidelines. This working group included two medical specialists in laboratory medicine (E.C.L. and S.S.), three researchers (A.S., A.E.S. and G.G.), and two laboratory advisors (biomedical laboratory scientists) (A.L.F. and S.B.) with multiple years of experience in guiding the use of POCT devices in primary healthcare.
Based on the literature review, the working group reached consensus in categorizing analytes and POCT devices that are used in primary healthcare. The work was consecutively presented and audited in meetings with all 110 employees (including 55 laboratory advisors and 20 medical specialists in laboratory medicine) in the Noklus organization in September 2018 and 2019. Input and questions for the proposals were encouraged. After a 14-month 11-meeting process, the whole organization received a final draft of the recommendations in February 2020. Twenty laboratory advisors and medical specialists in laboratory medicine commented in writing on the final draft, and all comments were considered by the working group before finalizing the report.

The scoring system
The general principle of the scoring system was that higher probabilities of erroneous test results and higher risk of harm to the patients mean that IQC should be performed more frequently. Four factors were considered to significantly impact IQC frequencies ( Figure 1). The factors were as follows: A) Risk of harm to the patient based on the importance of the analyte in diagnosing and monitoring patients. B) Type of POCT device. C) User-friendliness. D) Number of patient samples analyzed over a specific period.
For each POCT device, factors A, B, and C were given a certain number of points (a score), and the total score was calculated as the sum of the three scores. The total scores determined the general recommended IQC frequency for each POCT device. In addition, the number of patient samples analyzed over a specific period determined whether and how to adjust these frequencies in each individual practice.
The distribution of points and the interpretation of total scores were made by the working group members based on theoretical knowledge and practical feasibility. The principles behind the assessments are described in the following.
A) Risk of harm to the patient based on the importance of the analyte in diagnosing and monitoring patients Patient treatment and diagnosis are often supported by laboratory test results. All laboratory tests are important, but erroneous results from certain analytes are likely to cause more serious adverse effects than others [19]. The working group therefore identified 17 analytes that are recommended to be analyzed in primary healthcare [20] and divided them into two groups: moderate-risk (2 points) and high-risk (4 points) analytes, with high-risk analytes being controlled more frequently than moderate-risk analytes ( Figure 1). Analyses judged as essential for diagnostic or monitoring purposes were defined as high-risk analytes, such as glycated hemoglobin (HbA 1c ) and the prothrombin time international normalized ratio (INR) ( Table 1).

B) Type of POCT device
The types of POC instruments and complexity will vary in different countries depending on their healthcare policies [26]. POCT devices range from simple strip-based tests to advanced blood-cell counters. More complex devices have more potential errors. The devices were categorized into four types with different complexities: qualitative and semi-quantitative (visual reading) POC tests such as pregnancy tests and urine test strips were considered the least complex (1 point), strip-based tests with automatic readings such as glucose meters were considered more complex (2 points), single cartridge tests with automatic reading such as many HbA 1c analyzers were considered even more complex (3 points), and larger bench-top instruments such as automated blood-cell counters (reduced in both size and complexity compared to the blood-cell counters used in large medical laboratories) were considered the most complex (4 points) ( Figure 1).

C) User-friendliness
POCT devices in primary healthcare are often operated by employees without adequate education and experience in laboratory medicine to anticipate the many potential errors that may arise in the testing process [21]. Laboratory work is only one of the many tasks in primary healthcare staff's busy working day. Operators can fail to detect expired reagents, reagents can be stored at incorrect temperatures, and correct sample volumes or types might not be used [22]. Lack of operator competence and adherence to test procedures are the main sources of error in POCT [21]. Therefore, operators should receive initial training and demonstrate competence according to ISO 22870:2016 in terms of: "sample collection, its clinical utility and limitations, expertise in the analytical procedure, reagent storage, quality control and quality assurance, technical limitations of the device, response to results that fall outside of predefined limits, infection control practices, and correct documentation and maintenance of the results" [1]. The expected level of competence must be determined locally depending on the repertoire and instruments used. Continuous competency training and support have been found to maintain improved analytical quality [7,20].
Many potential errors exist in the pre-analytical, analytical and post-analytical phases of POCT testing [27]. IQC cannot detect all Scoring system to determine IQC frequency on POCT devices potential errors, such as sample collection or test result reporting and documentation, which are errors occurring in the pre-and postanalytical phases. Many such errors are better monitored by, e.g., quality indicators [28]. However, IQC can detect many potential user errors in the analytical phase. User-friendliness was scored based on the working group's overall assessment of the ease of use for the intended operators on preparation of test or instrument, preparation and application of the sample, number of procedure steps, instrument-or test design, reading of the test result, maintenance, hygiene, and storage conditions for reagents and tests in unopened and opened packages. The POCT devices were given the following scores: easy (1 point), moderately difficult (2 points), and difficult (3 points) (Figure 1).

Sum of A), B), and C)the general recommended IQC frequencies
Total scores were calculated as the sum of points given for factors A, B, and C and were the basis for the following general IQC frequency recommendations: daily, meaning every day that the test is in use (10 or 11 points); weekly (7-9 points); monthly (5 or 6 points); or occasionally (4 points) (Figure 1). IQC should also always be analyzed before a new batch of reagents or tests are being used, if unexpected test results are present, if an error is suspected due to reasons such as incorrect storage temperature or physical damage, or after instrument repair or maintenance (Figure 1). For the "occasionally" frequency, it is acceptable to analyze IQC only when any of these situations occur.
The general recommended IQC frequencies for each device were recently included in the electronic laboratory procedures of Noklus, which are available to all 3,600 participants in Noklus. The laboratory advisors will offer guidance on how laboratories can implement the recommended frequencies and how to individually adjust these based on the number of patient tests performed or other local considerations. If a laboratory should decide to perform IQC less frequently than recommended, for example monthly instead of weekly, it will delay the detection of a possible analytical error by three weeks, with the risk of releasing erroneous test results for the patients tested during this time. Also, the physician must re-evaluate test results and perhaps summon patients for a new test.

D) Number of patient samples analyzed in a specific time period
IQC frequencies based on this scoring system are general recommendations. These frequencies should also be adjusted for each individual  practice depending on the average number of patient samples analyzed. This is because the number of potentially erroneous results reported depends on the IQC interval. We suggest that practices analyzing many samples, in Norway judged by the working group as more than 50 patient samples each week, should perform IQC more frequently than that obtained using the general scoring system; for example, if the general scoring system suggests a weekly frequency, these practices should increase their IQC frequency from weekly to daily ( Figure 1, Table 2). We further suggest that practices performing only 1 or 2 patient samples each week should decrease their IQC frequency from weekly to monthly (Figure 1, Table 2). Finally, the practices analyzing only 0-3 patient samples each month should perform IQC before each patient sample to ensure the correct performance of their POC system ( Figure 1, Table 2).

Results
The general scoring system was applied to 17 analytes and 134 different POCT devices (153 analyte-device combinations in total), which were all devices used by more than five participants in the Noklus EQA schemes (Supplementary Table 2). Table 1 lists the scores for the most frequently used POCT devices.
Most of the devices analyzing high-risk analytes (71/74) ended up with an IQC frequency from the general scoring system of daily (n=14) or weekly (n=57). For INR analyzed on CUBE (Eurolyser Diagnostica GmbH, Salzburg, Austria) and Simple Simon PT (Zafena AB, Borensberg, Sweden), the suggested frequencies were daily, while for CoaguChek (Roche Diagnostics, Mannheim, Germany), iLine microINR (iLine Microsystems S.L., Gipuzkoa, Spain), and Xprecia Stride (Siemens Healthcare, Erlangen, Germany) suggested frequencies were weekly due to instrument complexity and user-friendliness scoring differently (Supplementary Table 2). All blood-cell counters and glucose meters ended up with IQC daily and weekly, respectively. For all mononucleosis tests (n=6) and helicobacter pylori tests (n=3), and almost all pregnancy tests (16/18), the recommended IQC frequency was occasionally (Supplementary Table 2). However, if the number of patient samples is very low, such as 0-3 each month, IQC should be performed before every patient sample ( Table 2).
The Afinion (Abbott, CA, USA) multiassay analyzer obtained different IQC frequencies for the available analytes: CRP, HbA 1c , and urine albumin-creatinine ratio (ACR) obtained weekly, whilst cholesterol obtained monthly. In contrast, all analytes measured in the multiassay analyzers Cobas b101 (Roche Diagnostics International Ltd, Rotkreuz, Switzerland), Cobas h232 (Roche Diagnostics International Ltd, Rotkreuz, Switzerland), DCA (Siemens Healthcare, Erlangen, Germany), CUBE, and QuikRead (Aidian Oy, Espoo, Finland) obtained the same frequency for all assays ( Table 1, Supplementary Table 2). Table 3 shows that more laboratories will be recommended to increase rather than decrease their present IQC frequency when using the scoring system. For example, data registered in conjunction with an EQA survey for HbA 1c provided by Noklus in November 2019 showed that 55% (n=413) of the laboratories using Afinion analyzed IQC with a frequency in accordance with the scoring system, 41% (n=308) would be recommended to increase the IQC frequency, and 5% (n=36) would be recommended to decrease the IQC frequency after implementation of the scoring system.

Discussion
This study presents a consensus-based proposal for a scoring system that aims to provide differentiated and device-specific IQC frequency recommendations based on a model that considers different risks associated with the reporting of incorrect patient test results. This scoring system provides specific recommendations for primary healthcare laboratories. Its development was based on a literature review by and the professional judgment of experienced Noklus working group members, along with refinement based on input from laboratory advisors and medical specialists in laboratory medicine within Noklus. The risk of harm to the patient is a key factor in the ISO 15189:2012 [6]. Dividing analytes into two groups (moderate-risk and high-risk analytes) was challenging and may be controversial, since most tests with erroneous results can have serious consequences. We still believe that erroneous results from some analytes cause more harm than others, and we therefore categorized them based on what, to our knowledge, was the most common role of each analyte in primary healthcare consultations. The analyte scores and therefore frequency outcomes can easily be adjusted if clinicians assess the analytes differently within the scoring system. Analytes were categorized into two groups to indicate which analytes should be controlled more frequently than others, which is consistent with a collective opinion paper by Cooper et al. [19] arguing that tests with a high potential for harm to the patient should have more frequent IQC.
The categorization of POCT devices into four types in this study was based on the diversity of devices used in primary healthcare, and is similar to two previous studies categorizing devices into four types [3] and three types [17]. Our model considers type of device to be one of four factors that determine IQC frequency, while these previous studies primarily based IQC frequencies on only the device type [3,17]. The technology of POCT instruments is constantly improving, with some having built-in electronic checks or optical test cartridges that detect and minimize the risk of analytical errors. The two previously mentioned studies have argued, similarly to others, that devices should have less frequent IQC intervals when they have built-in electronic checks or optical test cartridges run every day that the device is used [3,17,18,29]. We also believe that these electronic checks may reduce analytical errors, but we have not included these in our scoring system as they, in our opinion, do not reduce the need for performing control samples due to them not controlling the entire analytical process [9,18,24,29]. Our experience is also that package inserts and user manuals from the manufacturer often lack information on which internal components are checked and which acceptance thresholds are used. It was therefore impossible to determine which built-in checks have an impact on the test result quality. However, instruments with integral IQC systems such as the iLine microINR that perform IQC as part of analyzing each patient sample may not require further IQC.
Our scoring system suggested a weekly IQC frequency for glucose meters. This is less frequent than that proposed in two previous studies, which recommended the analysis of IQC on glucose meters every day on which the instrument is used due to a lack of built-in checks [3,17]. However, this does agree with the results of a study conducted by Noklus, which indicated that one of the factors associated with good glucose meter performance was performing IQC weekly [7]. Previous studies have suggested that the number of tests performed must be considered when deciding an IQC frequency [3,8,19,30]. We propose to increase the frequency to daily if the practice analyzes more than 50 glucose samples each week. For practices that collect 0-3 patient samples each month, the recommendation was to analyze IQC before each patient sample to ensure the correct performance of the POC system. These numbers can though be defined differently in other environments.
Operators that do not adhere to test procedures are a major source of error in POCT [21], and more difficult or comprehensive procedures have higher probabilities of creating errors and producing erroneous test results [18]. Therefore, the possibility of user error has been included as a POCT, point-of-care testing; b IQC, internal quality control. c The recommended frequency change is based on participant-reported IQC frequency by primary healthcare laboratories participating in EQA surveys in  (response rates -%), compared to the recommendations that will be given when using the scoring system. The recommended frequency change is individually adjusted as recommended for each participant, taking into account the number of patient samples analyzed. a separate factor in our general scoring system. Classifying ease of use into three groups is also done when the Scandinavian evaluation of laboratory equipment for primary healthcare (SKUP), which Noklus is a part of, evaluates user-friendliness in real-life environments in their POCT device assessments [20]. The working group has identified many potential errors in the preanalytical, analytical and postanalytical phases of POCT testing that generally apply to many primary healthcare laboratories. However, the general scoring system does not implement local conditions in the risk assessment, such as for example in an individualized quality control plan (IQCP) [23,31]. However, in our opinion, making local adjustments to our general recommended IQC frequencies is something that is easy and thus feasible for the small primary healthcare laboratories to do rather than using more comprehensive tools themselves.
It could be argued that the device type and userfriendliness may be related, meaning that the least and most complex devices may be the easiest and most difficult to use, respectively. However, our study indicated that this is not the case. POCT devices that scored as moderately difficult to use ranged from the least complex (e.g., the qualitative group A streptococcus tests) to the most complex (e.g., blood-cell counters) device types.
Competence and training of personnel in analyzing patient samples and IQCs are important factors in reducing the risk of errors [1,13,18]. Therefore, practices with many operators may consider a local increase in IQC frequency to ensure staff user competence. It is also important that practices purchase POCT devices that are adapted to the knowledge and education of the employees in the practice [25]. Noklus evaluates device performance based on user manuals, SKUP evaluations, EQA-data, literature review and instrument-related problems reported by users and laboratory advisors.
Many large medical laboratories base their IQC frequencies and control rules on statistical quality control, such as six sigma and Westgard rules [32]. In theory, the six sigma calculations and Westgard rules can, of course, also be used in primary healthcare for POCT. In an ideal world, this would be preferable. However, our experience is that non-laboratory personnel find this cumbersome and difficult, and they are not easily motivated to for example calculate the analytical coefficient of variation (CV) and implement control rules. Therefore, in primary healthcare, we think that we must start pragmatically with simpler rules like for example accepting IQC if the control results are within the manufacturer's limits found in the package inserts. Hence, the IQC procedures from large medical laboratories cannot be adopted but must be specially designed to the POCT environment. For the moment, we think it is important to educate users that IQC procedures are important to implement, and as a start this is what we think is possible and meaningful with the resources available in primary healthcare. In the future, after evaluating the current recommendations, we will examine more closely whether more advanced control rules should be implemented for POCT in primary healthcare.
A limitation of our study is that the scoring system is based on a consensus approach from only one organization, but we think that the system can be adapted to other environments if desired. However, it is important to emphasize that any national accreditation-or regulatory requirements on IQC frequency should be followed, and the frequency should not be less than what the manufacturer requires. The scoring system has not been validated to test if the frequency recommendations are adequate to prevent errors and reduce patient risk. It is, however, difficult to design such a study in any laboratory, not only for primary healthcare but also for IQC procedures in large medical laboratories. The present paper is an attempt to systemise some of our experience on this topic and at the same time build on the few previous publications in this field. Noklus is now implementing this recommendation in primary healthcare laboratories and will monitor the degree of implementation through quality indicators collected in conjunction with each Noklus EQA survey. Data obtained before introducing the scoring system showed that more laboratories should increase rather than decrease their IQC frequency when introducing this scoring system (Table 3).
IQC plays an important role in quality assurance. However, there is currently no universal consensus on obtaining device-specific recommendations for when IQC should be analyzed. Our proposed points-based scoring system is easy to use and may induce systematic and effective assessments of the important factors for deciding IQC frequency in primary healthcare.