Monitoring quality indicators (QIs) is an important part of laboratory quality assurance (QA). Here, the Canadian Society of Clinical Chemists (CSCC) Point of Care Testing (POCT) and QI Special Interest Groups describe a process for establishing and monitoring QIs for POCT glucose testing.
Key, error prone steps in the POCT glucose testing process were collaboratively mapped out, followed by risk assessment for each step. Steps with the highest risk and ability to detect a non-conformance were chosen for follow-up. These were positive patient identification (PPID) and repeat of critically high glucose measurements. Participating sites were asked to submit aggregate data for these indicators from their site(s) for a one-month period. The PPID QI was also included as part of a national QI monitoring program for which fifty-seven sites submitted data.
The percentage of POCT glucose tests performed without valid PPID ranged from 0–87%. Sites without Admission-Discharge-Transfer (ADT) connectivity to POCT meters were among those with the highest percentage of POCT glucose tests performed without valid PPID. The percentage repeated critically high glucose measurements ranged from 0–50%, indicating low compliance with this recommendation. A high rate of discordance was also noted when critically high POCT glucose measurements were repeated, demonstrating the importance of repeat testing prior to insulin administration.
Here, a process for establishing these QIs is described, with preliminary data for two QIs chosen from this process. The findings demonstrate the importance of QIs for identification and comparative performance monitoring of non-conformances to improve POCT quality.
Monitoring quality indicators (QIs) is an important part of laboratory quality assurance (QA) and essential to patient safety. QIs are useful tools to aid in priority setting and for improving performance , , . Monitoring of QIs is also an important aspect of the quality improvement process and is included as an accreditation standard requirement for laboratories as well as for point of care testing (POCT) programs . POCT refers to diagnostic testing performed outside the laboratory environment, closer to the patient, often at the patient bedside. POCT is typically not performed by laboratory staff and instead by clinical staff, such as physicians or nurses, thus increasing the need for clear guidance and standardization for process monitoring.
The International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) and the Working Group, Laboratory Errors and Patient Safety (WG-LEPS) published multiple guidelines on QI monitoring, including a recommended set of 53 QI covering the entire testing process [2, 3, 5]. Oriented toward central laboratory processes, these recommendations do not include any POCT specific QIs. As the establishment of benchmarks is essential for improvement of quality, standardization of QIs is essential for the development of quality specifications [3, 5]. Studies have demonstrated that QI monitoring is an important component of quality assurance for POCT, specifically to identify areas for training and process improvement . Error rates in the analytical phase are relatively low compared to the pre and post-analytical phases in Laboratory medicine, partly because of the implementation of QI monitoring and quality specifications . POCT remains vulnerable to quality issues, especially where process checks are lacking, which can negatively affect patient care , particularly as POCT use expands.
Materials and methods
A prospective risk failure mode and effect analysis, adapted from that recommended by Janssens, was applied to prioritize potential QI for POCT . The most error prone steps in the POCT glucose testing process were identified by consensus of the authors, comprised of a group of twenty-two clinical biochemists representing nine Canadian provinces. These individuals have expertise in quality management of POCT and in supporting POCT programs across Canada. The mapping included key pre-analytical, analytical and post-analytical processes.
Once the process steps were agreed upon, the individual responding for each group of participating sites was asked to perform a self-assessment of risks for each step of the process according to the experience at their site. A total of fifteen sites participated, with representation from tertiary academic hospitals, community hospitals, community health clinics and long-term care. The risks were assessed according to a process adapted from that recommended by Janssens et al. . Risk at each step was assessed based on the probability of occurrence, the consequence of occurrence, and the chance of detecting the occurrence, as outlined in Table 1. A risk score at each step was calculated by multiplying the probability of failure (on a scale of 0–10) by the consequence of failure (on a scale of 0–10). Overall risk for each step was calculated as the average calculated by all sites. The overall chance of detection of a failure (on a scale of 1–3) for each step was calculated as the average result across all sites (n=15). Each step was classified as either “acceptable”, “suboptimal” or “unacceptable”, based on the criteria recommended by Janssens et al.  and outlined in Table 2. The overall risk score for each step in the process was determined based on data submitted from all participating groups of sites (n=15). Risk scores for each step were calculated by multiplying probability of failure (p) by consequence of failure (C). An average of the risk provided by each site was calculated, and these data are shown in Table 4. The chance of detection (D) for each step was calculated as the average chance of detection from data provided by all participating sites.
|of failure (p)||1||Less than once a year|
|2||Once a year|
|3||Several times a year|
|4||Once a month|
|5||Several times a month|
|6||Once a week|
|7||Several times a week|
|9||Several times a day|
|10||Many times a day|
|of failure (C)||1||Non-compliance with accreditation standards and no patient safety issue|
|2||Minor patient safety issue|
|3||Non-compliance with accreditation standards and minor patient safety issue|
|4||Moderate patient safety issue|
|5||Non-compliance with accreditation standards and moderate patient safety issue|
|6||Major patient safety issue|
|7||Non-compliance with accreditation standards and major patient safety issue|
|D||Chance of detecting a failure (D)||1||Low chance of detection|
|2||Medium chance of detection|
|3||High chance of detection|
Adapted from Janssens (2014). Annals of Clinical Biochemistry 51 (6): 695–704.
|Max risk 5, low chance of detection||Risk 5–30, low chance of detection||Risk >30, low chance of detection|
|Max risk 15, medium chance of detection||Risk 15–40, medium chance of detection||Risk >40, medium chance of detection|
|Max risk 25, high chance of detection||Risk 25–50, high chance of detection||Risk >50, high chance of detection|
Quality indicator choice and data for monitoring
The highest risk pre-analytical step, positive patient ID and one of the highest risk post-analytical steps, repeat of critically high glucose measurements, were chosen for monitoring by QI. Group consensus informed the QI data for monitoring. Positive patient ID refers to the POCT operator using a valid patient identifier for testing, as per their institutional policy. For example, medical record number or encounter number. For this indicator, the data collected were the percentage of all POCT glucose tests performed with an invalid patient identifier. For monitoring repeat of critically high glucose measurements, the percentage of critically high glucose results repeated within 10 min, either by POCT or in-laboratory, was tracked.
Analysis of quality indicator performance
Based on the results of the risk assessment described above and as a feasibility check, participating sites were asked to submit the total number of POCT glucose patient tests performed for a given month and the number of tests that were performed using an invalid patient identifier. An invalid patient identifier is defined as an identifier not associated with a known, registered patient. This included “dummy” or unregistered patient identifiers, used for testing unknown patients in urgent situations. In this study, the patient identifier is considered invalid when the local standard operating procedure for PPID is not compliant. An example of such non-compliance would be using an identifier other than that defined by the site procedure or use of an unregistered patient identifier without proper follow-up to ensure documentation of the result in the patient chart. The percentage of tests performed with an invalid patient identifier was calculated for each site. Participants extracted these data from their POCT middleware, laboratory information system (LIS) or electronic medical record (EMR).
Participants were also asked to submit the total number of critically high POCT glucose results that were repeated according to their local policy and the number of discordant repeat test results. The time limit for repeat tests was defined as a second POCT or sample collection for a laboratory measurement within 10 min of the initial result. For this study, a repeat test result was considered discordant if it was greater than a 20% difference from the initial measurement, regardless of the repeat method (POCT or laboratory).
For both QIs, the 25th and 75th percentiles of results were calculated, and sites were classified as either high, medium, or low performance, based on the IFCC recommendations for analysis of QI data [3, 5].
Process mapping for POCT glucose
A process map was developed, identifying the most error prone steps in the POCT glucose testing process, as shown in Table 3.
|Positive patient identification|
|Operator training – does a formal program exist?|
|Operator lock-out – can only trained operators use the instrument?|
|Reagent expiry date labeling|
|Washing of patient hands|
|Storage of reagent strips|
|Validation of reagents – is there a process?|
|Validation of QC material – is there a process?|
|Storage of meters on the clinical units|
|Sharing of operator IDs/inappropriate use of emergency operator ID (if applicable)|
|Proper PPE practices (wearing gloves etc.)|
|Inventory of management/lot sequestering|
|Storage of QC solutions on the clinical units|
|Choice of specimen – is there awareness by operators of when a capillary specimen may not be appropriate?|
|Meter validation – is there a process for this?|
|Wiping away first drop of blood prior to patient testing|
|QC – are operators performing QC according to the procedure?|
|QC lock-out – do the instruments have QC lock-out and is it on?|
|Follow-up on QC failures by clinical area. Is the follow-up appropriate (e.g., do they just repeat and repeat until it’s in?)|
|Testing procedure – is there a procedure and is it followed by the operators?|
|Meter interferences – are operators aware of interferences?|
|EQA – is there a formal EQA program?|
|Regular comparisons with the lab – are instruments regularly compared to the lab?|
|Results reporting – are operators compliant with charting requirements?|
|Cleaning of instrument|
|Meter communication with middleware/LIS – are there challenges?|
|Critical results reporting – is there a process for reporting?|
|Critical results follow-up – are processes adhered to if they exist?|
|Periodic review of reference intervals and/or critical values|
|Lab confirmation for discrepant results. Do clinical areas confirm suspicious results?|
|Proper disposal of samples/lancets|
|Docking of meters (if applicable). Clinical compliance with docking for charging and results transmission.|
EQA, external quality assurance; ID, identification number; LIS, laboratory information system; PPE, personal protective equipment; QC, quality control.
|Step of the process||Phase||Risk (C × p)||Detection||Overall risk|
|Positive patient ID||Pre-analytical||28.1||2.1||Suboptimal|
|Washing of patient hands||Pre-analytical||23.7||1.0||Suboptimal|
|Sharing of operator IDs/inappropriate use of emergency operator ID (if applicable)||Pre-analytical||19.0||1.3||Suboptimal|
|Wiping away first drop||Pre-analytical||18.3||1.0||Suboptimal|
|Choice of specimen – is there awareness by operators of when a capillary specimen may not be appropriate?||Pre-analytical||16.3||1.0||Suboptimal|
|Proper PPE practices (wearing gloves etc.)||Pre-analytical||15.5||1.2||Suboptimal|
|Reagent expiry date labeling||Pre-analytical||14.7||1.8||Acceptable|
|Storage of reagent strips||Pre-analytical||12.0||1.3||Suboptimal|
|Storage of QC solutions on the clinical units||Pre-analytical||10.6||1.4||Suboptimal|
|Operator lock-out – can only trained operators use the instrument?||Pre-analytical||9.0||2.1||Acceptable|
|Storage of meters on the clinical units||Pre-analytical||8.5||2.1||Acceptable|
|Operator training – does a formal program exist?||Pre-analytical||7.5||2.4||Acceptable|
|Validation of QC material – is there a process for this?||Pre-analytical||3.2||2.6||Acceptable|
|Validation of reagents – is there a process for this?||Pre-analytical||2.9||2.8||Acceptable|
|Inventory of management/lot sequestering||Pre-analytical||2.1||2.8||Acceptable|
|Meter validation – is there a process for this?||Pre-analytical||1.6||2.8||Acceptable|
|Meter interferences – are operators aware of interferences?||Analytical||20.7||1.3||Suboptimal|
|Testing procedure – is there a procedure and is it followed by the operators?||Analytical||17.6||1.3||Suboptimal|
|Follow-up on QC failures by clinical area. Is the follow-up appropriate (e.g., do they just repeat and repeat until it’s in?)||Analytical||14.5||2.0||Acceptable|
|Regular comparisons with the lab – are instruments regularly compared to the lab?||Analytical||8.5||2.8||Acceptable|
|QC – are operators performing QC according to the procedure?||Analytical||8.3||2.3||Acceptable|
|QC lock-out – do the instruments have QC lock-out and is it on?||Analytical||1.5||2.8||Acceptable|
|EQA – is there a formal EQA program?||Analytical||0.7||2.8||Acceptable|
|Lab confirmation for discrepant results. Do clinical areas confirm suspicious results?||Post-analytical||27.2||1.3||Suboptimal|
|Critical results follow-up – are processes adhered to if they exist?||Post-analytical||23.5||1.7||Suboptimal|
|Critical results reporting – is there a process for reporting?||Post-analytical||17.3||2.0||Suboptimal|
|Cleaning of instrument||Post-analytical||14.7||1.2||Suboptimal|
|Results reporting – are operators compliant with charting requirements?||Post-analytical||13.0||1.8||Acceptable|
|Meter communication with middleware/LIS – are there challenges?||Post-analytical||10.3||2.3||Acceptable|
|Proper disposal of samples/lancets||Post-analytical||7.8||1.2||Suboptimal|
|Docking of meters (if applicable). Clinical compliance with docking for charging and results transmission.||Post-analytical||7.8||2.5||Acceptable|
|Periodic review of reference intervals and/or critical values||Post-analytical||3.6||2.8||Acceptable|
EQA, external quality assurance; ID, identification number; LIS, laboratory information system; PPE, personal protective equipment; QC, quality control; POCT, point of care testing.
Using the Risk Assessment Criteria in Table 2, each step in the process map was assigned an overall risk as follows: acceptable, suboptimal, or unacceptable. Upon analysis, none of the steps were assigned an unacceptable risk based on the information provided by participating sites.
Half (8/16) of the pre-analytical, and more than half (5/9) of the post-analytical steps were assigned as suboptimal risks, whereas most analytical steps were assigned acceptable risk. With data accessibility, such as from POCT middleware, laboratory information system, as a potential barrier for QI reporting, steps with errors deemed to have a higher chance of detection were prioritized.
Indicators chosen for routine monitoring
The highest risk step in the POCT glucose process was identified as positive patient identification, followed by lab confirmation of discrepant results, washing of patient hands prior to testing, confirmation of critical results and appropriate follow-up of critical results. Based on group discussion and consensus, two steps were selected for initial QI monitoring: positive patient ID and repeat testing for critically high glucose results. These two steps were chosen due to their high-risk rating and the feasibility of detecting error. Group consensus informed the QI data to collect for both chosen steps.
Field validation of high priority QIs
Positive patient identification quality indicator (PPID QI)
Preliminary validation of this QI was performed within the working group, where the data submission criteria were determined as the number of POCT glucose performed with invalid patient ID as a percentage of the total number of POCT glucose performed in one month. Members of the working group, representing seventeen sites, submitted these data. The percentage of POCT glucose performed with invalid patient ID ranged from 0.1 to 72.6% with an average of 12% and median of 6%. The Program for Quality Indicators Comparison of the Société Québécoise de Biologie Clinique (SQBC) was then used to implement the PPID QI developed by the CSCC SIG POCT QIs to facilitate data submission from laboratories across Canada. This QI comparison program was developed by the SQBC who collaborates with the CSCC to promote participation from laboratories across Canada as well as with the WG LEPS of the IFCC to participate in the international QI standardization effort . To further validate the QI, institutions across Canada with POCT glucose programs were invited to submit data for one month in 2022. Fifty-seven sites across Canada submitted data for the PPID QI. The percentage of glucose tests performed with an invalid patient ID ranged from 0 to 87%. The percentage ranged from 26 to 87% in sites without an ADT connection to the POCT meters and 0–67% in sites with an ADT connection (Table 5, Figure 1), which reached statistical significance, demonstrating the importance of connectivity. The 25th and 75th percentiles for all submitted data were calculated, based on the recommendation by the WG-LEPS of the IFCC. Data were also separated by sites with ADT vs. without ADT connectivity as shown in Table 5. Sites with the percentage of POCT glucose tests performed without valid PPID less than the calculated 25th percentile are considered high performing sites, moderate performing with a percentage between the calculated 25th and 75th percentiles and low performing if the percentage of POCT glucose tests performed without valid PPID exceeds the calculated 75th percentile.
ADT, admission, discharge, transfer; PPID, positive patient identification. Low performing sites are those with percentage of POCT glucose tests performed without valid PPID >75th percentile, moderate between the 25th–75th percentile and high performing if the percent POCT glucose performed without valid PPID is <25th percentile.
Repeat of critical glucose results
Participants were asked to submit the total number of critically high POCT glucose results at their site for one month in 2022, as well as the number of critically high results that were repeated within 10 min (n=12 sites provided data). Repeat tests could be either measured by a POCT or a central laboratory method. The percentage of discordant repeat results were also submitted (Table 6). The percentage of repeated results ranged from 0 to 50% (target 100%) and the percentage of discordant repeat results ranged from 0 to 56% (target 0%). Discordant results were defined as ≥20% different from the initial result obtained.
aSite data not included in calculations as this site does not have a critical POCT glucose repeat policy. Discordant repeat results were defined as repeats ≥20% different than the original result. n/a means no data were submitted from this participant.
The 25th and 75th percentiles of performance were calculated for the percentage of repeated results. Sites were considered low performing for repeat of critical results when the percent repeated was less than 7% (25th percentile), moderate between 7 and 37% and high if more than 37% (75th percentile) of results were repeated.
In this report, we demonstrate an approach to establishment and monitoring of QI for POCT. The process begins with mapping of the testing process, to identify error prone steps, followed by risk assessment of each step, to prioritize steps for QI monitoring, based on risk and ability to monitor. The data presented here illustrate the value and importance of establishing, monitoring and follow-up of QIs, specifically for POCT using blood glucose meters as an example. Sites with the poorest compliance with the use of a valid patient identification for POCT glucose testing were those that do not have ADT connectivity to the POCT meters with results transmission to the electronic medical record. These findings demonstrate the importance of investments in these capabilities by healthcare facilities. The non-compliance data presented here include instances where an unregistered, or “dummy” patient identifier was used for testing with no indication that a result was documented in the correct patient chart. Many sites employ unregistered patient identifiers as a safeguard, to be used only in emergency situations when the patient identification is unknown. Generally, there are follow-up processes in place for results to be documented in the correct patient chart once the patient’s medical record number is reconciled and the emergency has been dealt with. Sites participating in this study reported low compliance with the follow-up charting processes and frequent inappropriate use of unregistered patient identifiers for testing. Invalid PPID suggests that these results are often not documented in the patient chart and are therefore missing from the patient’s medical record.
We found generally low operator compliance with repeat testing of critically high POCT glucose results. Even in sites deemed “high performing” based on the 25th and 75th percentiles calculated, only up to half of all critically high results were repeated. Low compliance exists despite all but one of the participating institutions having a recommendation in place that repeat testing should be performed or a formal policy stating that all critically high results must be repeated. This also demonstrates the importance of professional judgment on what constitutes acceptable performance along with interpreting results within the context of the clinical picture.
The low rate of compliance with repeating critically high glucose measurements is concerning given the high rates of discordant repeat results, up to 56% in one site. These data demonstrate the importance of repeating critically high POCT glucose results before dosing insulin. Our findings are not dissimilar to previous studies, which found insufficient handwashing as the primary cause of falsely elevated POCT glucose results .
The approach taken here to establish QI for POCT has been used by others . A recent study used a Failure Mode and Effects Analysis (FEMA) to perform a risk assessment for potential pre-analytical errors when performing POCT blood gas analysis . The authors identified twelve quality assurance or process steps in the pre-analytical phase and recommended key performance indicators (KPI) to monitor for several of the steps. Positive patient identification was among the steps with highest risk of failure and identified with the recommended calculated metric as the percentage of unidentified samples, consistent with QI recommended in the current study.
A strength of our study is the inclusion of field validation of QI, with data gathered from several institutions across health jurisdictions and located throughout Canada. Gathering of these data was challenging for sites that do not employ connectivity for POCT instruments, resulting in manual data extraction and analysis. Data extraction from the laboratory information system of middleware was reported as relatively straightforward for sites with connectivity.
The main limitation of the current study is that the process map and risk assessment are based on the self-assessment and experiences of the authors, who are largely Clinical Biochemists and Laboratory Directors who oversee POCT in their institutions within Canada. The process map and risk assessment do not include the views of the individuals who perform POCT, meaning there may be factors or situations that have been overlooked. Another limitation is that the error rates identified for these two QIs are a combination of non-compliance with the procedure as well as the ‘true error’.
In conclusion, monitoring QIs for POCT are important to ensure quality and patient safety. Here a Canadian consensus process for establishing these QIs is described. The findings demonstrate the utility of QIs for identification and comparative performance to improve the quality of POCT.
Research funding: None declared.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: Authors state no conflict of interest.
Informed consent: Not applicable.
Ethical approval: Not applicable.
2. Plebani, M, Astion, ML, Barth, JH, Chen, W, de Oliveira Galoro, CA, Escuer, MI, et al.. Harmonization of quality indicators in laboratory medicine. A primary consensus. Clin Chem Lab Med 2014;52:951–8. https://doi.org/10.1515/cclm-2014-0142.Search in Google Scholar PubMed
3. Sciacovelli, L, Padoan, A, Aita, A, Basso, D, Plebani, M. Quality indicators in laboratory medicine: state-of-the-art, quality specifications and future strategies. Clin Chem Lab Med 2023;61:688–95. https://doi.org/10.1515/cclm-2022-1143.Search in Google Scholar PubMed
4. ISO 15189:2022. Medical laboratories – requirements for quality and compliance.Search in Google Scholar
5. Sciacovelli, L, Lippi, G, Sumarac, Z, West, J, del Pino Castro, IG, Furtado Vieira, K, et al.. IFCC Working Group on Laboratory Errors and Patient safety. Quality indicators in laboratory medicine: the status of the progress of IFCC Working Group “laboratory errors and patient safety” project. Clin Chem Lab Med 2017;55:348–57. https://doi.org/10.1515/cclm-2016-0929.Search in Google Scholar PubMed
6. Shaw, JL, McCudden, CR, Colantonio, DA, Booth, RA, Lin, DC, Blasutig, IM, et al.. Effective interventions to improve the quality of critically high point-of-care glucose results. Prac Lab Med 2020;22:e00184. https://doi.org/10.1016/j.plabm.2020.e00184.Search in Google Scholar PubMed PubMed Central
7. Plebani, M, Sciacovelli, L, Marinova, M, Marcuccitti, J, Chiozza, ML. Quality indicators in laboratory medicine: a fundamental tool for quality and patient safety. Clin Biochem 2013;46:1170–4. https://doi.org/10.1016/j.clinbiochem.2012.11.028.Search in Google Scholar PubMed
8. Brun, M, Fuzery, AK, Henschke, B, Rozak, K, Venner, AA. Identifying sources of error and selecting quality indicators for point of care testing. Prac Lab Med 2021;25:200216.10.1016/j.plabm.2021.e00216Search in Google Scholar PubMed PubMed Central
9. Janssens, PMW. Practical, transparent prospective risk analysis for the clinical laboratory. Ann Clin Biochem 2014;51:695–704. https://doi.org/10.1177/0004563214521160.Search in Google Scholar PubMed
11. Van Hoof, V, Bench, S, Soto, AB, Luppa, PP, Malpass, A, Schilling, UM, et al.. Failure Mode and Effects Analysis (FMEA) at the preanalytical phase for POCT blood gas analysis: proposal for a shared proactive risk analysis model. Clin Chem Lab Med 2022;60:1186–201. https://doi.org/10.1515/cclm-2022-0319.Search in Google Scholar PubMed
© 2023 the author(s), published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.