BY 4.0 license Open Access Published by De Gruyter December 16, 2020

Variability of cycle threshold values in an external quality assessment scheme for detection of the SARS-CoV-2 virus genome by RT-PCR

Christoph Buchta, Irene Görzer, Peter Chiba, Jeremy V. Camp, Heidemarie Holzmann, Elisabeth Puchhammer-Stöckl, Maximilian Mayerhofer, Mathias M. Müller and Stephan W. Aberle

Abstract

Objectives

The qualitative results of SARS-CoV-2 specific real-time reverse transcription (RT) PCR are used for initial diagnosis and follow-up of Covid-19 patients and asymptomatic virus carriers. However, clinical decision-making and health management policies often are based additionally on cycle threshold (Ct) values (i.e., quantitative results) to guide patient care, segregation and discharge management of individuals testing positive. Therefore, an analysis of inter-protocol variability is needed to assess the comparability of the quantitative results.

Methods

Ct values reported in a SARS-CoV-2 virus genome detection external quality assessment challenge were analyzed. Three positive and two negative samples were distributed to participating test laboratories. Qualitative results (positive/negative) and quantitative results (Ct values) were assessed.

Results

A total of 66 laboratories participated, contributing results from 101 distinct test systems and reporting Ct values for a total of 92 different protocols. In all three positive samples, the means of the Ct values for the E-, N-, S-, RdRp-, and ORF1ab-genes varied by less than two cycles. However, 7.7% of reported results deviated by more than ±4.0 (maximum 18.0) cycles from the respective individual means. These larger deviations appear to be systematic errors.

Conclusions

In an attempt to use PCR diagnostics beyond the identification of infected individuals, laboratories are frequently requested to report Ct values along with a qualitative result. This study highlights the limitations of interpreting Ct values from the various SARS-CoV genome detection protocols and suggests that standardization is necessary in the reporting of Ct values with respect to the target gene.

Introduction

Virus genome-specific real-time reverse transcription PCR (RT-qPCR) is the gold standard for laboratory diagnosis of infections with SARS-CoV-2. Apart from a binary classification of samples into virus-positive or virus-negative, some have advocated for providing a (semi)quantitative parameter for the estimation of viral loads [1]. The discussion about the usefulness of cycle threshold (Ct) values as a basis for diagnostic, prognostic and therapeutic decisions in the management of SARS-CoV-2 infected patients is ongoing [2], [3]. Recently, the guidelines for the discharge management of patients were based not only on time-lines of infection (a minimum of 10 days since the development of first symptoms and an asymptomatic period of at least 48 h), but also on momentary Ct values (time consideration and/or a Ct value >30) [4], [5]. These potentially significant decisions – both from a medical and an epidemiological point of view – require a high degree of reliability from the laboratory results on which they are based. Qualitative results have for the most part proven reliable and only some of the test protocols fail to detect SARS-CoV2 at lower virus loads (higher Ct values) [6], [7]. However, data on comparability are presently not available. This prompted us to compare Ct values from different test systems. We decided to include a clearly positive sample, a weakly positive sample (i.e., near the limit of detection), as well as a sample with a Ct value that is close to the proposed guideline for discharge (Ct=30).

Results obtained in external quality assessment (EQA) schemes are ideally suited to allow comparing the analytical performance of EQA participants and/or different protocols used by these laboratories. In order to examine comparability, we analyzed the Ct values reported in the second distribution of the Austrian national SARS-CoV-2 virus genome detection EQA scheme. This EQA scheme was performed in cooperation between the Center for Virology of the Medical University of Vienna and the Austrian Association for Quality Assurance and Standardization of Medical and Diagnostic Tests (ÖQUASTA).

Materials and methods

The EQA test panel consisted of five samples (900 μL each), three positive and two negative. Positive samples were obtained by 100-fold dilution of three individual SARS-CoV-2 positive oro-nasopharyngeal swab samples with 1 mL of sterile sodium chloride solution (1.5 mL swab samples + 148.5 mL 0.9% NaCl). The negative samples consisted of the diluent (0.9% NaCl solution). In order to guarantee stability of SARS-CoV-2 genomic RNA, all samples were stored at −80 °C and one panel was tested before shipment as follows: after thawing, samples were kept at ambient temperature for two days (anticipated maximum shipping time) before further storage at 4 °C for four days (maximal expected storage time before testing). Thereafter, samples were quantified with an in-house SARS-CoV-2 molecular detection assay. For this, nucleic acid (NA) was extracted from a 200 μL aliquot of the original sample using the NucliSENS easyMAG platform (bioMérieux, France). The elution volume was 50 μL. For real-time RT-PCR, each 25 μL reaction mixture contained 12.5 μL of 2× reaction buffer, 0.4 μL of a 50 mM magnesium sulfate solution, 1 μL of SuperScript III RT/ Platinum Taq Mix (SuperScript III One-Step RT-PCR System, Invitrogen, Germany), 0.5 μL each of LightMix® Modular SARS-CoV (COVID19) E-gene and LightMix® Modular EAV RNA Extraction Control (TIB Molbiol, Berlin, Germany) and 10 μL of eluted NA as the template. Thermal cycling was performed using a Light Cycler 480 II (Roche, Vienna, Austria) under the following conditions: reverse transcription at 55 °C for 20 min; denaturation at 94 °C for 3 min; 45 amplification cycles of 94 °C for 15 s and 58 °C for 30 s (acquisition steps); a final incubation at 40 °C for 30 s. The Ct values of the three positive samples were repeated 6 times and mean and 95% confidence interval were as follows: S1=29.47 (29.41–29.52); S3=24.37 (24.05–24.68); and S5=35.65 (34.71–36.59) for the E-gene of the SARS-CoV-2 genome. Number of replicates was not determined. Samples S2 and S4 (containing a sodium chloride solution only) were tested negative.

Samples were distributed to participant laboratories by overnight delivery at ambient temperature conditions. Participants were instructed to either immediately test or alternatively store the material at 4 °C until analysis for a maximum of four days after arrival and to perform testing in the same way as for routine samples. Individual laboratories which intended to participate with more than one protocol received independently shipped test panels for each test system.

Results of a qualitative interpretation (positive, negative) were to be submitted to the EQA provider via a web portal, email, fax or on paper in regular mail within seven days. Individual results were scored as either correct or incorrect and in addition to individual reports, data for all test systems were made available in a general report. Participants were asked to report target gene(s) and Ct values along with qualitative results. Only the latter formed a basis for the assessment of the performance of participants in this EQA scheme. The schematic sequence of an EQA round is shown in Figure 1.

Figure 1: 
Schematic sequence of an EQA round.

Figure 1:

Schematic sequence of an EQA round.

Results

Sixty-six laboratories participated with a total of 101 individual test systems. These included combinations of extraction platforms and reagents and RT PCR platforms and reagents. With respect to the former, 31 different extraction platforms, 14 manual methods, 34 commercial extraction reagents, and four in-house extraction reagents were reported by the participants. Concerning PCR platforms, 28 different systems, 36 commercial PCR reagents and six in-house reagents were reported. In total, 63 different combinations of extraction and detection systems were used; these are shown in Table 1. Fifty-five laboratories participated with one test system/protocol, and 11 laboratories used more than one test system/protocol. In particular, nine laboratories used two, one lab used three, and one lab used five distinct test systems. For the negative samples S2 and S4, 199 correct negative results (98.5%) and two false positive results (1.0%) were reported; two participants did not report results for S2 and/or S4. A total of 91 out of 101 test systems (90.1%) reported all three positive samples correctly as positive, 10 (10%) found S5 (Ct value of E-gene 35.7) to be (false) negative. A total of 466 individual Ct values were reported for 92 test systems. For the E-gene, 163 Ct values were reported: 159 for the N-gene; 41 for the S-gene; 40 for the RdRp-gene; 34 for the ORF1ab-gene, and 29 for other gene targets (“nd” in Figures 2 and 3, including CDC-N1, CDC-N2, CDC-N3, ORF1a, ORF1b, 5′-UTR) that were combined into one group for statistical analysis because of a low number of results for each individual test protocol.

Table 1:

C t values as determined by diverse test systems for the five most frequently reported gene targets.

Platform/reagent, n Sample E (n=163) N (n=159) S (n=41) RdRp (n=40) ORF1ab (n=34) Other genes (n=29)
Mean S.D. Range 95% C.I. Mean S.D. Range 95% C.I. Mean S.D. Range 95% C.I. Mean S.D. Range 95% C.I. Mean S.D. Range 95% C.I. Mean S.D. Range 95% C.I.
All test systems 1 28.9 26.4–35.7 29.6 19.1–42.5 28.6 22.9–32.6 30.4 19.1–39.7 29.0 25.8–33.3 30.6 18.0–34.0
1.94 28.4–29.4 3.21 28.7–30.5 2.54 27.2–29.9 3.36 28.1–32.8 1.92 27.8–30.2 5.0 25.3–32.4
3 24.8 22.3–30.7 25.6 14.2–43.2 24.9 18.1–28.6 25.9 14.2–33.4 24.9 20.4–30.5 24.7 13.7–29.0
1.84 24.4–25.3 3.66 24.6–26.7 2.94 23.4–26.5 4.15 23.6–28.2 2.62 23.2–26.6 4.33 21.6–27.8
5 36.3 33.5–40.0 37.0 28.1–40.4 35.7 29.7–39.7 36.9 28.1–40.0 35.2 32.2–36.9 35.0 23.1–41.0
1.39 35.9–36.7 2.47 36.3–37.7 3.31 33.2–38.3 3.63 34.5–39.3 1.67 34.1–36.4 6.20 30.3–39.8
GeneXpert® + Xpert® Xpress SARS-CoV-2 (n=23) 1 27.8 26.8–28.7 30.2 29.4–31.1
1.31 27.6–28.0 0.46 30.0–30.4
3 23.9 23.3–25.4 26.3 25.6–27.8
1.29 23.7–24.1 0.47 26.1–26.5
5 36.1 34.5–38.9 38.8 37.2–40.4
1.71 35.6–36.6 0.95 38.4–39.2
Cobas® 6800/Cobas® 8800 + Cobas® SARS-CoV-2 (n=8) 1 29.3 28.4–30.8 28.9 28.3–29.9
0.92 28.6–30.0 0.72 28.1–29.7
3 25.0 23.8–26.5 24.7 23.9–26.0
0.96 24.2–25.8 0.74 23.9–25.5
5 36.3 34.8–37.9 36.0 35.6–36.8
0.87 35.4–37.2 0.42 35.6–36.5
RealStar® SARS-CoV-2 RT-PCR kit RUOa (n=8, 7 different protocols) 1 29.4 27.3–31.8 29.6 26.2–31.1
1.68 28.1–30.7 1.80 28.2–31.0
3 25.7 23.5–28.0 26.0 22.4–28.1
1.75 24.3–27.0 1.95 24.5–27.5
5 36.8 35.0–38.5 36.9 35.3–37.8
1.80 34.3–39.4 1.42 35.0–39.8
ViroReal® kit SARS-CoV-2 & SARSb (n=7, 7 different protocols) 1 29.3 25.9–32.2
2.25 27.2–31.3
3 25.1 22.4–27.2
1.91 23.3–26.9
5 35.5 32.7–37.2
1.56 34.1–37.0
LightMix® modular SARS CoV (Covid19) E-genec (n=7, 7 different protocols) 1 29.6 27.0–33.1
2.09 27.1–31.0
3 26.3 22.9–30.7
3.33 22.2–28.3
5 36.8 35.3–38.4
1.60 34.5–37.5
Other systemsd (n=39, 32 different protocols) 1 29.5 26.4–34.9 29.5 19.1–42.5 27.9 22.9–32.6 30.4 19.1–39.7 29.1 25.8–33.3 30.6 18.0–34.0
2.61 28.2–32.5 4.60 27.3–31.2 2.85 25.2–30.1 4.50 28.0–32.8 2.75 26.2–32.0 5.0 25.3–32.4
3 25.0 22.3–29.8 25.2 14.2–43.2 24.1 18.1–28.6 25.9 14.2–33.4 25.1 20.4–30.5 24.7 13.7–29.0
2.22 24.2–27.4 5.38 22.9–27.4 2.85 20.9–26.9 4.28 23.6–28.2 3.80 21.1–29.1 4.33 21.6–27.8
5 36.2 33.5–40.0 35.5 28.1–37.9 35.1 29.7–39.7 36.9 28.1–39.7 34.1 32.2–36.9 35.0 23.1–41.0
2.36 34.7–39.1 2.57 34.4–36.7 3.93 29.6–39.3 3.56 34.5–39.3 2.27 30.5–37.7 6.20 30.3–39.8

  1. Means, ranges, standard deviations (S.D.) and 95% confidence intervals (C.I.) are shown of Ct values obtained by two automated platforms, three frequently used PCR reagents and the remaining other test systems.

    aUsed in the following combinations (Extraction platform/extraction reagent/PCR reagent): Alto Star AM16/Alto Star Purification Kit 1.5/Altostar DT16; Alto Star AM16/Alto Star Purification Kit 1.5/CFX96 Touch; EASYMAG/EMAG/CFX96 Touch; MaNa Pure LC 2.0/MagNa Pure Compact RNA Isolations Kit/LightCycler 480; Macherey & Nagel/Macherey & Nagel/Light Cycler 96/Roche; manual/QIAamp DNA Mini Kit/LightCycler 480; manual/QIAamp DNA Mini Kit/Rotor Gene Q.

    bUsed in the following combinations: Hamilton NGS STAR/Macherey & Nage/Roche/Cobas z 480; Macherey & Nagel/Macherey & Nagel/LightCycler 480; Maxwell/Promega/LightCycler 480; Zymo/Quick-RNA/Zymo/manual; manual/Favorgen Viral RNA/DNA Extraction Kit/MIC; manual/Macherey & Nagel/CFX96 Touch; manual/säulen-basierte RNA-Extraktion/LightCycler 480.

    cUsed in the following combinations: MaNa Pure Compact/MagNa Pure Compact Nucleic Acid Isolations KIT I/LightCycler 480; MaNa Pure Compact/MagNa Pure Compact Nucleic Acid Isolations KIT I/Roche/Cobas z 480; QIACube HT/QIAamp DNA Mini Kit/LightCycler 480; EASYMAG/EASYMAG/LightCycler 480; KingFisher Flex 96/BioExtract SuperBall/ABI 7500; KingFisher Flex 96/IndiMag Pathogen Kit/ABI 7500; chemagic 360-D (Perkin Elmer)/chemagic Viral DNA/RNA Kit special H96/LightCycler 480.

    d“other” comprises (Extraction platform/extraction reagent/PCR platform/PCR reagent): BioSprint 96/IndiMag Pathogen Kit/Q-Tower Analytik Jena/in-house Assay; BioSprint 96/IndiMag Pathogen Kit/Mastercycler nexus/in-house Assay; Abbott sp2000/DNA Extraction Abbott/ABI 7500/Abbott RT Sars-CoV-2; Abbott-Alinity/Alinity m SARS-Cov-2 Assay/Abbott-Alinity/Abbott RT Sars-CoV-2; Abbott-Alinity/Alinity m SARS-Cov-2 Assay/Abbott-Alinity/Alinity m SARS-Cov-2 Assay; Hamilton Nimbus/Universal Extraction System Seegene/CFX96 Touch/Allpex 2019-nCoV (Seegene); Hamilton Nimbus/STARMag/CFX96 Touch/Allpex 2019-nCoV (Seegene); GenoXtract RNA/DNA Hain/GenoXtract/CFX96 Touch/Anchor SARS-CoV-2 PCR Kit; BD MAX/BD MAX ExK TNA-3/BD MAX/BD MAX SARS-CoV-2; QIACube HT/QIACube HT Viral RNA/AriaMX (Aligent)/Brilliant III Ultra-Fast QPCR Master Mix; manual/QIAamp DNA Mini Kit/Roche/Cobas z 480/ cobas/SARS-Cov-2; Life Ready 1000/Life Ready 1000/Life Ready 1000/COVID-19 Nucleic acid (RNA) Detection Kit; MagCore Plus II/MagCore Viral Nucleic Acid Extraction/Rotor Gene Q/EliGene COVID 19 Basic A RT; VERSANT kPCR Sample Prep/VERSANT kPCR Sample prep 1.0 Reagent Kit/VERSANT kPCR Amp and Detection module/FTD SARS CoV-2; MaNa Pure Compact/MagNa Pure Compact Nucleic Acid Isolations KIT I/LightCycler 480/genesig Real-Time PCR; Zymo/Quick-RNA/Zymo/Q-Tower Analytik Jena/GenomeCoV19 Detection kit; KingFisher Flex 96/IndiMag Pathogen Kit/ABI 7500/in-house Assay; manual/Lucigen Quick Extract/CFX96 Touch/in-house Assay; KingFisher Duo Prime/MagMax Viral/Pathogen Kit/Q-Tower Analytik Jena/in-house Assay; Ideal TM96HP/IndiMag Pathogen Kit/Stratagene MX 3005p/in-house Assay; QIACube HT/QIAamp DNA Mini Kit/LightCycler 480/Light Mix Modular; MaNa Pure Compact/MagNa Pure Compact Nucleic Acid Isolations KIT I/Roche/Cobas z 480/Light Mix Modular; MaNa Pure Compact/MagNa Pure Compact Nucleic Acid Isolations KIT I/LightCycler 480/Light Mix Modular; AnalytikJena/CyBio-Felix/AnalytikJena Innuprep Virus TS RNA Kit 2.0 FX/Applied Biosystem QS7/Luna Universal/ One-Step-RT-qPCR (Covid); Q-Tower/Q-Tower/Q-Tower Analytik Jena/Luna Universal/ One-Step-RT-qPCR (Covid); KingFisher Flex 96/in-house/CFX96 Touch/ Luna Universal/ One-Step-RT-qPCR (Covid); manual/in-house/MIC/MutaPlex/RT-PCR Kit; manual/Lucigen Quick Extract/CFX96 Touch/NEB 2x RT-Lamp- Mastermix; NeuMoDX/NeuMoDx 96/NeuMoDX/NeuMoDx SARS-CoV-2 Test Strip; NeuMoDX/in house/NeuMoDX/NeuMoDx SARS-CoV-2 Test Strip; Zymo/Quick-RNA/Zymo/Elite Igenius/PhoenixDX; manual/QIAamp DNA Mini Kit/7900HT Fast Real Time PCR/RADI Covid-19 Detection Kit; manual/in-house/MIC/Real AccurateR Quadruplex/SARS-CoV-2 Multiplex RT; manual/column-based RNA-Extraction/CFX96 Touch/Real Fast PCR; IndiMag 48/IndiMag Pathogen Kit/CFX96 Touch/Reliance One-Step Multiplex Supermix; MaNa Pure Compact/MagNa Pure Compact RNA Isolations Kit/Roche/Cobas z 480/Rida Gene; manual/column-based RNA-Extraction/Rotor Gene Q/SARS-CoV-2-N+RdRp Cubedx; Simplexa Liaison MDX Realtime PCR/Simplexa Liaison MDX Realtime PCR/SIMPLEXA Liaison MDX Realtime PCR/SIMPLEXA Covid-19 Direct Reaction Mix; manualQIAamp/DNA Mini Kit/QuantStudio 12k flex /TaqPath TM; BD MAX/BD MAX ExK TNA-3/BD/Viasure SARS-CoV-2 (S gene); manual/column-based RNA-Extraction/LightCycler 480/Virella SARS-CoV-2 seqc RT-PCR Kit 2.0; QIAsymphony /QIAsymphony DSP Virus/Pathogen Kit/Rotor Gene Q/Virella SARS-CoV-2 seqc RT-PCR Kit 2.0; VitaPCR/VitaPCR/VitaPCR /VitaPCR SARS-CoV-2 Assay.

Figure 2: 
Boxplots of Ct values obtained for each sample and the indicated genes.
Boxplots represent quartiles, showing 50% IQR and median within the box. nd = test systems using other, less frequently used, gene targets.

Figure 2:

Boxplots of Ct values obtained for each sample and the indicated genes.

Boxplots represent quartiles, showing 50% IQR and median within the box. nd = test systems using other, less frequently used, gene targets.

Figure 3: 
Deviations of Ct values from the respective mean.
Bars represent absolute deviations of Ct values obtained by individual test systems from the respective mean value. Numbers are pseudonyms for participant test systems. nd = test systems using other, less frequently used, gene targets.

Figure 3:

Deviations of Ct values from the respective mean.

Bars represent absolute deviations of Ct values obtained by individual test systems from the respective mean value. Numbers are pseudonyms for participant test systems. nd = test systems using other, less frequently used, gene targets.

Regardless of the analyzed gene targets, mean Ct values were 29.2, (range 18.0–42.5; standard deviation 3.60) in sample S1, 25.2 (13.7–43.2; 3.52) in sample S3, and 36.4 (23.1–41.0; 4.14) in sample S5. Means, ranges, standard deviations and 95% confidence intervals of Ct values obtained for individual targets, the E-, N-, S-, RdRp-, ORF1ab-gene are shown in Figure 2 and Table 1.

For further analysis of variability of Ct values, results were grouped according to detection systems (Table 1). Because of the diversity of test systems, a statement about the performance can only be made for a few of them, since most systems or combinations of platforms were used by three or fewer participants. The two most frequently used automated test systems were GeneXpert® + Xpert® Xpress SARS-CoV-2 (Cepheid Europe SAS, Maurens-Scopont, France) (n=23) and cobas® 6800/cobas® 8800 + cobas® SARS-CoV-2 (Roche Molecular Systems, Branchburg, NJ, USA) (n=8). The three most frequently used PCR reagents were the RealStar® SARS-CoV-2 RT-PCR Kit RUO (altona Diagnostics, Hamburg, Germany) (n=8), the ViroReal® Kit SARS-CoV-2 & SARS (Ingenetix, Vienna, Austria) (n=7), and the LightMix® Modular SARS CoV (Covid19) E-gene (TIB Molbiol, Berlin, Germany) (n=7).

The variability of Ct values for individual gene targets obtained by different test systems (pseudonymized as 1–92) is shown in Figure 3. There were 170/466 (36.5%) results that deviated from the respective mean value by <±2 cycles and 36 (7.7%) that deviated by >±4 cycles. The latter comprised six single outliers (deviation >±4 cycles), each obtained by one individual protocol, and the remaining 30 were obtained by nine protocols that generated between two and six outliers each. Of the 10 protocols that did not detect S5 (false negatives), four reported Ct values which deviated by >+2.7 for the other two positive samples, compared to their respective means: test system no. 32 yielded +5.4 for S1 and +2.8 cycles for S3 (E-gene); no. 39 yielded for S1 +5.0 for in S3 +4.1 cycles (E-gene); no. 55 yielded for S1 +5.8 and for S3 +7.8 cycles (N-gene); no. 83 yielded for S1 +5.0 and for S3 +4.7 cycles (S-gene). Three protocols that also reported S5 as false negative (no. 41, 76, and 80) reported Ct values for the other two positive samples that did not deviate by more than ±2.7 from the respective means. For three of the test systems that did not detect S5 as positive, no Ct values were reported.

In general, the combined results produced rather uniform distributions about their respective means per gene target, i.e. a consistent skew greater than or less than the mean was not evident (Figure 3). Standard deviations of results for the respective gene targets are shown in Table 1. However, three systems (no. 5 for the E-gene, no. 36 for the ORF1ab-gene, and no. 88 for the E-gene) presented results deviating to both sides of (i.e., either greater than or less than) the respective mean for the same gene target, of which at least one result deviated >2 cycles. The deviation in the other direction was only very small (≤1.0). Additionally, three test systems (no. 50 for the E-, N- and RdRp-gene, no. 82 for the E-, N-, and RdRp-gene, and no. 85 for the S- and RdRp-gene) yielded results for different gene targets that were entirely greater than or less than the respective mean.

The GeneXpert® and the cobas® 6800/8800 systems use primers for two different gene targets. These are the E- and N-gene in the GeneXpert® system and the E- and ORF1ab-gene in the cobas® 6800/8800 systems. Ct values obtained by the GeneXpert® for the E-gene were lower than for the N-gene (medians for S1: 27.8 vs. 30.2, for S2: 23.9 vs. 26.3, and for S5: 36.1 vs. 38.8). The cobas® system yielded slightly higher results for the E-gene than for the ORF1ab-gene (medians for S1: 29.3 vs. 28.9, for S3: 25.0 vs. 24.7, and for S5: 36.6 vs. 36.0).

Discussion

Qualitative results of the present EQA challenge were improved as compared to the previous round [7]. More than 90% of the test systems detected all positive samples correctly, as compared to 60% in the previous round. However, the false negative rate strongly depends on the limit of detection for each assay (i.e., viral loads in weakly positive samples), which were higher in this (Ct 35.7 for the E-gene) than in the previous challenge (Ct 38.5 for the E-gene).

Evaluation of quantitative results showed slight intra-protocol variability, a moderate dependency of the Ct values on the gene target and a clear inter-protocol variability of Ct values in some protocols, but not in others.

High reproducibility (low variance) was observed for the 31 participants using the GeneXpert® (1–23, Figure 3) and the cobas® 6800/8800 (24–31, Figure 3) systems. Absolute deviations were higher for other test systems (middle and right columns, Figure 3) and, for certain test protocols, lie outside a range of ±2 of the respective mean value. Notably, these deviations were not random, but systematic errors to one or the other side of the respective mean values (i.e., consistently greater than or less than the mean, depending on the protocol). This questions the comparability of Ct values obtained for different genes and test protocols, but also offers prospect for standardization. Recommendations for standardization of nucleic acid tests have been published and test material has been made available [8], [9]. The responsibility resides with the manufacturers of IVD/CE-marked test systems or individual laboratories, when using in-house methods.

Two test systems produced two simultaneous Ct values from a given reaction: the E- and N-genes in the GeneXpert® system and the E- and ORF1ab-genes in the cobas® 6800/8800 system. In the GeneXpert® system the difference in Ct values between the two gene targets was approximately three cycles for each sample, while the difference between Ct values between the two targets in the cobas® system was much less. It is not clear whether this represents differences in relative available template of each gene target in the same sample, or whether it is due to differences in the reaction conditions. However, perhaps multiplexed assays, such as these two, could be used to standardize results between various gene targets, as reaction conditions are more controlled.

In conclusion, caution should be exerted when patient management procedures are based on SARS-CoV-2 (RT) PCR Ct values. In the absence of standardization, these technical values are appropriate metric values for correct classification of samples. However, they do not meet the standard of a diagnostic test, when estimating viral loads. For the use of Ct values in medical guidelines or government regulations it is advisable to validate the method according to well-defined external reference material, e.g. from an external quality assessment.


Corresponding author: Stephan W. Aberle, Center for Virology, Medical University of Vienna, Kinderspitalgasse 15, 1090 Vienna, Austria, E-mail:
Christoph Buchta and Irene Görzer contributed equally to this work.

Acknowledgments

We gratefully acknowledge Tanja Amon and Andreas Rohorzka for excellent technical assistance, and Magdalena Blecha for graphic design (Figures 1 and 3).

  1. Research funding: None declared.

  2. Author contributions: Ch. Buchta: project administration, formal analysis, writing – original draft. I. Görzer: conceptualization, writing – original draft, visualization. P. Chiba: project administration, formal analysis, writing – review and editing. J. V. Camp: visualization, writing – review and editing. E. Puchhammer-Stöckl: supervision, writing – review and editing. M. Mayerhofer: writing – original draft. M. M. Müller: Resources, conceptualization, writing – review and editing. S. W. Aberle: Resources, conceptualization, supervision, writing – review and editing. All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Ethical approval: Not applicable.

References

1. Service, RF. A call for diagnostic tests to report viral load. Science 2020;370:22. https://doi.org/10.1126/science.370.6512.22. Search in Google Scholar

2. Magleby, R, Westblade, LF, Trzebucki, A, Simon, MS, Rajan, M, Park, J, et al.. Impact of SARS-CoV-2 viral load on risk of intubation and mortality among hospitalized patients with coronavirus disease 2019. Clin Infect Dis 2020:ciaa851. https://doi.org/10.1093/cid/ciaa851 [Epub ahead of print]. PMID: 32603425; PMCID: PMC7337625. Search in Google Scholar

3. Singanayagam, A, Patel, M, Charlett, A, Lopez Bernal, J, Saliba, V, Ellis, J, et al.. Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020. Euro Surveill 2020;25:pii=2001483. https://doi.org/10.2807/1560-7917.ES.2020.25.32.2001483. Search in Google Scholar

4. https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Vorl_Testung_nCoV.html#x [Accessed 10 Oct 2020]. Search in Google Scholar

5. https://www.sozialministerium.at/dam/jcr:531f7e21-0f53-4180-b214-848e19668b52/Empfehlung%20zur%20Entlassung%20von%20COVID-19-Fällen%20aus%20der%20Absonderung_final_230720.pdf [Accessed 10 Oct 2020]. Search in Google Scholar

6. Matheeussen, V, Corman, VM, Donoso Mantke, O, McCulloch, E, Lammens, C, Goossens, H, et al.. On behalf of the RECOVER project and collaborating networks: international external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020. Euro Surveill 2020;25:pii=2001223. https://doi.org/10.2807/1560-7917.ES.2020.25.27.2001223. Search in Google Scholar

7. Görzer, I, Buchta, C, Chiba, P, Benka, B, Camp, JV, Holzmann, H, et al.. Aberle SW First results of a national external quality assessment scheme for the detection of SARS-CoV-2 genome sequences. J Clin Virol 2020;129:104537. https://doi.org/10.1016/j.jcv.2020.104537. [Epub 6 Jul 2020]. PMID: 32659712; PMCID: PMC7336937. Search in Google Scholar

8. Baylis, SA, Wallace, P, McCulloch, E, Niesters, HGM, Nübling, CM. Standardization of nucleic acid tests: the approach of the World Health Organization. J Clin Microbiol 2019;57:e01056–18. https://doi.org/10.1128/JCM.01056-18. PMID: 30257900; PMCID: PMC6322456. Search in Google Scholar

9. https://www.nist.gov/programs-projects/sars-cov-2-research-grade-test-material. Search in Google Scholar

Received: 2020-10-27
Accepted: 2020-12-06
Published Online: 2020-12-16
Published in Print: 2021-04-27

© 2020 Christoph Buchta et al., published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.