Novel point-of-care antigen assays present a promising opportunity for rapid screening of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections. The purpose of this study was the clinical assessment of the new Roche SARS-CoV-2 Rapid Antigen Test.
The clinical performance of Roche SARS-CoV-2 Rapid Antigen Test was evaluated vs. a reverse transcription polymerase chain reaction (RT-PCR) laboratory-based assay (Seegene AllplexTM2019-nCoV) in nasopharyngeal swabs collected from a series of consecutive patients referred for SARS-CoV-2 diagnostics to the Pederzoli Hospital (Peschiera del Garda, Verona, Italy) over a 2-week period.
The final study population consisted of 321 consecutive patients (mean age, 46 years and IQR, 32–56 years; 181 women, 56.4%), with 149/321 (46.4%) positive for SARS-CoV-2 RNA via the Seegene AllplexTM2019-nCoV Assay, and 109/321 (34.0%) positive with Roche SARS-CoV-2 Rapid Antigen Test, respectively. The overall accuracy of Roche SARS-CoV-2 Rapid Antigen Test compared to molecular testing was 86.9%, with 72.5% sensitivity and 99.4% specificity. Progressive decline in performance was observed as cycle threshold (Ct) values of different SARS-CoV-2 gene targets increased. The sensitivity was found to range between 97–100% in clinical samples with Ct values <25, between 50–81% in those with Ct values between 25 and <30, but low as 12–18% in samples with Ct values between 30 and <37.
The clinical performance of Roche SARS-CoV-2 Rapid Antigen Test is excellent in nasopharyngeal swabs with Ct values <25, which makes it a reliable screening test in patients with high viral load. However, mass community screening would require the use of more sensitive techniques.
The ongoing coronavirus disease 2019 (COVID-19) pandemic outbreak is causing dramatic clinical, societal, and economic consequences all around the world. Despite the many public health strategies that have been implemented to face this challenge, including lockdowns, social distancing, widespread use of face masks and hand hygiene , data suggests that positive case identification, isolation and contact tracing are the most important factors for preventing further spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) . While this strategy has been endorsed by many international and national organizations, evidence suggests that the current testing policy for diagnosing SARS-CoV-2 infections, mostly based on nucleic acid amplification tests (NAATs), appears to be failing , , , . It is not readily apparent that the ideal SARS-CoV-2 testing volume cannot be reached in most countries worldwide, including the most industrialized nations such as the United Kingdom . This generates a kaleidoscope of unfavourable consequences, including delayed diagnosis and late clinical management, which would then be associated with unfavourable disease progression , insufficient contact tracing and delayed isolation of SARS-CoV-2 positive cases, which in turn further contributes to propagate onward viral transmission . Moreover, insufficient testing of individuals with high-risk exposures, most importantly, front-line healthcare workers, can lead to prolonged self-quarantine, jeopardizing health care delivery in conventional, urgent, and intensive care settings .
Due to the practical limitations of NAATs, encompassing a long turnaround time, low throughput and the need for specific instrumentation and skilled employees to conduct the test, rapid antigen immunoassay are emerging as a viable prospective for mass testing, as recently endorsed by the World Health Organization (WHO) . Despite a reportedly lower diagnostic sensitivity compared to molecular testing, which would hence limit infection identification to subsets of patients with higher SARS-CoV-2 viral load, their widespread usage under well-defined circumstances may enable ample population screenings . Nonetheless, analytical and clinical validation of these rapid immunoassays is a necessary preamble before their implementation within locally defined diagnostic algorithms. To this end, the purpose of this study was the clinical assessment of the new Roche SARS-CoV-2 Rapid Antigen Test.
The Roche SARS-CoV-2 Rapid Antigen Test is an immunochromatographic assay for rapid qualitative detection of SARS-CoV-2 infection in nose and/or throat swabs. Briefly, the clinical specimen is collected and then deposited with twisting motion in a pre-filled extraction buffer container. After removing the swab, three drops of sample material are applied to the reagent tray (Figure 1). The presence of viral antigens in sufficient concentration enables their binding to specific mouse monoclonal anti-SARS-CoV-2 antibodies, then reflected by the appearance of a visual indication (i.e., a coloured line) in the lower section of the result window of the test strip, along with another “control” coloured line, which appears in the top section of the result window, when the device has been properly employed (Figure 1). The entire procedure can be completed within 15–30 min and does not require a dedicated environment (e.g., clinical laboratories), nor highly trained personnel to perform the test. According to manufacturer’s information, the limit of detection (LoD) per viral strain and the lowest concentration with uniform positivity per parameter are both 3.12×102.2 TCID50/mL, whilst the diagnostic sensitivity and specificity in nasopharyngeal swabs collected from both symptomatic and asymptomatic patients are reported to be 96.5 and 99.7%, respectively.
The study population consisted of all consecutive patients referred for SARS-CoV-2 diagnostic testing to the Pederzoli Hospital (Peschiera del Garda, Verona, Italy), over a 2-week period (16–30 November, 2020). Upper respiratory specimens were collected in agreement with WHO recommendations . A single swab (Virus swab UTM™, Copan, Brescia, Italy) was collected from each patient and concomitantly used for both Roche SARS-CoV-2 Rapid Antigen testing and molecular testing, which was performed using a commercial revere-transcription polymerase chain reaction (RT-PCR) assay (Seegene AllplexTM2019-nCoV Assay, Seegene, Seoul, South Korea). This method uses a volume of 350 µL and enables SARS-CoV-2 RNA identification by targeting three viral genes (N, E and RdRP), thus fulfilling internationally validated testing protocols . Real-time PCR was interpreted using Seegene’s Viewer software. The viral load was finally expressed as cycle threshold (Ct), and test results with Ct values <37 for all three SARS-CoV-2 gene targets were considered “reactive” for SARS-CoV-2 RNA, thus specifically aimed at increasing the specificity of RNA viral detection, in accordance with current recommendations , and avoiding potential false negative test results due to emerging variants such as the VUI-202012/01 recently identified in the UK.
Quantitative and qualitative test results were presented as median with interquartile range (IQR) or percentage, respectively. Correlations between the Ct values of each SARS-CoV-2 gene were compared using Spearman’s rank correlation coefficient. The diagnostic efficiency of Roche SARS-CoV-2 Rapid Antigen Test for diagnosing SARS-CoV-2 infection was assessed by calculating the diagnostic accuracy, sensitivity and specificity, both cumulative and stratified according to NAAT Ct values. The statistical analysis was carried out using Analyse-it (Analyse-it Software Ltd, Leeds, UK) and MedCalc (MedCalc Software Ltd, Ostend, Belgium). The investigation was based on pre-existing specimens, already collected for routine SARS-CoV-2 diagnostic testing in the local facility, and thereby no patient’s informed consent, nor Ethical Committee approval were necessary. This study was conducted in accordance with the Declaration of Helsinki, under the terms of relevant local legislation.
The final study population consisted of 321 consecutive patients (mean age, 46 years and IQR, 32–56 years; 181 women, 56.4%), who underwent SARS-CoV-2 diagnostic testing at the Pederzoli Hospital of Peschiera del Garda, Verona. Overall, 149/321 (46.4%) samples tested positive for SARS-CoV-2 RNA with the Seegene AllplexTM2019-nCoV Assay and 109/321 (34.0%) samples tested positive with the Roche SARS-CoV-2 Rapid Antigen Test, respectively. The median Ct values of positive samples were 25 (IQR, 21–32) for the E gene, 26 (IQR, 21–31) for the RdRP gene and 22 (IQR, 18–29) for the N gene, respectively. A high Spearman’s correlation was found between the Ct values of three gene targets, as follows: E vs. RdRP gene, r=0.98 (95% CI, 0.97–0.98; p<0.001); E vs. N gene, r=0.98 (95% CI, 0.97–0.98; p<0.001); RdRP vs. N gene, r=0.97 (95% CI, 0.96–0.97; p<0.001).
The distribution of Ct values is summarized in Table 1 and Figure 2. The overall accuracy of the Roche SARS-CoV-2 Rapid Antigen Test was 86.9%, with 72.5% sensitivity and 99.4% specificity. The best performance was found by comparison antigen test results with RdRP gene (86.0% accuracy, 71.0% sensitivity), followed by comparison with E gene (84.4% accuracy, 68.8% sensitivity) and N gene (80.7% accuracy, 63.9% sensitivity). A progressive decline in performance could be observed as the Ct values for the different SARS-CoV-2 gene targets increased. The sensitivity was found to range between 97–100% in clinical samples with Ct values <25, between 50–81% in those with Ct values between 25 and <30, but was as low as 12–18% in samples with Ct values between 30 and <37 (Table 1).
|Ct value||n||Accuracy||Sensitivity (95% CI)||Specificity (95% CI)|
|Overall||321||86.9% (82.7–90.4%)||72.5% (64.6–79.5%)||99.4% (96.8–100%)|
|E gene||321||84.4% (78.6–87.1%)||68.8% (60.9–75.9%)||99.4% (96.6–100%)|
|RdRP gene||321||86.0% (81.7–89.6%)||71.1% (63.2–78.1%)||99.4% (96.8–100%)|
|N gene||321||80.7% (75.9–84.9%)||63.9% (56.2–71.1%)||99.3% (96.4–100%)|
|E gene||29||N/C||97.8% (88.2–99.9%)||N/C|
|RdRP gene||27||N/C||100% (87.2–100%)||N/C|
|N gene||55||N/C||98.2% (90.3–100%)||N/C|
|E gene||45||N/C||97.8% (88.2–99.9%)||N/C|
|RdRP gene||45||N/C||97.8% (88.5–99.9%)||N/C|
|N gene||36||N/C||97.2% (85.5–99.9%)||N/C|
|E gene||34||N/C||79.4% (62.1–91.3%)||N/C|
|RdRP gene||36||N/C||80.6% (64.0–91.8%)||N/C|
|N gene||26||N/C||50.3% (29.9–70.1%)||N/C|
|E gene||49||N/C||16.3% (7.3–29.7%)||N/C|
|RdRP gene||44||N/C||18.2% (8.2–32.7%)||N/C|
|N gene||52||N/C||11.5% (4.4–23.4%)||N/C|
Ct, cycle thresholds; AUC, area under the curve; N/C, not calculable.
Although rapid, point-of-care antigen-based tests present a promising opportunity for mass (population) screening, and thereby for rapidly identifying, isolating and/or treating patients with SARS-CoV-2 infection, the reported analytical and diagnostic performance for many of the commercially available assays varies widely, as recently highlighted by the Cochrane COVID-19 Diagnostic Test Accuracy Group . Although the cumulative specificity was found to be excellent, between 90–100%, the diagnostic sensitivity ranged between as low as 0%, to over 90%. As predictable, the diagnostic sensitivity was highly dependent on viral load, whereby ranged between 21–100% in samples with Ct≤25, but then decreased to 8–72% in those with higher Ct values. It is thus vital that thoughtful local assessment of analytical and clinical performance be performed before implementing any rapid antigen SARS-CoV-2 assay in routine COVID-19 diagnostics , [, 14]. In fact, only by precisely acknowledging the local diagnostic characteristics of the test, assessed on a target population within an environment with a specific population and prevalence of disease, would enable a suitable and reliable implementation of the test within specific protocols for SASR-CoV-2 diagnostics.
To the best of our knowledge, a previous study has assessed the clinical performance of the Roche SARS-CoV-2 Rapid Antigen Test. Briefly, Krüttgen et al. compared the qualitative test results of Roche SARS-CoV-2 Rapid Antigen Test with those obtained using the Real Star Sars-CoV-2 RT PCR Kit (Altona, Germany) on nasopharyngeal swabs collected from 150 patients admitted to the RWTH Aachen University (Germany), 75/150 (50%) of whom had a positive NAAT results . The cumulative sensitivity and specificity of Roche SARS-CoV-2 Rapid Antigen Test were found to be 70.7% and 96.0%, respectively though, as observed in our results, the assay sensitivity was found to be highly dependent on the viral load present in the specimen. More specifically, Krüttgen et al. reported that the sensitivity was 95% or higher in samples with Ct <30, but then decreased to 45% and 22% in those with Ct between 30–35 and >35, respectively , thus comparing well with our findings on an almost double-sized population. This same rapid antigen test was also evaluated in a subsequent study, which analyzed its performance with either professional-collected or self-collected nasopharyngeal swabs . Interestingly, the cumulative diagnostic sensitivity was confirmed to be between 70–80%, gradually decreasing in parallel with the reduction of viral load, but not differing significantly between the two collection modalities.
Taken together, the results of our study thus attest that the clinical performance of Roche SARS-CoV-2 Rapid Antigen Test are excellent in nasopharyngeal swabs with Ct values <25, which makes it a reliable screening test in patients with high viral load. Notably, the current recommendations from the World Health Organization (WHO) concerning the minimum antigen-detection performance are that these kits should have diagnostic a sensitivity and specificity of ≥80% and ≥97%, respectively . Although the Roche SARS-CoV-2 Rapid Antigen Test does not completely match the WHO desired clinical performance, its cumulative sensitivity of 72%, which increases to 97–100% in specimens with Ct values <25, makes it one of the best commercially available assays for rapid SARS-CoV-2 antigen detection, equalling or even outperforming several other rapid SARS-Cov-2 antigen kits that are currently available on the market , , , and displaying diagnostic performance that are globally comparable to that of a rapid SARS-CoV-2 antigen chemiluminescence immunoassay .
Although the relationship between SARS-CoV-2 viral load and infectivity remains a matter of open debate, convincing evidence attests that SARS-CoV-2 positive subjects with high Ct values, especially when asymptomatic or if symptoms have disappeared by over 10–14 days, have relatively lower transmission potential . Arons et al. demonstrated that the likelihood of obtaining positive SARS-CoV-2 cultures strictly depends on viral load, irrespective of the presence of clinical symptoms, whereby the positive rate was found to decline below 15% with Ct values ≥28 in pre-symptomatic patients who later developed a suggestive clinical picture . Similar evidence has been reported in several other studies, such as those published by Basile et al. , La Scola et al.  and Jaafar et al. , who also concluded that the rate of positive SARS-CoV-2 cultures is inversely associated with viral load, achieving a virtually zero likelihood of a positive culture in clinical specimens with Ct values ≥33–34. Overall, a recent meta-analysis of published studies revealed a high heterogeneity in such Ct thresholds, with no growth ranging between Ct of 24 and 32 , thus paving the way to additional investigations aimed to more precisely define the correlation between NAAT results and viral culture of SARS-CoV-2.
In conclusion, the results of our clinical evaluation of Roche SARS-CoV-2 Rapid Antigen Test demonstrates that this method seems a suitable approach for rapid nasopharyngeal swab screening of patients with high SARS-CoV-2 viral load, especially those with symptoms or in the pre-symptomatic phase. However, mass population screening would still require the use of more sensitive techniques. A strategy based on more frequent testing shall also be envisaged when using rapid antigen testing rather than molecular assays, as recently endorsed by Mina et al. .
The authors acknowledge the staff of the Service of Laboratory Medicine of the Pederzoli Hospital (Peschiera del Garda, Italy) for the skill technical assistance.
Research funding: None declared.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: Authors state no conflict of interest.
Ethical approval: The investigation was based on pre-existing specimens, already collected for routine SARS-CoV-2 diagnostic testing in the local facility, and thereby no patient’s informed consent, nor Ethical Committee approval were necessary. This study was conducted in accordance with the Declaration of Helsinki, under the terms of relevant local legislation.
1. Lippi, G, Henry, BM, Bovo, C, Sanchis-Gomar, F. Health risks and potential remedies during prolonged lockdowns for coronavirus disease 2019 (COVID-19). Diagnosis (Berl) 2020;7:85–90. https://doi.org/10.1515/dx-2020-0041. Search in Google Scholar
2. Lippi, G, Plebani, M. The critical role of laboratory medicine during coronavirus disease 2019 (COVID-19) and other viral outbreaks. Clin Chem Lab Med 2020;58:1063–9. https://doi.org/10.1515/cclm-2020-0240. Search in Google Scholar
3. Iacobucci, G. Covid-19: testing service wasn’t prepared for increased demand, chief admits. BMJ 2020;370:m3676 and Todd B. The U.S. COVID-19 Testing Failure. Am J Nurs 2020;120;19–20. Search in Google Scholar
4. Alaa, A, Qian, Z, Rashbass, J, Benger, J, van der Schaar, M. Retrospective cohort study of admission timing and mortality following COVID-19 infection in England. BMJ Open 2020;10:e042712. https://doi.org/10.1136/bmjopen-2020-042712. Search in Google Scholar
5. Kretzschmar, ME, Rozhnova, G, Bootsma, MCJ, van Boven, M, van de Wijgert, JHHM, Bonten, MJM. Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study. Lancet Public Health 2020;5:e452–9. https://doi.org/10.1016/s2468-2667(20)30157-2. Search in Google Scholar
7. World Health Organization. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. Interim guidance 2020. Available at: https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays [Accessed 5 December 2020]. Search in Google Scholar
8. Mattiuzzi, C, Henry, BM, Lippi, G. Making sense of rapid antigen testing in SARS-CoV-2 diagnostics. Diagnosis (Berl) 2020 Nov 26. https://doi.org/10.1515/dx-2020-0131 [Epub ahead of print]. Search in Google Scholar
9. World Health Organization. Laboratory testing for coronavirus disease 2019 (COVID-19) in suspected human cases. Interim guidance 2020. Search in Google Scholar
10. Bohn, MK, Mancini, N, Loh, TP, Wang, CB, Grimmler, M, Gramegna, M, et al.. IFCC Interim guidelines on molecular testing of SARS-CoV-2 infection. Clin Chem Lab Med 2020;58:1993–2000. https://doi.org/10.1515/cclm-2020-1412. Search in Google Scholar
11. Yang, Z, Wu, J, Ye, F, Zhu, B, Guan, W, Huang, J, et al.. Expert consensus-based laboratory testing of SARS-CoV-2. J Thorac Dis 2020;12:4378–90. https://doi.org/10.21037/jtd-20-1928. Search in Google Scholar
12. Dinnes, J, Deeks, JJ, Adriano, A, Berhane, S, Davenport, C, Dittrich, S, et al.. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2020;8:CD013705. Search in Google Scholar
13. Lippi, G, Horvath, AR, Adeli, K. Editorial and executive summary: IFCC Interim guidelines on clinical laboratory testing during the COVID-19 pandemic. Clin Chem Lab Med 2020;58:1965–9. https://doi.org/10.1515/cclm-2020-0633. Search in Google Scholar
14. Lippi, G, Mattiuzzi, C, Bovo, C, Plebani, M. Current laboratory diagnostics of coronavirus disease 2019 (COVID-19). Acta Biomed 2020;91:137–45. https://doi.org/10.23750/abm.v91i2.9548. Search in Google Scholar
15. Krüttgen, A, Cornelissen, CG, Dreher, M, Hornef, MW, Imöhl, M, Kleines, M. Comparison of the SARS-CoV-2 rapid antigen test to the real star sars-CoV-2 RT PCR kit. J Virol Methods 2020:114024. https://doi.org/10.1016/j.jviromet.2020.114024. Search in Google Scholar
16. Lindner, AK, Nikolai, O, Kausch, F, Wintel, M, Hommes, F, Gertler, M, et al.. Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected anterior nasal swab versus professional-collected nasopharyngeal swab. Eur Respir J 2020:2003961. https://doi.org/10.1183/13993003.03961-2020 [Epub ahead of print]. Search in Google Scholar
17. World Health Organization. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. interim Guidance 2020. Available from: https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays [Accessed 5 Dec 2020]. Search in Google Scholar
18. Linares, M, Pérez-Tanoira, R, Carrero, A, Romanyk, J, Pérez-García, F, Gómez-Herruz, P, et al.. Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms. J Clin Virol 2020;133:104659. https://doi.org/10.1016/j.jcv.2020.104659. Search in Google Scholar
19. Albert, E, Torres, I, Bueno, F, Huntley, D, Molla, E, Fernández-Fuentes, MÁ, et al.. Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centers. Clin Microbiol Infect 2020;S1198–743X:30697–2. https://doi.org/10.1016/j.cmi.2020.11.004 [Epub ahead of print]. Search in Google Scholar
20. Liotti, FM, Menchinelli, G, Lalle, E, Palucci, I, Marchetti, S, Colavita, F, et al.. Performance of a novel diagnostic assay for rapid SARS-CoV-2 antigen detection in nasopharynx samples. Clin Microbiol Infect 2020;S1198–743X:30583–8. https://doi.org/10.1016/j.cmi.2020.09.030 [Epub ahead of print]. Search in Google Scholar
21. Hirotsu, Y, Maejima, M, Shibusawa, M, Nagakubo, Y, Hosaka, K, Amemiya, K, et al.. Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients. Int J Infect Dis 2020;99:397–402. https://doi.org/10.1016/j.ijid.2020.08.029. Search in Google Scholar
22. Walsh, KA, Jordan, K, Clyne, B, Rohde, D, Drummond, L, Byrne, P, et al.. SARS-CoV-2 detection, viral load and infectivity over the course of an infection. J Infect 2020;81:357–71. https://doi.org/10.1016/j.jinf.2020.06.067. Search in Google Scholar
23. Arons, MM, Hatfield, KM, Reddy, SC, Kimball, A, James, A, Jacobs, JR, et al.. Presymptomatic SARS-CoV-2 infections and transmission in a skilled nursing facility. N Engl J Med 2020;382:2081–90. https://doi.org/10.1056/NEJMoa2008457. Search in Google Scholar
24. Basile, K, McPhie, K, Carter, I, Alderson, S, Rahman, H, Donovan, L, et al.. Cell-based culture of SARS-CoV-2 informs infectivity and safe de-isolation assessments during COVID-19. Clin Infect Dis 2020:ciaa1579. https://doi.org/10.1093/cid/ciaa1579 [Epub ahead of print]. Search in Google Scholar
25. La Scola, B, Le Bideau, M, Andreani, J, Hoang, VT, Grimaldier, C, Colson, P, et al.. Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards. Eur J Clin Microbiol Infect Dis 2020;39:1059–61. https://doi.org/10.1007/s10096-020-03913-9. Search in Google Scholar
26. Jaafar, R, Aherfi, S, Wurtz, N, Grimaldier, C, Hoang, VT, Colson, P, et al.. Correlation between 3790 qPCR positives samples and positive cell cultures including 1941 SARS-CoV-2 isolates. Clin Infect Dis 2020:ciaa1491. https://doi.org/10.1093/cid/ciaa1491 [Epub ahead of print]. Search in Google Scholar
27. Jefferson, T, Spencer, EA, Brassey, J, Heneghan, C. Viral cultures for COVID-19 infectious potential assessment - a systematic review. Clin Infect Dis 2020:ciaa1764. https://doi.org/10.1093/cid/ciaa1764 [Epub ahead of print]. Search in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston