COVID-19 has brought about tests from many manufacturers. While molecular and rapid antigen tests are targeted for early diagnosis, immunoassays have a larger role in epidemiological studies, understanding longitudinal immunity, and in vaccine development and response.
The performance of the LIAISON® SARS-CoV-2 TrimericS IgG assay was evaluated against the Beckman ACCESS SARS-CoV-2 IgG assay in New Mexico, and against the Siemens ADVIA Centaur COV2G assay in New York. Discordant samples were parsed using a microneutralization assay.
A SARS-CoV-2 antibody positivity rate of 23.8% was observed in the samples tested in New York (September 2020), while in the same month the positivity rate was 1.5% in New Mexico. Positive and negative agreement were 67.6% (95% CI 49.5–82.6%) and 99.8% (95% CI 99.5–99.9%), respectively, with the Beckman test, and 98.0% (95% CI 95.7–99.3%) and 94.8% (95% CI 93.4–96.0%), respectively, with the Siemens test. Receiver operating characteristic analysis for the detection of SARS-CoV-2 antibodies discloses an AUC, area under the curve, of 0.996 (95% CI 0.992–0.999) for the LIAISON® SARS-CoV-2 TrimericS IgG assay. The criterion associated to the Youden Index was determined to be >12.9 kAU/L with a sensitivity of 99.44% and a specificity of 99.82%.
The LIAISON® SARS-CoV-2 TrimericS IgG assay is highly sensitive and specific. The balance of these parameters, without emphasis on high specificity alone, is particularly important when applied to high prevalence populations, where a highly sensitive assay will result in reporting a lower number of false negative subjects.
The SARS-CoV-2 virus is a betacoronavirus belonging to the family of Coronaviruses, named for the crown-like spikes on their surface, and causes coronavirus disease 2019 (COVID-19). An outbreak of SARS-CoV-2 began in December 2019 in Wuhan City, Hubei Province, China and has since spread globally . Consequent to being declared a public health emergency of international concern by the World Health Organization (WHO) on January 30 of 2020, worldwide prevalence has now passed 48 million subjects with over one million deaths by early November 2020 . Patients with COVID-19 have symptoms that range from asymptomatic to severe, and include respiratory illness, fever, cough and shortness of breath along with a possible loss of smell and taste . Severely infected individuals often succumb to acute respiratory distress. Infection with SARS-CoV-2 results in the generation of neutralizing antibodies, with the magnitude of the neutralizing antibody response in asymptomatic individuals shown to be smaller, and more rapidly decreasing than that in symptomatic individuals .
COVID-19 has spurred the development of a plethora of tests from various manufacturers. While molecular tests and rapid antigen tests are targeted for early diagnosis, serology tests play a different and perhaps even larger role by supporting epidemiological studies, furthering the understanding of longitudinal immunity, and in assessing the efficacy of vaccine development . The efficient use of serology tests in understanding immunity, both in response to disease or vaccination, requires evaluation of any given assay’s diagnostic performance and discriminative ability , .
The spike protein has been the preferred antigen for use in vaccines, because most neutralizing antibodies are directed against this surface exposed protein on the virus that is essential for entry into human cells . Coronavirus spikes form large heavily glycosylated homo-trimeric complexes, yet due to their instability in vitro, monomeric presentations of the spike protein, including the receptor binding domain (RBD) only, have been most prevalently used in immunoassays. While the diagnostic performance of assays using RBD as an antigen have shown better results compared to spike and nucleoprotein antigens , the trimeric form was shown to be associated with greater sensitivity for the detection of SARS-CoV-2 IgG antibody in human samples . Here, the performance of the LIAISON® SARS-CoV-2 TrimericS IgG assay (Emergency Use Authorization, EUA) was evaluated against two assays that utilize the spike RBD as capture antigen: the Siemens ADVIA Centaur COV2G assay in sample remainders from a routine laboratory workflow in New York experiencing high disease prevalence; the Beckman ACCESS SARS-CoV-2 IgG assay from a New Mexico laboratory confronted with a lower disease prevalence.
Materials and methods
Clinical sample characteristics
The study was observational following a retrospective design performed on fresh or frozen de-identified residual serum or plasma samples collected from subjects for clinical care in September 2020 at two U.S. laboratories with different disease prevalence at the time of the study. Specimens were collected from the general population including persons seeking care for general wellness visits. There was no a priori selection for patients seeking care for COVID-19 related illnesses, and the vast majority of specimens were collected from individuals not seeking medical care related to COVID-19. The NY site, did however, include samples from asymptomatic people attending community drives for SARS-CoV-2 antibody detection. Samples were collected in line with institutional IRB guidelines.
The LIAISON® SARS-CoV-2 TrimericS IgG assay (Emergency Use Authorization, EUA) is an indirect chemiluminescence immunoassay (CLIA) technology for the detection of IgG antibodies to SARS-CoV-2 in human serum and plasma samples. The principal components of the test are paramagnetic particles (solid phase) coated with recombinant trimeric SARS-CoV-2 spike protein  and a conjugate reagent containing an anti-human IgG mouse monoclonal antibody linked to an isoluminol derivative. The assay is intended to assess the presence of antibodies to SARS-CoV-2, including neutralizing antibodies, in human serum or plasma. The positive percent agreement of the LIAISON® SARS-CoV-2 TrimericS IgG test compared to PCR for samples ≥15 days from diagnosis was determined to be 98.7% (95% CI 94.5%–99.6)% according to manufacturer’s instruction for use, while the negative percent agreement was 99.5% (95% CI 99.0–99.7%) in 1899 blood donor samples collected prior to COVID-19 outbreak.
The ADVIA Centaur COV2G assay is a fully automated 2-step antigen sandwich immunoassay using acridinium ester chemiluminescent technology, in which recombinant SARS-CoV-2 S1 RBD antigen-coated microparticles are used to capture SARS-CoV-2 specific antibodies in the specimen. Overall positive percent agreement for samples ≥14 days from diagnosis by PCR as published in the manufacturer’s instructions for use is 100% (95% CI 91.59–100%, and negative percent agreement was determined to be 99.89% (95% CI 99.61–99.99%) in 1831 samples collected prior to the COVID-19 outbreak from apparently healthy individuals .
The ACCESS SARS-CoV-2 IgG assay is a two-step enzyme immunoassay with chemiluminescent substrate. As capture, paramagnetic particles are coated with recombinant SARS-CoV-2 protein specific for the RBD of the S1 protein. Detection occurs via a monoclonal anti-human IgG alkaline phosphatase conjugate that binds to the IgG antibodies from the samples captured on the particles. The positive percent agreement of the Access SARS-CoV-2 IgG assay compared to PCR for samples ≥15 days from diagnosis was determined to be 96.8% (95% CI 91.1−98.8%) according to manufacturer’s instruction for use, while the negative percent agreement was 99.6% (95% CI: 99.2%–99.9) in 1,400 samples collected prior to December 2019 .
Analytical assay performance
A 5 day precision study according to CLSI EP5-A3 guidelines was performed using a panel of six serum samples covering the assay range. The panel samples were tested with LIAISON® SARS-CoV-2 TrimericS IgG assay in three replicates per run, two runs per day for five operating days at two sites with different lots (n=60). Total imprecision is reported at an average of 4.85% CV (3.6–5.8% CV range) at values ranging from 5 to 591 kAU/L, and including samples with target values near the negative/positive cut-off area of the assay.
To assess linearity, three specimens were chosen with concentrations of 1,060, 1,288, and 1,027 kAU/L. Sample dilutions of 100, 70, 50, 30, 10, 10, 0.1, and 0% (neat diluent) were assayed in quadruplicates. Diluted sample concentrations were measured and compared against calculated values. The average slope of the Passing and Bablock fit was 1.04 (range 0.99–1.08) with an average intercept of 0.037 (range −0.07 to 0.21). Average recovery was 103.9% (range 89.9–121%).
According to the respective instructions for use, the three tests report a lack of cross-reactivity with specimens from individuals with medical conditions unrelated to SARS-CoV-2 infection but with similar symptoms, as well as lack of interference from endogenous and exogenous substances at typical analyte concentrations. Total imprecision for the Siemens assay is reported at an average of 10.8% CV (8.7–13.9% CV range) at indices ranging from 1.00 to 19.24, and no imprecision data were available for the Beckman assay.
Semi-quantitative live SARS-CoV-2-based neutralization assay
Neutralization capabilities were assessed in discordant samples using the method described in Andreano et al. . Briefly, the samples were heat-inactivated and serially diluted then mixed with a viral solution containing 100× of the tissue culture infectious dose 50 (TCID50) of SARS-CoV-2. After 1 h incubation, the mixture was added to Vero E6 cell monolayers, and incubated for three days at 37 °C The highest plasma dilution that resulted in more than 50% inhibition of cytopathic effect was defined as the sample’s neutralization titer. Accordingly, corresponding titers ≥1:10 were taken as functional equivalents for the positive presence of SARS-CoV-2 neutralization antibodies in the LIAISON® SARS-CoV-2 TrimericS IgG assay. This microneutralization assay has been used as the gold standard confirmation assay for Influenza virus, and other types of viruses, and performs comparably to the plaque reduction neutralization test , , . It was also previously validated for SARS-CoV-2 .
The statistical program MedCalc 19.6 was utilized for all analyses presented. Sensitivity, specificity and % agreement were calculated using a diagnostic 2 × 2 test. Agreement between assays was quantified by the κ statistic using an inter-rater agreement test. Receiver operating characteristics were calculated using the binomial exact methodology. Data supporting Figures and Tables in this manuscript are available as Supplementary Material.
In 1,491 randomly selected residual serum samples from patients in New York, a positive prevalence rate of 23.8% was observed (n=355), while in the New Mexico cohort, ∼1.6% of samples were found to be positive (27 or 34 of 1931 by the DiaSorin or Beckman assays, respectively). Assessment of the LIAISON® SARS-CoV-2 TrimericS IgG assay specificity performance in these two populations against two different automated chemiluminescent immunoassays resulted in negative percent agreements of 99.8% (95% CI 99.5–99.9%) with the Beckman test, and 94.8% (95% CI 93.4–96.0%) with the Siemens test (Table 1). The level of agreement between the LIAISON® SARS-CoV-2 TrimericS IgG assay and the ACCESS SARS-CoV-2 IgG or the Siemens ADVIA Centaur COV2G assay was evaluated by inter-rater agreement: calculated κ=0.750 (95%CI 0.628 to 0.872) and κ=0.868 (95% CI 0.837–0.898) values present “good” and “very good” agreement between the tests, respectively .
|Siemens Centaur SARS-CoV-2 IgG||% Agreement (95% CI)|
|Beckman ACCESS SARS-CoV-2 IgG||% Agreement (95% CI)|
In the low prevalence area, even with the low positive percent agreement of 67.6% (95% CI 49.5–82.6%) between the DiaSorin and Beckman assays, only four samples were discordant, whereas, in the high prevalence area, the positive percent agreement of 98% (95% CI 95.7–99.3%) between the DiaSorin and Siemens assay resulted in 67 discordant samples (Table 1). Positive predictive values for each pair of comparisons were 85.2% (95% CI 67.8–94.0%) and 82.6% (95% CI 78.9–85.8%), respectively. The discordants between the DiaSorin and Siemens assays were analyzed by microneutralization assay: 58 of 60 samples, identified as positive by the DiaSorin assay and negative by the Siemens assay, contained SARS-CoV-2 neutralizing antibodies (dilution titers ≥ 1:10). Two samples were not tested due to insufficient volume. Conversely, of the six samples flagged negative by the DiaSorin assay and positive by Siemens, two contained SARS-CoV-2 neutralizing antibodies, while the remaining four were negative for neutralizing antibodies. Figure 1 presents the distribution of the discordant samples in relation to their neutralization titers according to the Siemens ADVIA Centaur COV2G (Figure 1A), and the LIAISON® SARS-CoV-2 TrimericS IgG result (Figure 1B): samples in the shaded areas show agreement between the immunoassay and microneutralization results (Figure 2).
A receiver operating characteristic analysis was performed to identify how well the LIAISON® SARS-CoV-2 TrimericS IgG assay can detect the presence of SARS-CoV-2 antibodies in tested samples. Classification was determined by consensus of the Siemens and DiaSorin tests, with microneutralization analysis as tie-breaker for discordant samples in a group of 1,489 samples (n=355 positives). The area under the curve was calculated at AUC=0.997 (95% CI 0.993–0.999). The criterion associated to the Youden Index was determined to be >12.9 kAU/L with a sensitivity of 99.44% and a specificity of 99.82%.
The LIAISON® SARS-CoV-2 TrimericS IgG assay was evaluated against the Siemens ADVIA Centaur COV2G test in a New York population where prevalence of COVID-19 is fairly high. While the two tests were in good agreement, 4.6% of the samples had discordant results, which were resolved by microneutralization assay which indicated that, whereas the Siemens assay is highly specific, the LIAISON® SARS-CoV-2 TrimericS IgG assay delivers both high sensitivity and high specificity. In populations with low COVID-19 prevalence, assays which emphasize specificity over sensitivity, while mitigating or reducing the number of apparent false positives, may be acceptable as observed in New Mexico, but in high prevalence populations such as New York, will result in unacceptably high numbers of false negative results. This points to the need for well-balanced assays, with both high sensitivity and high specificity, for the ability to ascertain accurate demographics in areas of high and low COVID-19 infection.
The implications of a more sensitive assay for the detection of SARS-CoV-2 antibodies are particularly important when determining vaccine efficacy. Consequences of false negative results could be two-fold: first, vaccine trial enrollment of subjects that unknowingly already have SARS-CoV-2 antibodies would confound their apparent response to first immunization doses and potentially increase risk of serious disease , with higher boost responses vs. those levels measured from truly naïve subjects . Second, the risk of having a screening test that is not sensitive enough may also result in false negative results post-immunization thus impacting efficacy estimates of the vaccine.
Funding source: DiaSorin
Funding source: Department of Health and Human Services; Office of the Assistant Secretary for Preparedness and Response; Biomedical Advanced Research and Development Authority
Award Identifier / Grant number: Contract No. 75A50121P00009
We thank the product development team at DiaSorin, and all the laboratory technical staff for sample collection and analysis, and data collection.
Research funding: This project was funded in whole or in part with Federal funds from the Department of Health and Human Services; Office of the Assistant Secretary for Preparedness and Response; Biomedical Advanced Research and Development Authority, under Contract No. 75A50121P00009. This funding organization played no role in the study design, in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication. We also acknowledge financial support from DiaSorin Inc.
Author contributions: FB, DGG, LP, VAC and JJW contributed to the conception or design of the work. TB, EC, ADO, GM, EM, LO, AT, AW, DZ contributed to data acquisition. FAB, TB, CZ contributed to the analysis and interpretation of the data. CZ and FAB drafted the manuscript, all other authors contributed to revisions. All authors approved of the final version to be published. All authors are accountable for all aspects of the work, ensuring accuracy and integrity of the work.
Competing interests: FB, FAB, TB, LP, DZ, and JJW are employees of DiaSorin the manufacturer of the LIAISON® TrimericS IgG test. CZ is a consultant to DiaSorin. Employees and consultant of DiaSorin participated in the study design, data collection, data interpretation, and in the preparation of the manuscript. All others have no conflicts of interest.
Informed consent: Not applicable.
Ethical approval: The study was performed on fresh or frozen de-identified residual serum or plasma samples. Samples were collected in line with institutional IRB guidelines.
1. Dhama, K, Sharun, K, Tiwari, R, Sircar, S, Bhat, S, Malik, YS, et al.. Coronavirus disease 2019–COVID-19. Clin Microbiol Rev 2020;33e:00028. https://doi.org/10.1128/CMR.00028-20.Search in Google Scholar
3. Machhi, J, Herskovitz, J, Senan, AM, Dutta, D, Nath, B, Oleynikov, MD, et al.. The natural history, pathobiology, and clinical manifestations of SARS-CoV-2 infections. J Neuroimmune Pharmacol 2020;15:359–86.10.1007/s11481-020-09944-5Search in Google Scholar PubMed PubMed Central
4. Jeyanathan, M, Afkhami, S, Smaill, F, Miller, MS, Lichty, BD, Xing, Z. Immunological considerations for COVID-19 vaccine strategies. Nat Rev Immunol 2020;20:615–32. https://doi.org/10.1038/s41577-020-00434-6.Search in Google Scholar
6. Mekonnen, D, Mengist, HM, Derbie, A, Nibret, E, Munshea, A, He, H, et al.. Diagnostic accuracy of serological tests and kinetics of severe acute respiratory syndrome coronavirus 2 antibody: a systematic review and meta-analysis. Rev Med Virol 2020:e2181. https://doi.org/10.1002/rmv.2181.Search in Google Scholar
7. Padoan, A, Bonfante, F, Pagliari, M, Bortolami, A, Negrini, D, Zuin, S, et al.. Analytical and clinical performances of five immunoassays for the detection of SARS-CoV-2 antibodies in comparison with neutralization activity. EBioMedicine 2020;62:103101. https://doi.org/10.1016/j.ebiom.2020.103101.Search in Google Scholar
8. Walls, AC, Park, Y-J, Tortorici, MA, Wall, A, McGuire, AT, Veesler, D. Structure, function, and antigenicity of the SARS-CoV-2 spike glycoprotein. Cell 2020;181:281–92. https://doi.org/10.1016/j.cell.2020.02.058.Search in Google Scholar
9. Fenwick, C, Croxatto, A, Coste, AT, Pojer, F, André, C, Pellaton, C, et al.. Changes in SARS-CoV-2 Spike versus nucleoprotein antibody responses impact the estimates of infections in population-based seroprevalence studies. J Virol 2020. https://doi.org/10.1128/JVI.01828-20.Search in Google Scholar
10. Yuan, M, Wu, NC, Zhu, X, Lee, C-CD, So, RT, Lv, H, et al.. A highly conserved cryptic epitope in the receptor binding domains of SARS-CoV-2 and SARS-CoV. Science 2020;368:630–3. https://doi.org/10.1126/science.abb7269.Search in Google Scholar
12. ACCESS SARS-CoV-2 IgG assay instructions for use. Available from: https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas#individual-serological.Search in Google Scholar
13. Andreano, E, Nicastri, E, Paciello, I, Pileri, P, Manganaro, N, Piccini, G, et al.. Identification of neutralizing human monoclonal antibodies from Italian Covid-19 convalescent patients. bioRxiv 2020. https://doi.org/10.1101/2020.05.05.078154.Search in Google Scholar
14. WHO, Global Influenza, Surveillance Network. Manual for the laboratory diagnosis and virological surveillance of influenza. Geneva, Switzerland: WHO Press; 2011.Search in Google Scholar
15. Carpp, LN, Fong, Y, Bonaparte, M, Moodie, Z, Juraska, M, Huang, Y, et al.. Microneutralization assay titer correlates analysis in two phase 3 trials of the CYD-TDV tetravalent dengue vaccine in Asia and Latin America. PloS One 2020;15:e0234236. https://doi.org/10.1371/journal.pone.0234236.Search in Google Scholar
16. Smither, SJ, Lear-Rooney, C, Biggins, J, Pettitt, J, Lever, MS, Olinger, GGJr. Comparison of the plaque assay and 50% tissue culture infectious dose assay as methods for measuring filovirus infectivity. J Virol Methods 2013;193:565–71. https://doi.org/10.1016/j.jviromet.2013.05.015.Search in Google Scholar
17. Manenti, A, Maggetti, M, Casa, E, Martinuzzi, D, Torelli, A, Trombetta, CM, et al.. Evaluation of SARS‐CoV‐2 neutralizing antibodies using a CPE‐based colorimetric live virus micro‐neutralization assay in human serum samples. J Med Virol 2020;92:2096–104. https://doi.org/10.1002/jmv.25986.Search in Google Scholar
20. Zhu, FC, Guan, XH, Li, YH, Huang, JY, Jiang, T, Hou, LH, et al.. Immunogenicity and safety of a recombinant adenovirus type-5-vectored COVID-19 vaccine in healthy adults aged 18 years or older: a randomised, double-blind, placebo-controlled, phase 2 trial. Lancet 2020;396:479–88. https://doi.org/10.1016/s0140-6736(20)31605-6.Search in Google Scholar
The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2021-0023).
© 2021 Fabrizio Bonelli et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.