Analytical and clinical performances of a SARS-CoV-2 S-RBD IgG assay: comparison with neutralization titers

Objectives: SARS-CoV-2 serology presents an important role in several aspects of COVID-19 pandemic. Immunoassays performances have to be accurately evaluated and correlated with neutralizing antibodies. We investigated the analytical and clinical performances of a SARS-CoV-2 RBD IgG assay, automated on a high throughput platform, and the correlation of the antibodies (Ab) levels with the plaque reduction neutralization (PRNT 50 ) Ab titers. Methods: A series of 546 samples were evaluated by SARS-CoV-2 RBD IgG assay (Snibe diagnostics), including 171 negative and 168 positive SARS-CoV-2 subjects and a further group of 207 subjects of the COVID-19 family clusters follow-up cohort. Results: Assay imprecision ranged from 3.98 to 12.18% being satisfactory at low and medium levels; linearity was excellent in all the measurement range. Considering specimens collected after 14 days post symptoms onset, overall sensitivity and speci ﬁ city were 99.0 and 92.5%, respectively. A total of 281 leftover samples results of the PRNT 50 test were available. An elevated correlation was obtained between the SARS-CoV-2 RBD IgG assay and the PRNT 50 titer at univariate ( ρ =0.689) and multivariate ( ρ =0.712) analyses. Conclusions: SARS-CoV-2 S-RBD IgG assay shows satisfactory analytical and clinical performances, and a strong correlation with sera neutralizing activity.


Introduction
Current testing for SARS-CoV-2 largely depends on laborintensive molecular techniques, particularly reverse transcription real-time polymerase chain reaction (rRT-PCR), but a body of evidence highlights that individuals with positive molecular tests represent only a small fraction of all infections [1,2].
Serological assays for the accurate measurement of SARS-CoV-2 antibodies (Abs) are suboptimal tools for the early diagnosis of infection but provide important population-based data on pathogen exposure, on the prevalence of infection, also in asymptomatic subjects, and on the selection of convalescent plasma donors. Furthermore, SARS-CoV-2 serology represents a complementary tool of molecular virological assays to achieve a more accurate diagnosis in some "difficult" patients, for tracking transmission dynamics, gaining knowledge on population immunity levels and informing disease control policies [3]. In addition, serology plays a central role in clinical trials on vaccine development to provide evidence of potency and efficacy [4,5] and in supporting decisions on population groups who should be prioritized in vaccine administration [6]. Many assays have been developed for SARS-CoV-2 Ab detection, including lateral flow tests, enzyme-linked immunosorbent assays (ELISA), chemiluminescent (CLIA) assays and other platforms (https://www.fda.gov/medicaldevices/emergency-use-authorizations-medical-devices/ coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices); some rely on whole inactivated virions, while others adopted viral subunits such as the nucleocapsid protein, or viral spike protein; however, key issues such as the correlation between circulating antibodies and their neutralizing ability and persistence over time have not been adequately addressed, yet. More recently, a body of evidence has been collected to demonstrate that the recombinant SARS-CoV-2 receptor binding domain (RBD) is a highly sensitive and specific antigen for the detection of antibodies induced by SARS-CoV-2 and that the levels of RBD-binding antibodies present a strong correlation with neutralizing antibodies in COVID-19 patients [7,8].
Aim of this paper is the analytical and clinical evaluation of a SARS-CoV-2 RBD IgG assay, automated on a high throughput platform and the correlation of IgG levels with neutralizing antibodies.

Materials and methods Patients
A total of 546 leftover serum samples from 168 COVID-19 patients (24 asymptomatic or with Mild disease [Asympt/Mild], who recovered at home with supportive care and isolation, and 144 hospitalized subjects classified with mild or moderate/severe disease following WHO interim guidance [9]) and 171 SARS-CoV-2 negative subjects (97 pre-pandemic samples from healthy donors, 31 healthcare workers, 11 and 32 patients with rheumatic disease or with human immunodeficiency virus [HIV]) were included in the study. Furthermore, 207 subjects of the COVID-19 Family Cluster Follow-up Clinic (CovFC), set up at the Department of Women's and Children's Health of the University Hospital of Padua were studied. Families were enrolled when complied with the following inclusion criteria: a) having children of pediatric age; b) having a history of medically confirmed COVID-19 or being a household member of a COVID-19 confirmed case.
All subjects underwent at least one nasopharyngeal swab test analyzed by rRT-PCR. Healthcare workers were considered negative (HCW) on the basis of at least three recent negative sequential molecular test results obtained between February 26th and May 29th, 2020. Information concerning family clusters past and recent history were collected retrospectively through both patients interviews and the revision of clinical files. Family subjects who had tested positive for SARS-CoV-2 by rRT-PCR and/or by either of the two serological tests adopted in this study were considered confirmed COVID-19 cases. For each confirmed COVID-19 case, a baseline was defined as the most likely onset of infection, based on different criteria. In detail, for patients reporting COVID-19 related symptoms, the baseline coincided with the onset of symptoms; in case of asymptomatic patients the baseline referred to the date the first positive NP swab was recorded.
Among SARS-SARS-CoV-2 patients in family clusters, five were hospitalized for moderate disease, whereas the others were recovered at home.
The study protocol (number 23307) was approved by the Ethics Committee of the University-Hospital, Padova. All the patients were informed of the study and voluntarily agreed to participate, providing a written consent.

Analytical system under evaluation
In this study, a commercially available immunoassay was evaluated, the anti-SARS-CoV-2 S-RBD IgG (Snibe Diagnostics, New Industries Biomedical Engineering Co., Ltd [Snibe], Shenzhen, China). SARS-CoV-2 S-RBD IgG is a chemiluminescent immunoassay (CLIA) that determines IgG Ab against the RBD of the Spike (S) protein of the virus, in human serum or plasma. All analyses were performed on MAGLUMI™ 2000 Plus (Snibe Diagnostics), with results expressed in kiloastronomical unit.

Repeatability and intermediate precision evaluation
Precision estimation was performed using three human serum sample pools with different values, by means of triplicate measurements of same pool aliquots, performed for a total of five consecutive days. Nested analysis of variance (ANOVA) was used to estimate precision, following the CLSI EP15-A3 protocol [10]. The results for precision were compared to those claimed by the manufacturer, using the procedure recommended by EP15-A3.
Linearity assessment of Maglumi anti-SARS-CoV-2 S-RBD Linearity was assessed using two samples pools (high level pools), prepared with different levels of SARS-CoV-2 antibodies, serially diluted in low level pools, as specified in the CLSI EP06-A guideline (paragraph 4.3.1) [11]. The following high-level serum pools were prepared: 3.7 and 71 kAU/L. The pools were serially diluted with the corresponding low-level serum pools (0.181 and 0.59 kAU/L). All measurements were performed in triplicate. Polynomial regression was used to test deviation from linearity.

Plaque reduction neutralization test (PRNT)
A high-throughput PRNT method was used for the fast and accurate quantification of neutralizing antibodies in plasma samples collected from patients exposed to SARS-CoV-2, as described elsewhere [12]. Briefly, after heat-inactivation, samples were diluted in Dulbecco modified Eagle medium (DMEM) and then mixed with a virus solution containing 20-25 focus-forming units (FFUs) of SARS-CoV-2. After 1 h at 37°C, 50 μL of the virus-serum mixtures were added to confluent monolayers of Vero E6 cells, in 96-wells plates and incubated for 1 h at 37°C, in a 5% CO 2 incubator. After 26 h of incubation and cells fixing, visualization of plaques was obtained with an immunocytochemical staining method using an anti-dsRNA monoclonal antibody (J2, 1:10,000; Scicons) for 1 h, followed by 1 h incubation with peroxidaselabeled goat anti-mouse antibodies (1:1,000; DAKO) and a 7 min incubation with the True Blue™ (KPL) peroxidase substrate. FFUs were counted after acquisition of pictures at a high resolution of 4,800 × 9,400 dpi, on a flatbed scanner. The serum neutralization titer was defined as the reciprocal of the highest dilution resulting in a reduction of the control plaque count >50% (PRNT 50 ). From previous experiments, we defined a titer of 1:10 as the seropositive threshold [12].

Statistical analyses
For evaluation of precision, an in-house developed R (R Foundation for Statistical Computing, Vienna, Austria) script for implementing the CLSI EP15-A3 protocol was used for ANOVA and for calculating the upper verification limit [10]. The GraphPad Prism version 9.1 for Windows (GraphPad Software, LLC) was employed to evaluate plaque reduction neutralization test results. Stata v16.1 (Statacorp, Lakeway Drive, TX, USA) was used to evaluate the assays' clinical performances. Bonferroni's adjusted p-value (B-adj) was calculated for multiple comparisons. For ROC analyses, the non-parametric empirical method was used to estimate the area under the ROC curve (AUC), while the 'diagt' module was used to estimate sensitivity, specificity, and positive and negative predictive values. Youden index (calculated as sensitivity + specificity-1) was used to estimate the best performances of the assay. Considering a type I error α=0.05, a power of 0.8 and with 249 positive and 249 negative subjects, a sensitivity (or specificity) of 0.95 can be considered significant with respect to values above or equal to 0.99 (null hypothesis). PASS 2020 Power Analysis and Sample Size Software (2020), NCSS, Kaysville, Utah, USA, was used for sample size and power analyses.

Patients'characteristics
Demographic characteristics of the subjects included in the study are reported in Table 1. The overall mean age was 42.5 years, with a standard deviation (±SD) of 22.6 (range 0.7-92.2 years). Excluding family clusters, the remaining subjects (n=337) presented a mean age (±SD) of 53.7 ± 16.9 years. A multivariate ANOVA analysis was performed considering Age as dependent variable and Gender and studied groups (F=56.55, p<0.001) as independent variables. The ages of family clusters differed from other groups (Bonferroni's adjusted [B-adj] p<0.001 for all), except for Asympt/Mild positive patients (B-adj p=0.051). Age of negative healthcare workers (HCW), pre-pandemic subjects and Asympt/Mild patients were not statistically significant different (p=0.999), while these groups' age differ with respect to hospitalized COVID patients (B-adj p<0.0001). Age of Rheumatic disease/HIV patients differs from other groups (B-adj p<0.001 for all), with the exception of Asympt/Mild disease group (B-adj p=0.493). The percentage of females differed significantly from that of males (p<0.001), particularly in the Asympt/Mild disease group. For SARS-CoV-2 patients, the mean time interval from the onset of symptoms and serological determinations was 17.7 days (SD ± 16.3; range 1-103 days). In the family clusters, the mean time interval from the onset of symptoms and serological determinations was 148.2 days (SD ± 71.2; range 41-257 days). The differences in time from symptoms onset with respect to the studied groups of individuals were reported in Table 2.

Repeatability and intermediate precision
Results for precision of CLIA assay is reported in Table 3. Repeatability and within-laboratory precision were in accordance with the repeatability and intermediate precision conditions specified in the international vocabulary of metrology (VIM, JCGM 100:2012) for precision estimation within a five-day period. Obtained data show acceptable imprecision at low and medium levels, but significantly deviated from the values claimed by the manufacturer for the high-level control material.

Linearity assessment
Linearity results for Maglumi anti-SARS-CoV-2 S-RBD are summarized in Figure 1. Since the method is claimed to be quantitative, tested mixes were prepared for covering a wide range of values (the upper limit of the method without sample dilution is 100 kAU/L), including the manufacturers' cut-offs. Maglumi anti-SARS-CoV-2 S-RBD IgG does not deviate from linearity in the entire range of tested values, being the coefficients of the second-order polynomial non-statistically significant.    Sensitivities, specificities, and positive/negative likelihood ratios, estimated using the manufacturers' cut-offs and considering samples collected from 14 days postsymptoms onset, were reported in Figure 3 and Supplementary Table 1. Receiver operating characteristic (ROC) curves were further reported in the same table. A further analysis was performed, using the Youden index strategy, for identifying the most accurate cut-off; However, the cutoff calculated with Youden's index (0.96 kAU/L) does not significantly improve the clinical performances when compared to that recommended by the manufacturer (1.0 kAU/L).

Evaluation of clinical performances
Although sensitivity and specificity are helpful for clinical purposes, positive and negative predictive values (PPV and NPV) are more relevant in clinical decision making. Using two different scenarios of disease prevalence settings, (a) 4%, as found in a Veneto Region (Italy) survey [13] and (b) 10%, as described in a survey conducted in Geneva [14], PPV and NPV were then estimated. Considering performances derived from sera collected 14 days after the onset of symptoms on Asympt/Mild symptomatic subjects, mimicking a survey conducted in a population not reporting symptoms attributable to COVID-19, the PPV (95%CI) and NPV (95% CI) were 31.8% (21.8-43.9%) and 99.9% (98.6-100%) with a prevalence of 4%, and 55.4% (42.6-67.6%) and 99.0% (96.3-99.7%) with a prevalence of 10%.

CLIA results correlation with PRNT 50 results
Considering all individuals included in the study, a total of 281 leftover samples results of the PRNT 50 test were available (Table 2 and Supplementary Figure 1). The relationship among the anti-SARS-CoV-2 S-RBD IgG and the corresponding PRNT 50 titers is shown in Figure 4, panels A and B. Overall, positive associations were found between log 10 PRNT 50 titer and log 10 Ab results. An elevated correlation was obtained (ρ=0.689, p<0.001) at univariate analyses. At multivariate analyses, performed including Age, Gender and the time from symptom onset and serological determination in the linear regression model, a similar correlation coefficient was found (R 2 adj=0.508, ρ=0.712), being only Age (p=0.013) and time post symptom onset (p=0.041) statistically significant. In a further subanalysis, including also disease severity, this latter variable results not significantly associated with log 10 PRNT 50 titer.
Since for COVID-19 convalescent plasma treatments a high neutralization titer is advisable, a further analysis was performed [15]. Figure 4 (panel C) shows the percentage of samples with a PRNT 50 titer ≥1:160 with respect to the ranges of S-RBD IgG results. For CLIA result above 75 kAU/L, a neutralizing titer ≥1:160 was

Discussion
This paper reports a head-to-head evaluation of the Snibe anti-SARS-CoV-2 S-RBD IgG CLIA analytical performances, since this assay is claimed to be quantitative and the evaluation of these characteristics is especially important for monitoring seroconversion and antibody persistence. Results showed that this assay presents excellent analytical performances, both for precision and linearity. The repeatability was less than 6% for all the studied levels, while intermediate precision was more elevated at the lowest level (1 kAU/L), which is close to the cut-off proposed by the manufacturer (Table 3). Precision performances statistically deviated from the manufacturer's claims only at the highest level (6.14 kAU/L), as the precision value reported inside the inserts at 5.11 kAU/L were 2.25 and 2.40%. The adoption of serological testing for monitoring of Ab titers requires, in addition to assay robustness, a good method linearity, to effectively quantify differences between measured values. Our data demonstrate that anti-SARS-CoV-2 S-RBD IgG presents excellent linearity not only within the range of values including the cut-off (0.2-4 kA/ L) but also for the highest values (from 5 to 70 kAU/L) ( Figure 1); notably, these findings are relevant when considering that, in vaccinated subjects, Ab values above the limit of the method are often detected, requiring a  further dilution step for delivering results (data not shown).
On a large panel of blood samples, including prepandemic, negative HCW, and negative AI/HIV specimens and SARS-CoV-2 patients with different severity of disease (Asymp/Mild, Severe and Critical), using the pre-defined assay thresholds for calling test results positive or negative, overall sensitivity and specificity were around 97% and 92%, respectively (Figure 3 and Supplementary Table 1). The suboptimal specificity is related to the presence of some false-positive results obtained for 14 samples (including four pre-pandemic, two AI, and eight HCW specimens), and which may affect all currently available immunoassays. In agreement with the time-dependent nature of antibody response, different results are obtained assessing samples collected at least 14 days post symptoms onset [16]. Accordingly, two separate analyses were conducted. In the time frame from 14 days post symptom onset, using all negative subjects as references (Ref), better sensitivity results were achieved for critical rather than severe disease patients, despite the anti-SARS-CoV-2 S-RBD IgG did not differ between the groups of severe and critical patients. Comparing Ref and family clusters, performances of anti-SARS-CoV-2 S-RBD IgG were excellent, being sensitivity 100% and specificity above 92%; remarkably, all samples of this group were collected after 14 days post symptom onset. Considering samples from family clusters, a slight statistically significant timedependent decrease of anti-SARS-CoV-2 S-RBD IgG was observed, and linear regression allowed to estimate a change in Ab levels, with a confidence of 95%, from −0.17 to −0.04 kAU/L per day (Supplementary Figure 2) and in a further analysis, performed excluding individuals aged <30 years, findings confirmed the magnitude of the linear slope. These results are fully in accordance with our previously reported data [12], suggesting that, with the exception of some individuals, immunological memory remain persistently elevated for months up to 10 months [17,18].
The relationship between SARS-CoV-2 antibodies and neutralizing activity remains an essential and open issue. In fact, SARS-CoV-2 neutralizing antibodies (NAb) titer is currently gaining importance for supporting vaccine development, and to aid convalescent plasma therapy. Therefore, due to the high demand for the neutralization test, a surrogate method to evaluate their levels in patients with varying severity of illness at a various time points is strongly advisable, also for circumventing the need to handle live virus in BSL-3 laboratories. Alternatively, recently developed surrogate virus neutralization assays are coming on the marker, and these methods should be assessed and validated more extensively in future before a widely utilization [19]. For this reason, we assessed the correlation between the plaque reduction neutralization, the gold standard methods for determining the titer of NAb, with anti-SARS-CoV-2 S-RBD IgG levels. Overall, the anti-SARS-CoV-2 S-RBD IgG levels showed a good dynamic range and the response of the method was highly correlated with PRNT 50 titers (Pearson ρ=0.712 at multivariate analysis) (Figure 4). In addition, when the percentage of samples with a PRNT 50 titers ≥1:160 was calculated with respect to the ranges of anti-SARS-CoV-2 S-RBD IgG, results above 75 kAU/L presented a neutralizing titer ≥1:160 in the 94.3% of samples. These results are in accordance with our previously reported findings, performed in different assays, which gave similar results of this anti-SARS-CoV-2 S-RBD IgG. Currently, a small number of studies have validated a range of commercially available SARS-CoV-2 serological assays against a live-virus neutralization test [12,[20][21][22][23][24][25], and in our knowledge this is the first study comparing Snibe anti-SARS-CoV-2 S-RBD IgG levels and PRTN 50 titers. Walker et al. evaluated different commercially available assays for their correlation with the microneutralisation assay and reported values ranging from 69 to 100%, with assays measuring total antibodies being the most sensitive [18]. Differently, Legros et al. found that Diasorin SARS-CoV-2 S1/S2 kit anti-S IgG titers correlated highly with microneutralization nAb titers (Spearman's ρ = 0.7075) [21]. Patel et al., who evaluated five immunoassays with respect to NAb results, observed that the strongest correlation was ρ=0.81 (with the ELISA from Euroimmun) and the weakest correlation was ρ=0.40 (with Roche CLIA assay) [22].
This study presented several limitations. First, neutralizing antibodies were mainly tested in a welldefined cohort of family cluster, with sera collected at various time points and, therefore, should be confirmed in further studies; second, COVID-19 positive patients were selected retrospectively on the basis of available leftover samples, and third cross-reactivity with seasonal human coronaviruses was not assessed; therefore NPV and PPV could be overestimated. Another limitation of this study is that no longitudinal sera were analyzed and, therefore, we cannot exclude that some patients might have seroconverted at later time points.
In conclusion, the data reported in this study showed that anti-SARS-CoV-2 S-RBD IgG assay achieves excellent analytical and clinical performances. Since specificity results were not 100%, the assay might present a limited number of false-positive results and this characteristic could be further confirmed in a more representative number of samples. However, the correlation with sera neutralization activity was very elevated, demonstrating that the dynamic range of the assay is expanded enough to capture all clinically significant NAb results. Finally, an appropriate threshold could be derived for selecting samples for COVID-19 convalescent plasma therapy.
Further studies are needed to clarify whether the current generation of anti-SARS-CoV-2 neutralizing antibodies, and eventually circulating IgG antibodies, would retain clinical significance in samples of patients infected with virus variants, mainly B.1.351 and P.1, which may hence generate a class of antibodies non-reacting with the recombinant (RBD) antigen(s) of the evaluated assay [26].