To the Editor,
Coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus (SARS-CoV-2), continues to be a pandemic. The development of nucleic acid-based tests, in particular real-time reverse transcription polymerase chain reaction (rRT-PCR), has allowed a more accurate diagnosis of the disease to be made, particularly in asymptomatic patients . The development of serological assays, which measure the antibody responses induced by SARS-CoV-2, may improve disease management, but the limited knowledge gained on antibody kinetics, seroconversion rates and correlation with clinical severity are major factors to be evaluated for appropriate adoption of serological testing in clinical practice . Several authors described the analytical validation of available SARS-CoV-2 assays, as well as their diagnostic accuracy , . However, from a clinical viewpoint, the data reported on sensitivity and specificity fail to provide actionable information for either diagnosis or patient management. To determine whether an individual is immune to, or infected by, SARS-CoV-2, we need to know pre-test probability in the specific population being tested, since disease prevalence strongly affects the predictive positive value (PPV). This is also recommended by the US Centers for Disease Control and Prevention (CDC), in an interim guideline . To evaluate the effect of disease prevalence on serological tests, we used the following commercially available chemiluminescent immunoassays (CLIA) for IgG SARS-CoV-2 antibodies: (a) Maglumi (New Industries Biomedical Engineering Co. Ltd [Snibe], Shenzhen, China), (b) Liaison (Diasorin S.p.A. Saluggia [VC], Italy), and (c) iFlash (Shenzhen Yhlo Biotech Co. Ltd. China) in 151 subjects (64 SARS-CoV-2 patients and 87 healthcare workers) who underwent at least one nasopharyngeal swab test, analyzed by RT-PCR. For SARS-CoV-2 patients the mean time interval from the onset of symptoms and serological assessment was 24 days (SD ± 11; range 12–54 days). Of the 87 healthcare workers, 71 were considered negative, since at least three sequential molecular test results were negative, and the remaining 16 were considered positive, with mild disease. For each assay we selected two different thresholds to maximize overall performance (Youden index) and specificity (fixed at 95%), respectively.
Table 1 shows the PPV and negative predictive value (NPV) at three different disease prevalence settings: (a) 4%, as found in a Veneto Region (Italy) survey (data not shown); (b) 10%, as described in a survey conducted in Geneva  and (c) 23%, as reported in a pediatric dialysis unit . While NPV is elevated in all three cases (>94.8%), PPV ranges from 21.1% in the area with the lowest (4%), to 85.7% in the area with highest (23%) prevalence. Figure 1 shows the effect of pre-test probability in reducing uncertainty in diagnosing COVID-19 disease, using the Fagan nomogram and the data from one of the assays evaluated (Liaison). For the other two assays, the effect was similar, thus indicating that only at high disease prevalence does PPV provide valuable and actionable information.
|Disease prevalence||Maglumi (Snibe)||Liaison (Diasorin)||iFlash (Yhlo)|
|Cutoff: 2 kAU/La||Cutoff: 6 kAU/Lb||Cutoff: 6.2 kAU/La||Cutoff: 15 kAU/Lb||Cutoff: 15 kAU/La||Cutoff: 20 kAU/Lb|
|PPV (95% CI)||NPV (95% CI)||PPV (95% CI)||NPV (95% CI)||PPV (95% CI)||NPV (95% CI)||PPV (95% CI)||NPV (95% CI)||PPV (95% CI)||NPV (95% CI)||PPV (95% CI)||NPV (95% CI)|
|0.04 (4)||31.6 (17.7–49.9)||99.7 (99.3–99.9)||33.5 (17.7–54.1)||99.3 (98.9–99.6)||26.2 (13.5–44.8)||99.9 (99.5–100.0)||43.0 (16.3–74.6)||99.2 (98.7–99.5)||21.1 (13.1–32.1)||99.9 (99.5–100.0)||45.6 (21.7–71.8)||99.6 (99.2–99.8)|
|0.10 (10)||55.2 (36.4–72.6)||99.2 (98.3–99.7)||57.3 (36.4–75.8)||98.2 (97.1–99.0)||48.7 (29.3–68.4)||99.6 (98.6–99.9)||66.8 (34.1–88.7)||98.0 (96.7–98.8)||41.6 (28.7–55.8)||99.6 (95.6–99.9)||69.1 (42.5–87.1)||99.0 (97.9–99.5)|
|0.23 (23)||76.8 (60.6–87.7)||98.0 (95.4–99.1)||78.3 (60.6–89.4)||95.4 (92.5–97.2)||71.8 (52.8–85.4)||99.0 (96.2–99.7)||84.4 (58.2–95.5)||94.8 (91.5–96.8)||65.7 (51.9–77.2)||99.0 (96.2–99.4)||85.7 (66.5–94.8)||97.4 (94.6–98.8)|
PPV and NPV are expressed as percentages.
aCutoffs defined on the basis of Youden-index.
bCutoffs defined for maximizing specificities at 95%.
Our data confirm that serological assays are of utmost importance, not only for understanding the prevalence of and immunity against SARS-CoV-2, but also for ruling out suspected disease, being NPV always elevated. Diagnosis according to serology can only be effective in clinical settings with high estimated prevalence. Ideally, a diagnostic test should have excellent performance in both detecting and ruling out the disease, with both a high sensitivity and specificity. However, in the real world this is often a “mission impossible”, calling for optimization in both detection and exclusion of disease. Thus, it is often necessary to define the purpose of the test (detection or exclusion), and calculate the best possible threshold for maximizing sensitivity or specificity. High-quality serological assays have recently been developed and adopted. Yet, this is no time for complacency: the ‘take home’ message is that we must urgently face the new, pressing challenge of applying and deploying these tests in a rational manner. Any failure to do so will seriously undermine diagnostic precision, and quality of care, consequently endangering individuals, and communities at large.
Research funding: None declared.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: Authors state no conflict of interest.
Ethical approval: The local Institutional Review Board deemed the study exempt from review.
1. Mizumoto, K, Kagaya, K, Zarebsky, A, Chowell, G. Estimating the asymptomatic proportion of coronavirus disease 2019 (COVID-19) cases on board the diamond princess cruise ship, Yokohama, Japan, 2020. Euro Surveill 2020;25:10–5. https://doi.org/10.2807/1560-7917.ES.2020.25.10.2000180.Search in Google Scholar PubMed PubMed Central
2. Sethuraman, N, Jeremiah, SS, Ryo, A. Interpreting diagnostic tests for SARS-CoV-2 [published online ahead of print, 2020 May 6]. JAMA 2020. https://doi.org/10.1001/jama.2020.8259.Search in Google Scholar PubMed
3. Padoan, A, Cosma, C, Sciacovelli, L, Faggian, D, Plebani, M. Analytical performances of a chemiluminescence immunoassay for SARS-CoV-2 IgM/IgG and antibody kinetics. Clin Chem Lab Med 2020;58:1081–8.10.1515/cclm-2020-0443Search in Google Scholar PubMed
4. Krammer, F, Simon, V. Serology assays to manage COVID-19 Science. May 15 2020. https://doi.org/10.1126/science.abc1227. pii: eabc1227 [Epub ahead of print].10.1126/science.abc1227Search in Google Scholar PubMed
5. Centers for Disease Control and Prevention. Interim guidelines for COVID-19 antibody testing. https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html [Accessed 3 Jun 2020].10.46234/ccdcw2020.085Search in Google Scholar PubMed PubMed Central
6. Geurtsvankessel, CH, Okba, NMA, Igloi, Z, Embregts, CWE, Laksono, BM, Leijten, L, et al. Towards the next phase: evaluation of serological assays for diagnosis and exposure assessment. medRxiv preprint. https://doi.org/10.1101/2020.04.23.20077156.Search in Google Scholar
7. Hains, DS, Schwaderer, AL, Carroll, AE, Carroll, AE, Starr, MC, Wilson, AC, Amanat, F, et al. Asymptomatic seroconversion of immunoglobulins to SARS-CoV-2 in a pediatric dialysis unit [published online ahead of print, 2020 May 14]. JAMA 2020: e208438. https://doi.org/10.1001/jama.2020.8438.Search in Google Scholar PubMed PubMed Central
© 2020 Mario Plebani et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.