False negative RT-PCR or false positive serological testing in SARS-CoV-2 diagnostics? Navigating between Scylla and Charybdis to prevent misclassification bias in COVID-19 clinical investigations

Justin Benoit 1 , Stefanie W. Benoit 2 , 3 , Giuseppe Lippi 4 ,  and Brandon Michael Henry 5
  • 1 Department of Emergency Medicine, University of Cincinnati, Cincinnati, USA
  • 2 Division of Nephrology and Hypertension, Cincinnati Children’s Hospital Medical Center, Cincinnati, USA
  • 3 Department of Pediatrics, University of Cincinnati, College of Medicine, Cincinnati, USA
  • 4 Section of Clinical Biochemistry, Department of Neuroscience, Biomedicine and Movement, University of Verona, Verona, Italy
  • 5 Cardiac Intensive Care Unit, The Heart Institute, Cincinnati Children’s Hospital Medical Center, Cincinnati, USA
Justin Benoit, Stefanie W. Benoit
  • Division of Nephrology and Hypertension, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
  • Department of Pediatrics, University of Cincinnati, College of Medicine, Cincinnati, OH, USA
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Giuseppe Lippi
  • Section of Clinical Biochemistry, Department of Neuroscience, Biomedicine and Movement, University of Verona, Verona, Italy
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Brandon Michael Henry
  • Corresponding author
  • Cardiac Intensive Care Unit, The Heart Institute, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar

To the Editor,

The conduction of methodologically rigorous clinical research in coronavirus disease 2019 (COVID-19), a pandemic infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), [1] may often require the use of a control group. In observational studies, depending on the objective, a healthy or sick control group may be necessary to enable comparison of variables between COVID-19 positive vs. COVID-19 negative patients. In our ongoing investigations into this severe infectious disease, we aimed to develop a sick cohort of COVID-19 negative patients to serve as a control group, to enable identification of laboratory values unique to COVID-19 patients. Our purpose was to enroll an initial cohort of n=20 COVID-19 negative control patients.

Patients who presented to the Emergency Department (ED) of the University of Cincinnati Medical Center with undifferentiated respiratory symptoms between March and May 2020 were prospectively enrolled as the initial wave of COVID-19 moved through the community. Study samples were taken during collection of routine blood for clinical labs in the ED via an Institutional Review Board approved waiver of informed consent. The investigation was performed in accordance with the declaration of Helsinki and with the terms of local legislation. Patients were initially classified according to the results of their standard-of-care nasopharyngeal reverse transcription polymerase chain reaction (RT-PCR) test for COVID-19. The COVID-19 negative sick controls were defined as RT-PCR negative patients with respiratory symptoms. However, to limit the inclusion of false-negative RT-PCR patients in the control group, we subsequently performed serology testing using the EUROIMMUN Anti-SARS-CoV-2 ELISA IgG and IgA immunoassays (EUROIMMUN AG, Luebeck, Germany). Moreover, the medical records of all RT-PCR negative patients were reviewed for any clinical suspicion of COVID-19 by the treating medical team despite negative RT-PCR results.

Five patients (25%) in the initial COVID-19 negative (RT-PCR negative) control group were found to be positive for anti-SARS-CoV-2 IgA, and were hence excluded from the control group. No patient was found to be positive for anti-SARS-CoV-2 IgG. Two additional control patients who were anti-SARS-CoV-2 IgA and IgG negative, were excluded due to high clinical suspicion of COVID-19 despite multiple RT-PCR negatives results.

These results highlight the difficulties with the design of COVID-19 studies requiring a control group. There may be multiple reasons for our observations. First, suboptimal specificity of the IgA immunoassay for SARS-CoV-2 (i.e., cross-reacting with IgA antibodies generated against other coronaviruses) is a valid hypothesis. However, a recent study published by Beavis et al. showed that the number of Ig-A false positive tests using a panel of common human coronaviruses including types229E, NL63, HKU1 and OC43 was only 7%. [2] On the other hand, in the study of Jääskeläinen and colleagues [3], the specificity of the Euroimmun IgA immunoassay was found to be 73%, thus resulting in a possible rate of false positive tests as high 27%, globally comparable to that potentially found in our study. In another recent investigation published by Jääskeläinen at al. [4], Euroimmun SARS-CoV-2 IgA ELISA was also found to have poor specificity (i.e., 68%) against a reference microneutralisation test. Importantly, the cut-offs adopted by the manufacturers of this assay and other SARS-CoV-2 serology tests, may not provide the proper interpretation of the results in a given population. [5] Thus, it is important to independently and rigorously validate each new serology assay. That said, it cannot be excluded that these five patients with anti-SARS-CoV-2 IgA antibodies have been previously infected by SARS-CoV-2, with no relation to their active respiratory symptoms. However, the presence of IgA but not IgG antibodies may suggest an acute infection, whereby the onset of IgAs may often anticipate that of IgGs. [6]

In addition, samples were collected as COVID-19 initially spread through the United States. Thus, the occurrence of false negative RT-PCR results is another valid hypothesis for our observations, which may be due to several technical issues. In the United States, difficulties with COVID-19 testing, especially early in the pandemic, may have played a role in our observations. Over the time course of this study, the RT-PCR tests used for patient evaluation shifted from an outside facility to internal laboratories. Moreover, a variety of different RT-PCR tests have been used, including rapid testing. It is likely these tests have high variability with respect to their sensitivities, as clearly emphasized elsewhere. [7] Importantly, the test performance and diagnostic validity of many current testing methods have not been clearly reported or well investigated, which raises concern for widespread public testing of COVID-19 generating a false sense of security. [8] Other causes for false negatives may include low nasopharyngeal viral load, inadequate swabbing technique, inappropriate condition of transportation and storage, manual mistakes, sample contamination, and presence of antiretroviral drugs in the test sample. [9]

Regardless of the specific cause of discrepancy found in this study, we suggest a multi-step strategy to limit the likelihood that COVID-19 positive patients be labeled incorrectly as negative controls in future observational or interventional trials. First, standard-of-care RT-PCR tests should be used to exclude COVID-19 positivity. Second, patients with either IgA or IgG positive SARS-CoV-2 serology test results should be excluded from enrollment. Third, since thorough patient-by-patient chart review of controls may not be practical in large studies, we suggest screening serology negative controls with newly available prediction tools, such as the CoronaScore. [10] The CoronaScore is designed for rapid identification of SARS-CoV-2-infected patients at ED presentation via routine laboratory testing, and it has shown high sensitivity for predicting SARS-CoV-2 infection. Patients with scores suggestive of COVID-19 can then be re-tested by means of RT-PCR, further investigation by detailed chart review, or, as we recommend due to the potential for false-negatives discussed above, excluded for high clinical suspicion for SARS-CoV-2. Thus, COVID-19 negative control patients would then be considered only those in whom the three approaches are all negative, as shown in Figure 1.

Figure 1:
Figure 1:

Potential algorithm for recruitment of sick negative control subjects (SARS-CoV-2 negative) in coronavirus disease 2019 (COVID-19) studies.

Citation: Diagnosis 2020; 10.1515/dx-2020-0091

In conclusion, we have shown that recruitment of “sick” control subjects who are truly SARS-COV-2 negative remains a rather challenging enterprise. Otherwise well-designed studies of are likely to produce biased results if RT-PCR tests alone are used to define COVID-19 negative cohorts. We propose here a potential algorithm, which we are currently applying in our own COVID-19 studies, that we hope may be helpful for other working in this field.

Research funding: This study was funded by the University of Cincinnati College of Medicine Special Coronavirus (COVID-19) Research Pilot Grant Program.

Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

Competing interests: Authors state no conflict of interest.

Informed consent: Informed consent was obtained from all individuals included in this study.

Ethical approval: The local Institutional Review Board deemed the study exempt from review.

References

  • 1.

    Lippi, G, Sanchis-Gomar, F, Henry, BM. Coronavirus disease 2019 (COVID-19): the portrait of a perfect storm. Ann Transl Med 2020;8:497. https://doi.org/10.21037/atm.2020.03.157.

    • Crossref
    • PubMed
    • Export Citation
  • 2.

    Beavis, KG, Matushek, SM, Abeleda, APF, Bethel, C, Hunt, C, Gillen, S, et al. Evaluation of the EUROIMMUN anti-SARS-CoV-2 ELISA assay for detection of IgA and IgG antibodies. J Clin Virol 2020;129:104468. https://doi.org/10.1016/j.jcv.2020.104468.

    • PubMed
    • Export Citation
  • 3.

    Jääskeläinen, AJ, Kekäläinen, E, Kallio-Kokko, H, Mannonen, L, Kortela, E, Vapalahti, O, et al. Evaluation of commercial and automated SARS-CoV-2 IgG and IgA ELISAs using coronavirus disease (COVID-19) patient samples. Euro Surveill 2020;25. https://doi.org/10.2807/1560-7917.ES.2020.25.18.2000603.

    • PubMed
    • Export Citation
  • 4.

    Jääskeläinen, A, Kuivanen, S, Kekäläinen, E, Ahava, MJ, Loginov, R, Kallio-Kokko, H, et al. Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation. J Clin Virol 2020;129:104512. https://doi.org/10.1016/j.jcv.2020.104512.

    • PubMed
    • Export Citation
  • 5.

    Plebani, M, Padoan, A, Negrini, D, Carpinteri, B, Sciacovelli, L. Diagnostic performances and thresholds: the key to harmonization in serological SARS-CoV-2 assays?. Clin Chim Acta 2020;509:1–7. https://doi.org/10.1016/j.cca.2020.05.050.

    • Crossref
    • PubMed
    • Export Citation
  • 6.

    Lippi, G, Salvagno, GL, Pegoraro, M, Militello, V, Caloi, C, Peretti, A, et al. Assessment of immune response to SARS-CoV-2 with fully automated MAGLUMI 2019-nCoV IgG and IgM chemiluminescence immunoassays. Clin Chem Lab Med 2020;58:1156–9. https://doi.org/10.1515/cclm-2020-0473.

    • Crossref
    • PubMed
    • Export Citation
  • 7.

    Cradic, K, Lockhart, M, Ozbolt, P, Fatica, L, Landon, L, Lieber, M, et al. Clinical evaluation and utilization of multiple molecular in vitro diagnostic assays for the detection of SARS-CoV-2. Am J Clin Pathol 2020;154:201–7. https://doi.org/10.1093/ajcp/aqaa097.

    • Crossref
    • PubMed
    • Export Citation
  • 8.

    West, CP, Montori, VM, Sampathkumar, P. COVID-19 testing: the threat of false-negative results. Mayo Clin Proc 2020;95:1127–9. https://doi.org/10.1016/j.mayocp.2020.04.004.

    • Crossref
    • PubMed
    • Export Citation
  • 9.

    Lippi, G, Simundic, A-M, Plebani, M. Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19). Clin Chem Lab Med 2020;58:1070–6. https://doi.org/10.1515/cclm-2020-0285.

    • Crossref
    • PubMed
    • Export Citation
  • 10.

    Kurstjens, S, van der Horst, A, Herpers, R, Geerits, MW, Kluiters-de Hingh, YC, Göttgens, EL, et al. Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing. Clin Chem Lab Med 2020. https://doi.org/10.1515/cclm-2020-0593.

    • PubMed
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • 1.

    Lippi, G, Sanchis-Gomar, F, Henry, BM. Coronavirus disease 2019 (COVID-19): the portrait of a perfect storm. Ann Transl Med 2020;8:497. https://doi.org/10.21037/atm.2020.03.157.

    • Crossref
    • PubMed
    • Export Citation
  • 2.

    Beavis, KG, Matushek, SM, Abeleda, APF, Bethel, C, Hunt, C, Gillen, S, et al. Evaluation of the EUROIMMUN anti-SARS-CoV-2 ELISA assay for detection of IgA and IgG antibodies. J Clin Virol 2020;129:104468. https://doi.org/10.1016/j.jcv.2020.104468.

    • PubMed
    • Export Citation
  • 3.

    Jääskeläinen, AJ, Kekäläinen, E, Kallio-Kokko, H, Mannonen, L, Kortela, E, Vapalahti, O, et al. Evaluation of commercial and automated SARS-CoV-2 IgG and IgA ELISAs using coronavirus disease (COVID-19) patient samples. Euro Surveill 2020;25. https://doi.org/10.2807/1560-7917.ES.2020.25.18.2000603.

    • PubMed
    • Export Citation
  • 4.

    Jääskeläinen, A, Kuivanen, S, Kekäläinen, E, Ahava, MJ, Loginov, R, Kallio-Kokko, H, et al. Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation. J Clin Virol 2020;129:104512. https://doi.org/10.1016/j.jcv.2020.104512.

    • PubMed
    • Export Citation
  • 5.

    Plebani, M, Padoan, A, Negrini, D, Carpinteri, B, Sciacovelli, L. Diagnostic performances and thresholds: the key to harmonization in serological SARS-CoV-2 assays?. Clin Chim Acta 2020;509:1–7. https://doi.org/10.1016/j.cca.2020.05.050.

    • Crossref
    • PubMed
    • Export Citation
  • 6.

    Lippi, G, Salvagno, GL, Pegoraro, M, Militello, V, Caloi, C, Peretti, A, et al. Assessment of immune response to SARS-CoV-2 with fully automated MAGLUMI 2019-nCoV IgG and IgM chemiluminescence immunoassays. Clin Chem Lab Med 2020;58:1156–9. https://doi.org/10.1515/cclm-2020-0473.

    • Crossref
    • PubMed
    • Export Citation
  • 7.

    Cradic, K, Lockhart, M, Ozbolt, P, Fatica, L, Landon, L, Lieber, M, et al. Clinical evaluation and utilization of multiple molecular in vitro diagnostic assays for the detection of SARS-CoV-2. Am J Clin Pathol 2020;154:201–7. https://doi.org/10.1093/ajcp/aqaa097.

    • Crossref
    • PubMed
    • Export Citation
  • 8.

    West, CP, Montori, VM, Sampathkumar, P. COVID-19 testing: the threat of false-negative results. Mayo Clin Proc 2020;95:1127–9. https://doi.org/10.1016/j.mayocp.2020.04.004.

    • Crossref
    • PubMed
    • Export Citation
  • 9.

    Lippi, G, Simundic, A-M, Plebani, M. Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19). Clin Chem Lab Med 2020;58:1070–6. https://doi.org/10.1515/cclm-2020-0285.

    • Crossref
    • PubMed
    • Export Citation
  • 10.

    Kurstjens, S, van der Horst, A, Herpers, R, Geerits, MW, Kluiters-de Hingh, YC, Göttgens, EL, et al. Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing. Clin Chem Lab Med 2020. https://doi.org/10.1515/cclm-2020-0593.

    • PubMed
    • Export Citation
FREE ACCESS

Journal + Issues

Diagnosis aims at answering the question how diagnosis determines the quality of medical care. It focuses on how diagnosis can be advanced, how it is taught, and how and why it can fail, leading to diagnostic errors. The journal welcomes both fundamental and applied works, improvement initiatives, opinions, and debates to encourage new thinking on improving this critical aspect of healthcare quality.

Search

  • View in gallery

    Potential algorithm for recruitment of sick negative control subjects (SARS-CoV-2 negative) in coronavirus disease 2019 (COVID-19) studies.