To the Editor,
The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a new, rapidly spreading human beta coronavirus. It was first identified in Wuhan and caused a disease named Coronavirus Disease 2019 (COVID-19) . COVID-19 patients may commonly present with fever, cough, myalgia, or fatigue. The severe cases progress rapidly to acute respiratory distress syndrome, shock, organ dysfunction, and even death . According to the latest data, the WHO confirmed a total of 80,565 cases infected with SARS-CoV-2 and 3,015 deaths in China on March 05, 2020. And the virus has infected 95,333 individuals globally and spread to at least 79 countries, such as Korea, Italy, and Iran . The entire viral genome is more than 29,000 nt in length and has 12 protein coding regions (1ab, S, 3, E, M, 7, 8, 9, 10b, N, 13, 14) , . Based on the Diagnosis and Treatment Program of COVID-19 (seventh trial version) formulated by the National Health Commission of China, real-time polymerase chain reaction (PCR) assays have been regarded as the reference standard to make a definitive diagnosis of COVID-19 (see Supplementary Table 1). Therefore, various commercial kits were available in China during the outbreak of disease. Most of them targeted the viral genes so far include the ORF1ab, N, and E genes, as in the world health organization (WHO) Laboratory guidance document (https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/laboratory-guidance). Notably, different countries have distinct target genes for SARS-CoV-2 detection. The China centers for disease control and prevention (CDC) requires that the ORF1ab and N genes should be detected simultaneously in the positive samples (see Supplementary Table 2). At the time of submission of this paper, in consideration of the essential time diagnosis of COVID-19, various commercial kits were authorized for emergency use in the clinical laboratories. However, they lack validation for diagnostic use in the clinic. Hence, we selected seven representative commercial kits to evaluate their performance in SARS-CoV-2 RNA detection.
A total of seven domestic diagnosis kits from seven biopharmaceutical companies were included in our study: kit A (BGI Biotech Co., Ltd, Wuhan, China), kit B (Outdo Biotech Co., Ltd, Shanghai, China), kit C (Sansure Biotech Inc., Changsha, China), kit D (Perkin Elmer Medical Diagnostic Products, Co., Ltd, Shanghai, China), kit E (Daan Gene Co., Ltd. Of Sun Yat-Sen University, Guangzhou, China), kit F (Jiangsu Bioperfectus Technologies Co., Ltd, Taizhou, China), and kit G (Fosun Long March Medical Science Co., Ltd, Shanghai, China). Among these, all of them have been used in the clinic. But only the kit A, kit C, and kit E were approved by the National Medical Products Administration (NMPA). In different regions of China, the clinical laboratories selected assays from diverse manufacturers. For instance, the kit C and kit F had a higher frequency of clinical use in Hunan Province. Additionally, the kit C has even been applied in Japan. 42 archived SARS-CoV-2-positive samples (34 throat swabs and eight fecal samples) and 200 randomly selected SARS-CoV-2-negative samples were collected from the Second Xiangya Hospital. The reference results not only consider the clinical diagnosis of the case but also consider the laboratory nucleic acid testing results (see Supplementary Tables 1, 2). The interpretation criteria for SARS-CoV-2-positive samples included: 1) The result in both SARS-CoV-2 ORF1ab and N gene was positive (two commercial kits (kit C and kit F) were performed to validate the detection results); 2) Epidemiologic history, clinical situation, laboratory tests, and imaging examinations accord with COVID-19 diagnostic criteria. Total nucleic acid extractions were performed on the NP968-S Nucleic Acid Extraction System (Tianlong Science & Technology Co., Ltd, Xi’an, China) with paramagnetic particle method, using the Virus DNA/RNA Isolation Kit (Tianlong Science & Technology Co., Ltd, Xi’an, China) and the default instrument settings. The residual specimens and nucleic extracts were retained and restored immediately after initial clinical testing at −80 °C. These samples were thawed once before testing. Then, we diluted the unfrozen nucleic acid according to the instructions of the selected kits and actual sample volume. Seven kits tested samples in parallel and according to the manufacturer’s introductions. RNA extraction procedures have been qualified and validated for quality and purity before trial. All of the samples were conducted using strict quality control and quality assurance procedures. The researcher performing testing was blinded to the original SARS-CoV-2 screening outcomes of these 242 specimens.
Table 1 summarizes the specifications of SARS-CoV-2 commercial kits included in this study. Most of them were suitable for respiratory specimen’s detection. The kit D could also be applied for blood samples. The kit D required the highest volume of the nucleic acid, and the RNA volume was only 5 μL for kit E and kit F. The discrepancy of template volume may significantly contribute to the clinical performance of detection methods. The target genes determined by seven kits were not the same. All the kits were designed to target the virus ORF1ab gene, but kit F and kit G could detect three genes simultaneously. Based on the different results interpretation criteria of the seven kits, we categorized them into three groups: group Ⅰ: SARS-CoV-2-positive: ORF1ab gene-positive or N gene-positive in samples; group Ⅱ: ORF1ab gene-positive and N gene-positive in samples; group Ⅲ: E gene is included in the kit. The kit D and E kits in group Ⅱ were consistent with the results interpretation standard of China CDC.
|Study kit||Kit A||Kit B||Kit C||Kit D||Kit E||Kit F||Kit G|
|Specimens||OP, BALF||OP, BALF||OP, BALF||OP, NP, BALF, sputum, serum, plasma||OP, sputum||OP||OP, NP|
|Results Interpretation (SARS-CoV-2 detected)||ORF1ab+||
||ORF1ab+ and N+||ORF1ab+ and N+||
|Limit of detection (copies/mL)||100||200||30||500||1000||1000|
|RNA Vol. (μL)||10||15||20||40||5||5||10|
|Fluorescent Capture Tempb||60 °C||60 °C||60 °C||65 °C||55 °C||55 °C||60 °C|
|PCR cyclec, Ct||40, ≤38||40, <37||45, ≤40||45,≤42||45,≤40||45,≤35 (indeterminate# > 35– ≤ 38)||40, ≤38|
OP, oral pharyngeal; NP, nasal pharyngeal; BLAF, bronchoalveolar lavage fluid; Vol., volume.aInterpretation groups: Group Ⅰ: ORF1ab gene-positive or N gene-positive in SARS-CoV-2 positive samples, Group Ⅱ: ORF1ab gene-positive and N gene-positive in positive samples, Group Ⅲ: E gene included in the kit. bFluorescent Capture Temp, the temperature at which fluorescent signals are collected.cPCR cycle, the total number of PCR cycles used per kit instructions.
Subsequently, we evaluated the diagnostic performances of the seven kits in the clinical samples (Table 2). Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Youden index were calculated. Cohen’s Kappa coefficient, McNemar test, chi-square (χ2) values were carried out using SPSS software (version 26.0). As shown in Table 2, the diagnostic specificity of seven kits was all 100%. The concordance of kit A, kit B, kit D, and kit F and “reference result” was robust (Cohen’s Kappa coefficient>0.900, p<0.001). The diagnosis results of kit C, kit E, and kit G and “reference result” were not identical (p<0.05). The clinical sensitivity for three assays in group Ⅰ was 90.48, 92.86, and 83.33%, respectively (see Supplementary Table 3). The difference between them is not statistically significant. As for group Ⅱ and group Ⅲ, kit D and kit F demonstrated superior sensitivity (p<0.05).
|Kits||% Sensitivitya||% Specificitya||% Positive predictive value (95%CI)||% Negative predictive value (95%CI)||Youden index||Kappa test (kappa coefficient)||McNemar test (p-value)|
The combination of clinical diagnosis and laboratory nucleic acid results of cases was taken as the “reference result”; CI, confidence interval.aValue in parentheses represents the number of positives/total number of true positives (sensitivity) or the number of negatives/total number of true negatives (specificity); bp < 0.001. Bold values indicate significant differences.
We also calculated the detection rate on different target genes (ORF1ab, N, and E) of these assays (see Supplementary Table 4). It has to be mentioned that the kit B used only one fluorescent channel (FAM) to detect two target genes simultaneously, so we couldn’t differentiate which gene of the SARS-CoV-2 was discovered in the positive samples. Apart from the kit B, we observed that the kit D had the highest positive detection rate of the virus ORF1ab gene (92.86%). Though the kit C and kit G were able to detect the N gene of SARS-CoV-2, the detection rates of the ORF1ab gene for them were only 23.81 and 30.95%, respectively. The kit D and F had the same positive detection rate of the N gene (90.48%). The detection rate of the E gene for the group Ⅲ kit was 66.67 and 85.71%, respectively.
To sum up, we summarized and compared the detailed information of seven SARS-CoV-2 RNA detection kits in the present study. The assessment of commercial kits in the clinical application involves many factors such as specimen types, target genes, the limit of detection, interpretation criteria, etc., As a result of time to market, not all kits have satisfied with China CDC standards or been approved by NMPA. So far, more than 100 pharmaceutical companies have successfully developed SARS-CoV-2 RNA detection reagents in China. We discovered that all seven kits have excellent specificity, but there were differences in their clinical sensitivity. If the clinical laboratories need to perform virus single gene detection for COVID-19 screening, the clinical performances of group Ⅰ kits were similar. If selecting a kit that meets the China CDC standard is necessary, the kit D was better. Considering the E gene detection, the kit F had higher clinical sensitivity. Otherwise, the detection ability of the ORF1ab gene for some kits was weak. We speculated that this might be related to the primer design methods for different kits. Interestingly, we proposed that though respiratory samples have the greatest yield, the virus can be detected in other specimens, such as stool. Recently, the Chinese researches pointed out that they successfully isolated the SARS-CoV-2 in the urine. It remains to verify whether the kits in our study could also be used in urine samples detection.
It is worth noting that our study just made a simple comparison of the detection capacity of each reagent in a specific batch. The tested samples had been preserved at −80 °C refrigerator in days. Consequently, the results may be interfered with by freshness of samples, sample preservation solution, sample dilution ration, operation process, and other factors. Though efforts were made to design kits to conserved viral genomic regions, variability resulting in mismatches between the primers and probes and the target sequences can result in diminished assay performance. Therefore, our study does not represent the overall detection performance of these kits. Another limitation of our study was the inability to test seven kits with a certified reference material due to a lack of availability of such material at the time of this study. The commutability of the results requires accredited reference materials that can be used to determine performance across different platforms. In general, the clinical laboratory should select the most appropriate kit according to the locally epidemic trend of disease, the actual situation of clinical samples, the purpose of detection, and laboratory conditions.
Funding source: Hunan Scientific and Technological Department Special Project
Award Identifier / Grant number: No.2020SK3018
Research funding: This study was supported by the Hunan Scientific and Technological Department Special Project under the COVID-19 Emergency (No.2020SK3018).
Author contributions: BW: wrote the manuscript; MW, MH: designed the study; ZW, XL, WW, ZL, and XG: performed the sample collection; XX, YR, and MW: performed the sample analysis; LT and ZX: supervised the sample analysis. BW and YP: performed the statistical analysis. MW: critically revised the manuscript. All authors have accepted responsibility for the entire content of the submitted manuscript and approved submission.
Competing interests: Authors state no conflict of interest.
Informed consent: Informed consent was obtained from all individuals included in this study.
Ethical approval: The study complied with the Declaration of Helsinki Principles and was approved by the ethics committee of The Second Xiangya Hospital, Central South University (Changsha, China).
1. Lu, R, Zhao, X, Li, J, Niu, P, Yang, B, Wu, H, et al.. Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. Lancet 2020;395:565–74. https://doi.org/10.1016/S0140-6736(20)30251-8.10.1016/S0140-6736(20)30251-8Search in Google Scholar
2. General Office of National Health Committee. Office of state administration of traditional Chinese medicine. Notice on the issuance of a programme for the diagnosis and treatment of novel coronavirus (2019-nCoV) infected pneumonia (seventh trial edition) (2020-03-03)[EB/OL] . Available from: http://www.gov.cn/zhengce/zhengceku/2020-03/04/content_5486705.html].Search in Google Scholar
3. World Health Organization. WHO Director-General’s remarks at the media briefing on 2019-nCoV on 5 March 2020. Available from: https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19–5-march-2020 [Accessed: 5 Mar 2020].Search in Google Scholar
4. Chan, JF, Kok, KH, Zhu, Z, Chu, H, To, KK, Yuan, S, et al. Genomic characterization of the 2019 novel human-pathogenic coronavirus isolated from a patient with atypical pneumonia after visiting Wuhan. Emerg Microbes Infect 2020;9:221–36. https://doi.org/10.1080/22221751.2020.1719902.[Commentary].10.1080/22221751.2020.1719902Search in Google Scholar PubMed PubMed Central
5. Wu, A, Peng, Y, Huang, B, Ding, X, Wang, X, Niu, P, et al. Genome composition and divergence of the novel coronavirus (SARS-CoV-2) originating in China. Cell Host Microbe 2020. https://doi.org/10.1016/j.chom.2020.02.001. [Epub ahead of print][Commentary].10.1016/j.chom.2020.02.001Search in Google Scholar PubMed PubMed Central
The online version of this article offers supplementary material (https://doi.org/10.1515/j_cclm-2020-0271).
© 2020 Walter de Gruyter GmbH, Berlin/Boston