Clinical performance and potential of a SARS-CoV-2 detection kit without RNA purification steps


 
 
 Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is rapidly spreading globally. Early diagnosis plays an essential role in controlling the infection. Therefore, early and accurate SARS-CoV-2 detection assays along with easy operation are required. The aim of this study was to compare the clinical performance of the Ampdirect™ 2019-nCoV Detection Kit (SHIMADZU assay), which does not require RNA purification steps, with that of the preexisting SARS-CoV-2 detection assays, which use a purified RNA template.
 
 
 
 A total of 71 samples (65 nasopharyngeal specimens and 6 sputum specimens) were collected from 32 individuals, including patients infected with SARS-CoV-2 and those with suspected infection. The sensitivity and kappa (κ) coefficient were assessed between the SARS-CoV-2 detection assays using the reference standard, which was defined as a true positive result by any one of the four SARS-CoV-2 detection assays.
 
 
 
 The overall sensitivity and κ coefficient of the SHIMADZU assay were 86.0% (95% confidence interval [CI]: 77.9–94.2) and 0.83 (95% CI: 0.69–0.96), respectively. In particular, among the 18 samples collected within 10 days from symptom onset, the sensitivity and κ coefficient of the SHIMADZU assay were 100% and 1.0, respectively.
 
 
 
 Although a relatively small number of samples was evaluated, the SHIMADZU assay showed good analytical performance and as such would be highly useful for the detection of SARS-CoV-2. The test can be performed easily and quickly and has the potential for future applications in situations where a highly sensitive diagnosis is required.



Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection was first detected in a patient in December 2019 in Wuhan, Hubei, China [1]. SARS-CoV-2 causes the coronavirus disease 2019 (COVID-19), which is mainly presented with fever and respiratory symptoms [2]. COVID-19 has spread widely in many countries, including Japan. As of April 4, 2021, the estimated number of people infected worldwide was 130.5 million, and the death toll was approximately 2,842,325 [3], stressing the need to prevent its spread. SARS-CoV-2 is highly contagious [4], and it has been reported that infected people, both symptomatic and asymptomatic, have a large amount of virus in the body, which contributes to the spread of infection [5,6]. Therefore, early detection of infected patients is critical for appropriate infection control highlighting the need for sensitive and accurate SARS-CoV-2 detection assays. Currently, reverse transcription-polymerase chain reaction (RT-PCR) is the gold standard for detecting SARS-CoV-2. However, the RNA extraction step, which requires specialized techniques, takes approximately 3 h from sample collection to reporting.
In April 2020, Shimadzu Corporation released the Ampdirect™ 2019-nCoV Detection Kit, which enabled the reporting of results within 1.5 h. The Ampdirect™ 2019-nCoV Detection Kit was designed in a way that eliminates the RNA purification steps, which require 30-60 min, thus saving time from the overall process. However, studies on the clinical performance of this kit are insufficient. Here, we compared the performance of the kit with that of the preexisting SARS-CoV-2 detection assays.

Sample collection
We collected 71 samples (65 nasopharyngeal specimens and 6 sputum specimens), including samples from the same patients collected at different times, from 14 patients confirmed or 18 patients suspected with COVID-19 at the Sapporo Medical University Hospital in February and March 2020. The sample size was relatively small, with 39 overlapping samples. The sputum specimens were collected in sterile Petri dishes or spitz tubes, and the nasopharyngeal specimens were collected using a kit containing a nasopharyngeal nylon-flocked swab and a tube containing universal transport medium (UTM) viral transport medium (Copan Diagnostics, Murrieta, CA, USA). The samples were kept for up to 48 h at 2-8°C until RNA extraction and purification.

RNA extraction and molecular assays
The UTM containing nasopharyngeal specimens were vortexed for 5 s and were then used for RNA extraction and purification. For the sputum specimens, calcium-magnesium-free phosphate-buffered saline was added at a volume of 1-3 times that of the sputum, suspended with a vortex mixer, and the suspension (approximately 1 mL) was transferred to a 1.5 mL sterile microcentrifuge tube. The suspension was centrifuged at 20,000×g at 4°C for 30 min, and RNA was extracted from the supernatant of the suspension. The RNA extraction step of the Ampdirect™ 2019-nCoV Detection Kit (Shimadzu Corporation, Kyoto, Japan; SHIMADZU assay) was performed by adding a treatment solution to the sample. A mixture of the sample (5 µL) and the treatment solution (5 µL) was pretreated at 95°C for 5 min and then added to the PCR solution. For the preexisting SARS-CoV-2 detection assays, the RNA extraction step was performed using the QIAamp Viral RNA Mini kit (QIAGEN, Hilden, Germany), according to the manufacturer's instructions. Briefly, 140 μL of each sample was mixed with 560 μL of buffer AVL/carrier RNA and kept at room temperature for 10 min. After the addition of 560 µL of 99.5% ethanol, the sample was washed in two steps using buffer AW1 followed by buffer AW2. RNA was eluted in a volume of 60 μL of buffer AVE. The extracted RNA was stored at −80°C until further use. LightMix Modular EAV RNA Extraction Control (TIB Molbiol, Berlin, Germany) was added as an internal control for nucleic acid extraction and amplification processes. Table 1 and Supplementary Methods present an overview of the four SARS-CoV-2 detection assays used in this study. All samples were measured in duplicate, and the cycle threshold (Ct) values were averaged. The preexisting SARS-CoV-2 detection assays were performed using the LightMix Modular SARS, Wuhan CoV E-gene, LightMix Modular SARS, and Wuhan CoV N-gene kits (TIB Molbiol, Berlin, Germany; Roche assay) and the Loopamp SARS-CoV-2 detection kit (EIKEN CHEMICAL, Tokyo, Japan; Eiken assay) according to each manufacturer's instruction. The N2 assay of the Japanese National Institute of Infectious Diseases (NIID) assay was based on the Pathogen Detection Manual 2019-nCoV Ver.2.6 [7].

Limit of viral detection
The artificially synthesized RNA containing the N gene target from Shimadzu Corporation was diluted with Tris-EDTA buffer. The limit of detection (LOD) test was performed using the SHIMADZU assay. The diluted artificially synthesized RNA (100, 50, 25, 12.5, 6.3, 3.1, 1.6, and 0.8 copies/reaction) was added before the PCR step, and eight replicates were measured using the CFX96 Touch Deep Well Real-Time PCR Detection System (Bio-Rad Laboratories, Hercules, CA, USA). The detection rate for each serial dilution series was then calculated. Probit analysis was used to estimate LOD.

Definitions and analyses
Currently, there is no gold standard for comparative studies of SARS-CoV-2 infection. In this study, to avoid false positives, a true positive result for SARS-CoV-2 was defined as a positive result obtained using any one of the four SARS-CoV-2 detection assays, whereas a true negative result was defined as negative results obtained using all the SARS-CoV-2 detection assays. The sensitivity, specificity, and kappa (κ) coefficient, which indicated agreement among the assays, were assessed among the SARS-CoV-2 detection assays using the defined positive result as a reference. The number of days from symptom onset to sample collection was stratified and analyzed as less than 10 days and more than 10 days. We also analyzed the time course of the results of SARS-CoV-2 detection assays in six cases that were followed up for four or more days. Furthermore, we classified 70 samples measured using all four SARS-CoV-2 detection assays into four groups according to the number of positive results obtained using the detection assays (1-4). The Ct and threshold time (Tt) values of each detection assay were statistically analyzed.

Statistical analyses
The sensitivity of the assays was compared using McNemar's test. Using probit analysis, the LOD was determined as the lowest detectable dilution (copies/reaction) at which the synthetic RNA was positive with a 95% probability of detection. Ct and Tt values were compared using the Mann-Whitney U test. Statistical significance was set at p<0.05. All statistical analyses were performed using IBM SPSS Statistics version 24 (IBM Corp., Chicago, IL, USA) and JMP Pro15 statistical software (SAS Institute, Cary, NC, USA).

Ethical approval
All experiments involving human subjects complied with all relevant national regulations and institutional policies and were in accordance with the tenets of the Helsinki Declaration (as revised in 2013). This study was approved by the authors' institutional review boards (https://web.sapmed.ac.jp/byoin/rinshokenkyu/; number 312-241). Sapporo Medical University Hospital approved this study and waived the requirement for informed consent based on the retrospective design of the study.

Clinical performance of four SARS-CoV-2 detection assays
Out of the total, 44 (62%) samples were true positive, and 27 (38%) were true negative, as per our definition (Supplementary Table 1 We analyzed the overall sensitivity, specificity, and κ coefficient using 70 samples measured using all four assays because the SHIMADZU assay could not test one sample owing to sample shortage ( Table 2). The overall sensitivity and κ coefficient of the SHIMADZU assay N1 + N2 primer set were 86.0% (95% confidence interval [CI]: 77.9-94.2) and 0.83 (95% CI: 0.69-0.96), respectively, which were the highest among the four SARS-CoV-2 detection assays. Among 18 samples (28%) collected within less than 10 days from the onset of symptoms, high clinical performance was observed in all SARS-CoV-2 detection assays, in particular the SHIMADZU assay N1 + N2 primer set, and all agreed with the defined positive sample (Table 3). Although the clinical performance of all SARS-CoV-2 detection assays decreased in 46 samples (72%) collected after 10 days or more from symptom onset, the SHIMADZU assay N1 + N2 primer set had a sensitivity of 81.3% (95% CI: 70.0-92.5) and a κ coefficient of 0.73 (0.53-0.92), which was the highest among all assays (Supplementary Table 2).
Time course of the results of SARS-CoV-2 detection assays  19, 24, and 25); SARS-CoV-2 could not be detected in some samples after day 12. The Eiken assay could detect the virus up to day 8; however, it was not detected in some samples after day 9.

Detection limit of the SHIMADZU assay
The detection limit test indicated that the SHIMADZU assay could detect up to 1.0 copies/reaction of N1 and up to 4.2 copies/reaction of N2 (Table 4).

Relationship between the number of positive detection assays
Compared with the number of positive detection assays, the median Ct and Tt values were significantly lower in the group in which all assays were positive than in any other group (Figure 2, Supplementary Figure 2, and Supplementary Table 4). In the SHIMADZU assay N1, the NIID assay N2, and the Roche assay E, the median Ct values were less than 35, whereas the number of positive assays tended to decrease when the Ct values were 35 or higher.

Discussion
We compared the clinical performance of the SHIMADZU assay with that of three preexisting molecular tests using 71 samples from 32 patients with confirmed and suspected COVID-19. The sensitivity and κ coefficient of the SHIMADZU assay were higher than those of the NIID, Roche, and Eiken assays, and particularly remarkable in the samples collected within 10 days after symptom onset. It has the potential to be applied to situations in which a highly sensitive diagnosis is required. The newly developed SHIMADZU assay enables performing real-time RT-PCR without RNA purification, thereby substantially shortening the measurement time. This technology, called Ampdirect Technology [8], is also used in preexisting kits, such as those used for norovirus detection. In addition, the SHIMADZU assay can reduce the risk of human error, cost, and human resources required for RNA extraction. The SHIMADZU assay had a sufficient "detection power" of 1,000 copies/mL for N1 and 4,200 copies/mL for N2 in the detection limit. Patients with COVID-19 are normally reported to have more than 10 4 copies/mL of virus load [9]. Therefore, using artificially synthesized RNA, our study indicated that the SHIMADZU assay could adequately detect SARS-CoV-2 infection in patients with COVID-19. We presumed two explanations for the higher "detection power" in the SHIMADZU assay. First, real-time RT-PCR was performed directly on the samples. In preexisting real-time RT-PCR assays, the reaction may be inhibited by various PCR inhibitors derived from living organisms [10]. However, Ampdirect in the pretreatment reagent (sample treatment reagent) of the SHIMADZU assay suppresses these PCR inhibitors [8]. Thus, real-time RT-PCR can be performed directly on biological samples. Second, preexisting SARS-CoV-2 detection assays are affected by purification efficiency during the RNA purification process [11]. Thus, the SHIMADZU assay, which does not require an RNA purification step and is not affected by PCR inhibitors derived from the samples owing to insufficient purification or RNA purification efficiency, could be highly sensitive. The SHIMADZU assay has the potential to detect low virus copy numbers that are difficult to detect using conventional SARS-CoV-2 detection assays. In the future, it may apply to situations in which more sensitive tests are required, such as tests using various specimens (nasal vestibular, nasal discharge, urine, and stool) that have a low exposure risk for examiners or those with close contact with no symptoms who have recently come into contact with patients with COVID-19.
Matsumura et al. [12] compared the clinical performance of SARS-CoV-2 detection assays, including those of the NIID, Roche, and Eiken assays. The sensitivity and κ coefficient of all the assays were higher in their report than those in our study. This is presumably because our study included follow-up samples from the same patients for negative confirmation of their discharge and, therefore, many samples may have low viral copy numbers. The diagnostic performance of the samples collected within 10 days from symptom onset, with the bias considered, was higher in sensitivity and κ coefficient than among the samples collected 10 days after symptom onset.
Additionally, an increase in Ct and Tt values was observed over time after symptom onset. In Japan's current discharge criteria, patients symptomatic for COVID-19 can be discharged 10 days after symptom onset with symptom improvement for at least 3 days [13]. Thus, negative PCR confirmation is not required. However, the sample for diagnosis is not always collected in the early stages of symptom onset. The sample may have a low viral load because patients asymptomatic for COVID-19 would have a lower viral load than those who are symptomatic for COVID-19 [14]. Therefore, it is desirable to use a more sensitive SARS-CoV-2 detection assay suitable for each facility.
The Ct and Tt values were significantly lower in the group in which all four assays were positive than in any other group. This finding indicated that the virus copy number would significantly affect the performance of the assays and cause discrepancies in their results. In particular, we observed that the real-time PCR assays, not only the SHIMADZU assay but also the NIID and Roche assays, showed high performance for the sample with a high virus copy number at the level of a Ct value less than 35. Although it is possible that the eight samples that tested positive using only one assay are false positives [15], five of these were sequence-analyzed and showed the same homology as that of SARS-CoV-2, which could indicate true positives. All the remaining three samples were positive only in the SHIMADZU assay. The N1 and N2 primer-probe sets of the SHIMADZU assay have the same nucleotide sequences as the primer-probe set recommended by the Centers for Disease Control and Prevention (CDC) [16]. In a previous report, the CDC primer-probe set demonstrated no cross-reactivity with other respiratory virus infections [17]. Therefore, false positives may be rare in the SHIMADZU assay; however, further research is needed to confirm this.
In the time course of patients with COVID-19, some patients (Patient No. 16, 19, and 21) retested positive after negative results. We assumed that the sample collection procedure may have affected the results. The Ct values of the samples retested positive were 35 or higher. Thus, it is necessary to carefully interpret the results when the copy number is low. Patient No. 18 remained positive according to all four SARS-CoV-2 detection assays, even though 14 days had passed since symptom onset. The patient had severe pneumonia and required oxygen administration on day 10 from symptom onset, mechanical ventilation support on day 12, and venovenous extracorporeal membrane oxygenation on day 14. It has been reported that invasive viral pneumonia is associated with prolonged SARS-CoV-2 RNA shedding [18]. Patient No. 18 was also considered to continue to be positive by all four SARS-CoV-2 detection assays owing to persistent viral excretion; however, the virus activity was unknown.
This study has several limitations. First, a relatively small sample size was evaluated. Furthermore, the number of samples collected within 10 days from symptom onset was too small to evaluate clinical performance for diagnostic purposes. However, the SHIMADZU assay had high sensitivity and concordance even for samples with low viral copy numbers, and the performance of the test itself could be evaluated. Second, there may be variations in the sample collection method because different staff members collected the samples. Third, we used our definition of the positivity of the sample in this study, and not all samples were evaluated using sequence analysis. Therefore, we could not evaluate the specificity of the test for all the  . (.-.) a % CI could not be calculated. CI, confidence interval: The detection limit by probit analysis was determined as the lowest detectable dilution (copies/reaction) at which the synthetic RNA was positive with a % probability of detection. assays. Fourth, the SHIMADZU assay does not include the RNA purification step; the use of synthesized RNA results in a low LOD in comparison to that with the use of the real virus. Fifth, we were unable to add any direct PCR assays other than the SHIMADZU assay for comparison. For the SARS-CoV-2 detection assay, we used instruments and kits available in our hospital during the study period. Therefore, the comparison between the SHIMADZU assay and direct PCR assays should be examined in the future.

Conclusions
The SHIMADZU assay would be highly useful as a molecular assay for SARS-CoV-2 detection because of its simplicity, speed, and clinical performance compared to the preexisting assays. This would have the potential for future applications in situations where a highly sensitive diagnosis is required. However, caution must be exercised with false negatives, especially in cases of low virus copy numbers, even with molecular diagnostic tools, including the SHIMADZU assay. Considering infection opportunities, clinical symptoms, and the timing of sample collection, the results should be interpreted comprehensively.