The novel coronavirus disease 19 (COVID-19), caused by SARS-CoV-2, spreads rapidly across the world. The exponential increase in the number of cases has resulted in overcrowding of emergency departments (ED). Detection of SARS-CoV-2 is based on an RT-PCR of nasopharyngeal swab material. However, RT-PCR testing is time-consuming and many hospitals deal with a shortage of testing materials. Therefore, we aimed to develop an algorithm to rapidly evaluate an individual’s risk of SARS-CoV-2 infection at the ED.
In this multicenter retrospective study, routine laboratory parameters (C-reactive protein, lactate dehydrogenase, ferritin, absolute neutrophil and lymphocyte counts), demographic data and the chest X-ray/CT result from 967 patients entering the ED with respiratory symptoms were collected. Using these parameters, an easy-to-use point-based algorithm, called the corona-score, was developed to discriminate between patients that tested positive for SARS-CoV-2 by RT-PCR and those testing negative. Computational sampling was used to optimize the corona-score. Validation of the model was performed using data from 592 patients.
The corona-score model yielded an area under the receiver operating characteristic curve of 0.91 in the validation population. Patients testing negative for SARS-CoV-2 showed a median corona-score of 3 vs. 11 (scale 0–14) in patients testing positive for SARS-CoV-2 (p<0.001). Using cut-off values of 4 and 11 the model has a sensitivity and specificity of 96 and 95%, respectively.
The corona-score effectively predicts SARS-CoV-2 RT-PCR outcome based on routine parameters. This algorithm provides the means for medical professionals to rapidly evaluate SARS-CoV-2 infection status of patients presenting at the ED with respiratory symptoms.
In December 2019 the novel coronavirus disease 2019 (COVID-19), caused by SARS-CoV-2, spread rapidly from its origin in Wuhan, China . Symptoms can range from mild, common cold-like, to life threatening with intensive care unit admission and extensive mechanical ventilation , . On February 27th 2020 the first patient was identified in the Netherlands, and thousands of new patients were diagnosed within the first month.
The subsequent exponential increase in prevalence has resulted in overcrowding of emergency departments (ED) and has led to a shortage of isolation rooms . For correct triaging of patients diagnostic testing is of key importance. The leading standard test for detecting SARS-CoV-2 is an RT-PCR of nasopharyngeal swab material . However, RT-PCR testing is time-consuming and shortage of testing materials and capacity imposes a serious threat .
Doctors at the ED are required to assess the probability of SARS-CoV-2 infection in each patient entering the ED. Laboratory medicine provides an important contribution towards the diagnosis of diseases, including viral infections . Therefore, to accelerate the triage process at the ED, we integrated routine demographic, laboratory and imaging data of patients presenting at the ED with COVID-19-like symptoms to develop a point-based algorithm. The inclusion of parameters into the model was based on observed trends in differences between positive and negative patients with subsequent statistical analyses, as well as previously published data , , . This algorithm can assess whether a person, presenting at the ED with respiratory symptoms, is likely to have COVID-19. In case of a shortage of testing capacity, adoption of this algorithm could reduce the number of patients for whom RT-PCR testing is required. Moreover, implementation of the corona-score enables rapid decision making at the ED, lowering pressure on isolation rooms.
Materials and methods
In this retrospective multicenter study, 375 patients from three different hospitals presenting at the ED with respiratory symptoms, or suspected COVID-19 infection because of e.g. gastro-intestinal complaints (1–2% of this cohort), and subsequent SARS-CoV-2 RT-PCR testing were included (Figure 1 and Table 2). Patients from other departments and patients without any respiratory symptoms or suspicion of COVID-19 were excluded. Based on the same inclusion criteria, an independent cohort of 592 patients from four hospitals was used to validate the model (Figure 1 and Table 2). For the validation population, patients with missing values or hemolytic samples were excluded (n=97).
|CXR||No infiltrate||Unilateral infiltrate||Bilateral infiltrate|
CRP, C-reactive protein; LDH, lactate dehydrogenase; ALC, absolute lymphocyte count; ANC, absolute neutrophil count; CXR, chest X-ray.
For clinical chemistry analyses and RT-PCR, venous blood and pharyngeal plus nasal swab specimens, respectively, were collected. All analyses were performed at initial presentation at the ED. Clinical chemistry parameters (C-reactive protein (CRP), ferritin, lactate dehydrogenase (LDH), absolute lymphocyte and neutrophil counts (ALC and ANC)) were obtained on routine analyzers from Siemens (Jeroen Bosch Hospital and the [immuno-]chemistry of Bernhoven Hospital), Sysmex (Elisabeth TweeSteden Hospital and the hematology of Amphia Hospital), Roche (Elisabeth TweeSteden Hospital and the [immuno-]chemistry of Amphia Hospital) and Abbott (hematology of Bernhoven Hospital). SARS-CoV-2 RT-PCR testing at Amphia Hospital and Elizabeth TweeSteden Hospital was performed using tests from Microvida Laboratory (the Netherlands; ), whereas Jeroen Bosch Hospital and Bernhoven Hospital used in-house developed tests. Chest X-rays (CXR) and chest CT-scans were imaged using Siemens, GE Healthcare and Philips equipment. External quality assessment (EQA) in commutable materials by Dutch Foundation for Quality Assessment in Medical Laboratories (SKML) demonstrated that ferritin measured on Roche analyzers is on average 20% higher than on Siemens analyzers. For building the model and calculating corona-scores ferritins measured on Siemens analyzers were therefore multiplied by 1.2. All other measurands in the scoring system had no significant inter-method differences for Roche, Siemens and Sysmex in the particular SKML EQA schemes.
A scoring-based algorithm was developed using laboratory measurands (CRP, ALC, ANC, LDH and ferritin), age, sex and CXR/CT as input. Scores were assigned to each parameter according to Table 1 (or see www.corona-score.nvkc.nl for more information). The corona-score is obtained by the summation of the score for each parameter. The final score is truncated from a minimum of 0 to a maximum of 14 points. Cut-off points and weights of demographic, laboratory and imaging parameters were computationally sampled using Python (v3.7.0, Python Software Foundation, USA) to optimize for a maximum area under the receiver operating characteristic (AUROC) curve. When values were missing in the data of the model population (n=3 for ALC and ANC, n=31 for LDH and n=4 for ferritin) the median of the total population was used.
|Model population||COVID-19 negative (n=99)||COVID-19 positive (n=276)||p-Value|
|Mean ± SD||Median (25th–75th percentile)||Mean||Median (25th–75th percentile)|
|Age, years||62 ± 16||64 (52–74)||70 ± 12||72 (61–79)||<0.001a|
|CRP, mg/L||84 ± 97||47 (8–138)||106 ± 72||98 (46–153)||<0.001a|
|LDH, U/L||251 ± 111||233 (186–270)||391 ± 254||346 (270–449)||<0.001a|
|Ferritin, µg/L||617 ± 1457||222 (111–517)||933 ± 960||633 (363–1291)||<0.001a|
|Lymphocytes, ×109/L||1.5 ± 1.1||1.2 (0.8–1.7)||1.0 ± 0.8||0.90 (0.6–1.2)||<0.001a|
|Neutrophils, ×109/L||9.5 ± 6.9||7.7 (5.1–11.4)||5.7 ± 3.0||5.20 (3.5–7.1)||<0.001a|
|Corona-score, 0–14||3.9 ± 2.9||4.0 (2.0–6.0)||10.5 ± 2.8||11.0 (9.0–13.0)||<0.001a|
|Hospitalc||JBZ (69); BHZ (20); ETZ (10); AMP (0)||JBZ (107); BHZ (136); ETZ (43); AMP (0)|
|Validation population||COVID-19 negative (n = 199)||COVID-19 positive (n = 393)||p-Value|
|Mean ± SD||Median (25th–75th percentile)||Mean ± SD||Median (25th–75th percentile)|
|Age, years||63 ± 17||67 (51–76)||69 ± 12||71 (61–77)||0.001a|
|CRP, mg/L||78 ± 97||32 (9–127)||107 ± 70||95 (54–147)||<0.001a|
|LDH, U/L||279 ± 242||228 (190–296)||401 ± 155||371 (292–464)||<0.001a|
|Ferritin, µg/L||419 ± 552||211 (91–529)||1195 ± 1288||796 (394–1431)||<0.001a|
|Lymphocytes, ×109/L||1.5 ± 1.0||1.3 (0.7–2.0)||1.4 ± 4.6||0.9 (0.7–1.3)||<0.001a|
|Neutrophils, ×109/L||8.5 ± 5.3||7.0 (4.9–10.9)||6.0 ± 3.0||5.5 (3.8–7.4)||<0.001a|
|Corona-score, 0–14||3.9 ± 3.4||3.0 (1.0–6.0)||10.5 ± 2.9||11.0 (9.0–13.0)||<0.001a|
|Hospitalc||JBZ (66); BHZ (15); ETZ (80); AMP (38)||JBZ (136); BHZ (20); ETZ (139); AMP (98)|
aMann–Whitney U test (α = 0.05). bχ2 test. cJBZ, Jeroen Bosch Hospital; BHZ, Bernhoven Hospital; ETZ, Elisabeth-TweeSteden Hospital; AMP, Amphia Hospital.
Data were analyzed using the Excel 2010 (Microsoft Corporation, USA) plugin ‘Analyse-it v5.11’ (Analyse-it Software, Ltd, UK) and SPSS statistics v22 (IBM, USA). Continuous variables were tested for normal distribution using a Kolmogorov-Smirnov test. In case of non-normal distribution, a Mann-Whitney U test was performed to compare the medians. Categorical variables were compared by a χ2-test. A p-value <0.05 was considered statistically significant.
Using a cohort of 375 ED patients with respiratory symptoms a point-based algorithm was created and subsequently validated using a separate independent cohort of 592 patients (Table 2). At the time of presentation at the ED the parameters sex, age, CRP, ferritin, LDH, ALC, ANC and CXR were significantly different between the COVID-19 positive and negative patients (Figure 2 and Table 2). Together, these parameters were used to develop an algorithm, named ‘corona-score’. Inclusion of albumin, procalcitonin or clinical parameters such as fever, cough and dyspnea did not sufficiently improve the performance of the algorithm, possibly partially due to method differences among contributing laboratories and suboptimal anamnesis and/or registration thereof (data not shown). The corona-score resulted in a model with an AUROC of 0.94 (Figure 3A, 95% CI 0.91–0.96). Patients with a negative RT-PCR test had a median of 4 compared to a median of 11 for SARS-CoV-2 positive patients (Figure 3B and Table 2). The corona-score algorithm was validated with data from 592 patients, yielding an AUROC of 0.91 (Figure 3C, 95% CI 0.89–0.94). In the validation population SARS-CoV-2 negative patients had a median of 3 versus a median of 11 for SARS-CoV-2 positive patients (Figure 3D and Table 2).
By using different cut-off values the desired sensitivity and specificity for the test can be found (Table 3). For example, using corona-score cut-offs of 4 (96% sensitivity) and 11 (95% specificity) at a 70% prevalence (during the regional COVID-19 pandemic peak), this model showed negative and positive predictive values of 88 and 96% (Figure 3E). The total false rate given these conditions is 4% (Figure 3E).
|Corona-score cut-off value||Sensitivity (95% CI)||Specificity (95% CI)||True | false negative|
|2||98% (0.96–0.99)||42% (0.35–0.49)||83 | 7|
|3||98% (0.95–0.99)||53% (0.46–0.60)||105 | 10|
|4||96% (0.94–0.98)||63% (0.56–0.70)||125 | 15|
|5||94% (0.91–0.96)||72% (0.66–0.78)||144 | 25|
|9||78% (0.73–0.82)||89% (0.84–0.93)||305 | 22|
|10||68% (0.63–0.72)||92% (0.87–0.95)||267 | 17|
|11||56% (0.51–0.61)||95% (0.90–0.97)||219 | 11|
|12||45% (0.40–0.50)||97% (0.94–0.99)||177 | 6|
RT-PCR testing for SARS-CoV-2 is hampered by significant numbers of false negatives as the sensitivity of RT-PCR is estimated at approximately 70–90% . Indeed, many doctors request multiple COVID-19 tests when the RT-PCR result does not match the clinical presentation of the patient. Patients from the validation population of the JBZ hospital that showed positivity for SARS-CoV-2 after repeated RT-PCR testing (n=13) had an initial median corona-score of 12, while patients that remained negative (n=12) had an initial median corona-score of 4 (Figure 3F). This indicates that the corona-score may be able to distinguish between true and false negatives, but prospective studies are needed to confirm this.
Using a cohort of 967 patients we developed and validated a point-based algorithm to predict the likelihood of SARS-CoV-2 infection in patients presenting at the ED with respiratory symptoms. Validation of the model resulted in an AUROC of 0.91 with a 96% sensitivity and 95% specificity, using corona-score cut-offs of 4 and 11. Importantly, cut-offs can be selected to optimize the performance of the score depending on hospital needs and regional prevalence. Such an algorithm can be used to, (1) accelerate determination of isolation needs and (2) reduce RT-PCR testing: a reduction of about 60% can be achieved if cut-offs of 4 and 11, yielding 125 true negative and 219 true positive patients in the validation cohort of 592 patients, are used.
Our algorithm is optimized to predict the outcome of the SARS-CoV-2 RT-PCR test, which has limited (70–90%) sensitivity . Inclusion of patients having false negative RT-PCR tests into the study population distorts the dataset leading to an underestimation of the performance of the algorithm. Interestingly, by retrospectively investigating twenty-five patients that received multiple COVID-19 tests our algorithm could predict which patients were initially false negatives. Therefore, the sensitivity of the corona-score appears to exceed the sensitivity of the SARS-CoV-2 RT-PCR.
In a minority of cases, our model produces a corona-score of 0–5 in patients that tested positive for SARS-CoV-2 by RT-PCR. There are two common underlying reasons for this phenomenon. Firstly, the corona-score performed poorly in patients with a gastro-intestinal presentation of COVID-19 (only 1–2% of patients), but without respiratory symptoms. Therefore, this algorithm should only be used for patients at the ED with respiratory symptoms or suspected COVID-19 infection. Secondly, the corona-score performed unsatisfactory in a few patients that only have mild clinical symptoms, and therefore do not have large alterations in their laboratory parameters. On the other hand, some negatively-tested patients received a high corona-score. This could be due to false-negative RT-PCR testing or possibly other viral infections. Interestingly, four patients that were positive for influenza and negative for SARS-CoV-2 had a low corona-score (between 2 and 6). During this COVID-19 pandemic, the prevalence of other respiratory viruses appears very low; hence, the discriminative potential of the corona-score in patients infected by such viruses could not be systematically established. Notably, in case of any viral outbreak, a similar modelling approach could be considered to develop an algorithm as described here.
It is important to point out that the corona-score may only be used for the population for which it has been validated. Therefore, the corona-score should not be used for patients presenting at the ED without symptoms that may relate to COVID-19-infection or for patients presenting at other locations, such as the general practitioner or at nursing homes. Moreover, our study population did not include any pediatric patients, and therefore, the use of this score in the pediatric population has not been validated. The absence of pediatric patients presenting at the ED could also indicate that pediatric patients do not develop a sufficiently severe disease that requires an ED visit. Lastly, the model has been developed as a diagnostic tool and has no established prognostic value.
The four laboratories involved in this study deploy different instruments from the major in-vitro diagnostic device providers. Most measurands that were included in the algorithm have an identical metrological traceability and hence comparable results in the commutable EQA scheme of the SKML . However, there is no reference method for ferritin . The different calibrations lead to approximately 20% difference in ferritin results between the methods employed by the laboratories in this study. Therefore, a 1.2 harmonization factor was applied to the ferritin values obtained from Siemens instruments, before calculating corona-scores, correcting the lack of standardization. Generally, methodological harmonization between laboratories should be encouraged for better comparison of laboratory results .
To our knowledge, our algorithm is the first available validated tool to rapidly evaluate COVID-19 status in ED patients with respiratory symptoms based on routine laboratory tests. The model has already been implemented at the ED of several hospitals in the Netherlands. Implementation of this algorithm will accelerate the triage of patients and reduce the number of RT-PCR tests required.
We thank Rob Verheyen (University College London), Thomas de Bel (Radboudumc Nijmegen), Marleen Weijland, Daniëlle Verboogen, Ph.D. (Elizabeth TweeSteden Hospital), Kinki Jim, MD (Jeroen Bosch Hospital), Dr. Rein Hoedemakers (Jeroen Bosch Hospital) and Jasmijn van Balveren, MD (Jeroen Bosch Hospital) for their valuable technical help and scientific input.
Research funding: None declared.
Author contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following four requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: All authors have read the journal’s policy on disclosure of potential conflicts of interest and have none to declare.
Ethical approval: The study was conducted according to the declaration of Helsinki, Guidelines for Good Clinical Practice and the Dutch Medical Research Involving Human Subjects Act. The execution of this retrospective observational study of patient records was approved by the local Ethics Committee and the local Review Board of the Jeroen Bosch Hospital. This study had no effect on the behaviour of patients or medical decision-making.
Data availability: The data that support the findings of this study are available from the corresponding author upon reasonable request. More information can be obtained at www.corona-score.nvkc.nl.
1. Li, Q, Guan, X, Wu, P, Wang, X, Zhou, L, Tong, Y, et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N Engl J Med 2020;382:1199–207. https://doi.org/10.1056/NEJMOa2001316.Search in Google Scholar
2. Chen, N, Zhou, M, Dong, X, Qu, J, Gong, F, Han, Y, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet 2020;395:507–13. https://doi.org/10.1016/S0140-6736(20)30211-7.Search in Google Scholar
3. Guan, WJ, Ni, ZY, Hu, Y, Liang, WH, Ou, CQ, He, JX, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med 2020;395:507–13. https://doi.org/10.1056/NEJMoa2002032.Search in Google Scholar
4. Zhang, J, Zhou, L, Yang, Y, Peng, W, Wang, W, Chen, X. Therapeutic and triage strategies for 2019 novel coronavirus disease in fever clinics. Lancet Respir Med 2020;8:e11–e2. https://doi.org/10.1016/S2213-2600(20)30071-0.Search in Google Scholar
5. Chan, JF, Yuan, S, Kok, KH, To, KK, Chu, H, Yang, J, et al. A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster. Lancet 2020;395:514–23. https://doi.org/10.1016/S0140-6736(20)30154-9.Search in Google Scholar
6. Cheng, MP, Papenburg, J, Desjardins, M, Kanjilal, S, Quach, C, Libman, M, et al. Diagnostic testing for severe acute respiratory syndrome-related coronavirus-2: a narrative review. Ann Intern Med 2020;172:726–34. https://doi.org/10.7326/m20-1301.Search in Google Scholar
7. Lippi, G, Plebani, M. The critical role of laboratory medicine during coronavirus disease 2019 (COVID-19) and other viral outbreaks. Clin Chem Lab Med 2020. https://doi.org/10.1515/cclm-2020-0240.Search in Google Scholar
8. Liu, T, Zhang, J, Yang, Y, Ma, H, Li, Z, Zhang, J, et al. The potential role of IL-6 in monitoring severe case of coronavirus disease 2019. medRxiv 2020:2020. https://doi.org/10.1101/2020.03.01.20029769.Search in Google Scholar
9. Zhou, F, Yu, T, Du, R, Fan, G, Liu, Y, Liu, Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020;395:1054–62. https://doi.org/10.1016/S0140-6736(20)30566-3.Search in Google Scholar
10. Kluytmans, M, Buiting, A, Pas, S, Bentvelsen, R, van den Bijllaardt, W, van Oudheusden, A, et al. SARS-CoV-2 infection in 86 healthcare workers in two Dutch hospitals in march 2020. medRxiv 2020:2020. https://doi.org/10.1101/2020.03.23.20041913.Search in Google Scholar
12. Tate, JR, Johnson, R, Barth, J, Panteghini, M. Harmonization of laboratory testing - current achievements and future strategies. Clin Chim Acta 2014;432:4–7. https://doi.org/10.1016/j.cca.2013.08.021.Search in Google Scholar
13. Hoofnagle, AN. Harmonization of blood-based indicators of iron status: making the hard work matter. Am J Clin Nutr 2017;106:1615S–9S. https://doi.org/10.3945/ajcn.117.155895.Search in Google Scholar
14. Miller, WG, Jones, GR, Horowitz, GL, Weykamp, C. Proficiency testing/external quality assessment: current challenges and future directions. Clin Chem 2011;57:1670–80. https://doi.org/10.1373/clinchem.2011.168641.Search in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston