Accuracy of automated computer aided-risk scoring systems to estimate the risk of COVID-19 and in-hospital mortality: a retrospective cohort study

Objectives

Although a set of computer-aided risk scoring systems (CARSS), that use the National Early Warning Score and routine blood tests results, have been validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital, little is known about their performance for COVID-19 patients. We compare the performance of CARSS in unplanned admissions with COVID-19 during the first phase of the pandemic.

Design

a retrospective cross-sectional study

Setting

Two acute hospitals (Scarborough and York) are combined into a single dataset and analysed collectively.

Participants

Adult (>=18 years) non-elective admissions discharged between 11-March-2020 to 13-June-2020 with an index NEWS electronically recorded within ±24 hours. We assessed the performance of all four risk score (for sepsis: CARS_N, CARS_NB; for mortality: CARM_N, CARM_NB) according to discrimination (c-statistic) and calibration (graphically) in predicting the risk of COVID-19 and in-hospital mortality.

Results

The risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89). For predicting in-hospital mortality, the CARM_NB model had the highest discrimination 0.84 (0.82 to 0.75) and calibration slope 0.89 (0.81 to 0.98).

Conclusions

Two of the computer-aided risk scores (CARS_N and CARM_NB) are reasonably accurate for predicting the risk of COVID-19 and in-hospital mortality, respectively. They may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions because they are automated and require no additional data collection.

Article Summary

• In this study, we found that two of the automated computer-aided risk scores are reasonably accurate for predicting the risk of COVID-19 and in-hospital mortality, respectively.

• They may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions because they are automated and require no additional data collection.

• Although we focused on in-hospital mortality (because we aimed to aid clinical decision making in the hospital), the impact of this selection bias needs to be assessed by capturing out-of-hospital mortality by linking death certification data and hospital data.

• We identified COVID-19 based on ICD-10 code ‘U071’ which was determined by COVID-19 swab test results (hospital or community) and clinical judgment and so our findings are constrained by the accuracy of these methods