Predictive accuracy of computer-aided versions of the on-admission National Early Warning Score in estimating the risk of COVID-19 for unplanned admission to hospital: a retrospective development and validation study
Abstract
Objectives
To consider the potential of the National Early Warning Score (NEWS2) for COVID-19 risk prediction on unplanned admission to hospital.
Design
Logistic regression model development and validation study using a cohort of unplanned emergency medical admission to hospital.
Setting
York Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset.
Participants
Unplanned adult medical admissions discharged over 3 months (11 March 2020 to 13 June 2020) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within ±24 hours of admission. We used logistic regression modelling to predict the risk of COVID-19 using NEWS2 (Model M0’) versus enhanced cNEWS models which included age + sex (model M1’) + subcomponents (including diastolic blood pressure + oxygen flow rate + oxygen scale) of NEWS2 (model M2’). The ICD-10 code ‘U071’ was used to identify COVID-19 admissions. Model performance was evaluated according to discrimination (c statistic), calibration (graphically), and clinical usefulness at NEWS2 ≥5.
Results
The prevalence of COVID-19 was higher in SH (11.0%=277/2520) than YH (8.7%=343/3924) with higher index NEWS2 (3.2 vs 2.8) but similar in-hospital mortality (8.4% vs 8.2%). The c-statistics for predicting COVID-19 for cNEWS models (M1’,M2’) was substantially better than NEWS2 alone (M0’) in development (M2’: 0.78 (95%CI 0.75-0.80) vs M0’ 0.71 (95%CI 0.68-0.74)) and validation datasets (M2’: 0.72 (95%CI 0.69-0.75) vs M0’ 0.65 (95%CI 0.61-0.68)). Model M2’ had better calibration than Model M0’ with improved sensitivity (M2’: 57% (95%CI 51%-63%) vs M0’ 44% (95%CI 38%-50%)) and similar specificity (M2’: 76% (95%CI 74%-78%) vs M0’ 75% (95%CI 73%-77%)) for validation dataset at NEWS2≥5.
Conclusions
Model M2’ is reasonably accurate for predicting the on-admission risk of COVID-19. It may be clinically useful for an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions.
Related articles
Related articles are currently not available for this article.