Multicenter Validation of a Machine Learning Algorithm for Diagnosing Pediatric Patients with Multisystem Inflammatory Syndrome and Kawasaki Disease
Abstract
Background
Multisystem inflammatory syndrome in children (MIS-C) is a novel disease identified during the COVID-19 pandemic characterized by systemic inflammation following SARS-CoV-2 infection. Delays in diagnosing MIS-C may lead to more severe disease with cardiac dysfunction or death. Most pediatric patients recover fully with anti-inflammatory treatments, but early detection of MIS-C remains a challenge given its clinical similarities to Kawasaki disease (KD) and other acute childhood illnesses.
Methods
We developed KIDMATCH (<underline>K</underline>awasak<underline>I</underline><underline>D</underline>isease vs <underline>M</underline>ultisystem Infl<underline>A</underline>mma<underline>T</underline>ory syndrome in <underline>CH</underline>ildren), a deep learning algorithm for screening patients for MIS-C, KD, or other febrile illness, using age, the five classical clinical KD signs, and 17 laboratory measurements prospectively collected within 24 hours of admission to the emergency department from 1448 patients diagnosed with KD or other febrile illness between January 1, 2009 and December 31, 2019 at Rady Children’s Hospital. For MIS-C patients, the same data was collected from 131 patients between May 14, 2020 to June 18, 2021 at Rady Children’s Hospital, Connecticut Children’s Hospital, and Children’s Hospital Los Angeles. We trained a two-stage model consisting of feedforward neural networks to distinguish between MIS-C and non MIS-C patients and then KD and other febrile illness. After internally validating the algorithm using 10-fold cross validation, we incorporated a conformal prediction framework to tag patients with erroneous data or distribution shifts, enhancing the model generalizability and confidence by flagging unfamiliar cases as indeterminate instead of making spurious predictions. We externally validated KIDMATCH on 175 MIS-C patients from 16 hospitals across the United States.
Findings
KIDMATCH achieved a high median area under the curve in the 10-fold cross validation of 0.988 [IQR: 0.98-0.993] in the first stage and 0.96 [IQR: 0.956-0.972] in the second stage using thresholds set at 95% sensitivity to detect positive MIS-C and KD cases respectively during training. External validation of KIDMATCH on MIS-C patients correctly classified 76/83 (2 rejected) patients from the CHARMS consortium, 47/50 (1 rejected) patients from Boston Children’s Hospital, and 36/42 (2 rejected) patients from Children’s National Hospital.
Interpretation
KIDMATCH has the potential to aid frontline clinicians with distinguishing between MIS-C, KD, and similar febrile illnesses in a timely manner to allow prompt treatment and prevent severe complications.
Funding
Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Heart, Lung, and Blood Institute, Patient-Centered Outcomes Research Institute, National Library of Medicine
Related articles
Related articles are currently not available for this article.