Multireader Evaluation of Radiologist Performance for COVID-19 Detection on Emergency Department Chest Radiographs
Abstract
Background
Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CXR at the time of patient presentation in the Emergency Department (ED).
Materials and Methods
We extracted RT-PCR results, clinical history, and CXRs of all patients from a single institution between March and June 2020. 984 RT-PCR positive and 1043 RT-PCR negative radiographs were reviewed by 10 emergency radiologists from 4 academic centers. 100 cases were read by all radiologists and 1927 cases by 2 radiologists. Each radiologist chose the single best label per case: Normal, COVID-19, Other – Infectious, Other – Noninfectious, Non-diagnostic, and Endotracheal Tube. Cases labeled with endotracheal tube or non-diagnostic were excluded. Remaining cases were analyzed for label distribution, clinical history, and inter-reader agreement.
Results
1727 radiographs (732 RT-PCR positive, 995 RT-PCR negative) were included from 1,594 patients (51.2% male, 48.8% female, age 59 ± 19 years). For 89 cases read by all readers, there was poor agreement for RT-PCR positive (Fleiss Score 0.36) and negative (Fleiss Score 0.46) exams. Agreement between two readers on 1,638 cases was 54.2% (373/688) for RT-PCR positive cases and 71.4% (679/950) for negative cases. Agreement was highest for RT-PCR negative cases labeled as Normal (50.4%, n= 479). Reader performance did not improve with clinical history or time between CXR and RT-PCR result.
Conclusion
At the time of presentation to the emergency department, emergency radiologist performance is non-specific for diagnosing COVID-19.
Related articles
Related articles are currently not available for this article.