A preliminary study of commercially available general-purpose chest radiography artificial intelligence-based software for detecting airspace opacity lesions in COVID-19 patients
Abstract
Purpose
To validate commercially available general-purpose artificial intelligence (AI)-based software for detecting airspace opacity in chest radiographs (CXRs) of COVID-19 patients.
Materials and Methods
We used the ieee8023-covid-chestxray-dataset to validate commercial AI software capable of detecting “Nodule/Mass” and “Airspace opacity” as regions of interest with probability scores. From this dataset, we excluded computed tomography images and CXR images taken using an anteroposterior spine view and analyzed CXR images tagged with “Pneumonia/Viral/COVID-19” and “no findings.” A radiologist then reviewed the images and rated them on a 3-point opacity score for the presence of airspace opacity. The maximum probability score of airspace opacity for each image was calculated using this software. The difference in each maximum probability for each opacity score was evaluated using Wilcoxon’s rank sum test. The threshold of the probability score was determined by receiver operator characteristic curve analysis for the presence or absence of COVID-19, and the true positive rate (TPR) and false positive rate (FPR) were determined for the individual and overall opacity scores.
Results
Images from 342 patients with COVID-19 and 15 normal images were included. Opacity scores of 1, 2, and 3 were observed in 44, 70, and 243 images, respectively, of which 33 (75%), 66 (94.2%), and 243 (100%), respectively, were from COVID-19 patients. The overall TPR and FPR were 0.82 and 0.13, respectively, at an area under the curve of 0.88 and a threshold of 0.06, while the FPR for opacity score 1 was 0.18 and the TPR for score 3 was 0.97.
Conclusion
Using a public database containing CXR images of COVID-19 patients, commercial AI software was shown to be able to detect airspace opacity in severe pneumonia.
Summary
Commercially available AI software was capable of detecting airspace opacity in CXR images of COVID-19 patients in a public database.
Related articles
Related articles are currently not available for this article.