Observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19

Objectives

To assess inter-observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19.

Methods

From 16th to 24th March 2020, 241 consecutive patients addressed to hospital for COVID-19 suspicion had both chest CT and SARS-CoV-2 RT-PCR. Eight observers (2 thoracic and 2 general senior radiologists, 2 junior radiologists and 2 emergency physicians) retrospectively categorized each CT into one out of 3 categories (evocative, compatible for COVID-19 pneumonia, and not evocative or normal). Observer agreement for categorization between all readers and pairs of readers with similar experience was evaluated with the Kappa coefficient. The results of a consensus categorization were correlated to RT-PCR.

Results

Observer agreement across the 3 categories was good between all readers (κ value 0.68 95%CI 0.67-0.70) and good to very good between pairs of readers (0.64-0.85). It was very good (κ 0.81 95%CI 0.79-0.83), fair (κ 0.32 95%CI 0.29-0.34) and good (κ 0.74 95%CI 0.71-0.76) for the categories evocative, compatible and not evocative or normal, respectively. RT-PCR was positive in 97%, 50% and 27% of cases classified in the respective categories. Observer agreement was lower (p=0.045) and RT-PCR positive cases were less frequently categorized evocative in presence of an underlying pulmonary disease (p<0.001).

Conclusion

Inter-observer agreement for chest CT reporting using categorization of findings is good in patients suspected of COVID-19. Among patients considered for hospitalization in an epidemic context, CT categorized evocative is highly predictive of COVID-19, whereas the predictive value of CT decreases between the categories compatible and not evocative.

Key results

• Inter-observer agreement for chest CT reporting into categories is good in patients suspected of COVID-19

• Chest CT can participate in estimating the likelihood of COVID-19 in patients presenting to hospital during the outbreak, CT categorized «evocative» being highly predictive of the disease whereas up to a quarter of patients with CT «not evocative» had a positive RT-PCR in our study.

• Observer agreement is lower and CTs of positive RT-PCR cases less frequently “evocative” in presence of an underlying pulmonary disease