Walsh SLF, et al. - The utility of a deep learning algorithm for providing automated classification of fibrotic lung disease on high-resolution computed tomography (CT) was assessed according to criteria specified in two international diagnostic guideline statements: the 2011 American Thoracic Society (ATS)/European Respiratory Society (ERS)/Japanese Respiratory Society (JRS)/Latin American Thoracic Association (ALAT) guidelines for diagnosis and management of idiopathic pulmonary fibrosis and the Fleischner Society diagnostic criteria for idiopathic pulmonary fibrosis. Low-cost, reproducible, near-instantaneous classification of fibrotic lung disease with human-level accuracy could be provided with high-resolution CT evaluation by a deep learning algorithm. Findings suggested these methods may be beneficial to centers at which thoracic imaging expertise is uncommon. They also noted the benefit of these methods for stratification of patients in clinical trials.

Methods

• Researchers conducted a case-cohort study, for algorithm development and testing, a database in which 1,157 anonymized high-resolution CT scans showed the evidence of diffuse fibrotic lung disease was generated from two institutions.
• The scans were separated into three non-overlapping cohorts (training set, n=929; validation set, n=89; and test set A, n=139) and classified them using 2011 ATS/ERS/JRS/ALAT idiopathic pulmonary fibrosis diagnostic guidelines.
• For each scan, they segmented the lungs and resampled them to create a maximum of 500 unique four slice combinations, which we converted into image montages.
• A total of 420,096 unique montages for algorithm training were in the final training dataset.
• They assessed the algorithm performance, reported as accuracy, prognostic accuracy, and weighted κ coefficient (κw) of interobserver agreement, on test set A and a cohort of 150 high-resolution CT scans (test set B) with fibrotic lung disease compared with the majority vote of 91 specialist thoracic radiologists drawn from multiple international thoracic imaging societies.
• High-resolution CT scans were then reclassified according to Fleischner Society diagnostic criteria for idiopathic pulmonary fibrosis.
• Using these criteria, they retrained the algorithm and assessed its performance on 75 fibrotic lung disease specific high-resolution CT scans compared with four specialist thoracic radiologists using weighted κ coefficient of interobserver agreement.

Results

• Findings suggested the accuracy of the algorithm on test set A to be 76.4%, with 92.7% of diagnoses within one category.
• In order to evaluate 150 four slice montages (each montage representing a single case from test set B), the algorithm took 2.31 seconds.
• Results demonstrated the median accuracy of the thoracic radiologists on test set B to be 70.7% (IQR 65.3-74.7), and the accuracy of the algorithm was 73.3% (93.3% were within one category), outperforming 60 (66%) of 91 thoracic radiologists.
• They noted good median interobserver agreement between each of the thoracic radiologists and the radiologist's majority opinion (κw=0.67 [IQR 0.58–0.72]).
• They found good interobserver agreement between the algorithm and the radiologist's majority opinion (κw=0.69), outperforming 56 (62%) of 91 thoracic radiologists.
• Compared to the majority opinion of the thoracic radiologists (2.74, 1.67-4.48, p < 0.0001), equally prognostic discrimination between usual interstitial pneumonia and non-usual interstitial pneumonia diagnoses was provided by the algorithm (hazard ratio 2.88, 95% CI 1.79-4.61, p < 0.0001).
• Data suggested that for Fleischner Society high-resolution CT criteria for usual interstitial pneumonia, median interobserver agreement between the radiologists was moderate (κw=0.56 [IQR 0.55-0.58]) but was good between the algorithm and the radiologists (κw=0.64 [0.55-0.72]).