V. S. Tan1, E. Wang1, J. Chong2, A. J. Hope3, T. Tadic4, S. Kandel5, C. McIntosh6, A. Warner1, D. A. Palma7, and P. Lang1; 1Department of Radiation Oncology, Western University, London Health Sciences Centre, London, ON, Canada, 2Department of Medical Imaging, Western University, London Health Sciences Centre, London, ON, Canada, 3Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada, 4Department of Radiation Oncology, Princess Margaret Cancer Centre, University of Toronto, Toronto, ON, Canada, 5Department of Medical Imaging, University of Toronto, Toronto, ON, Canada, 6Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada, 7Department of Radiation Oncology, London Health Sciences Centre, London, ON, Canada
Purpose/Objective(s): Patients with interstitial lung disease (ILD) are at high risk of pneumonitis with lung stereotactic ablative radiotherapy (SABR). The MIRACLE-ILD AI screening tool is a convolutional neural network (CNN) developed to screen patients receiving radiation to identify undiagnosed ILD using the radiation planning CT. This tool has been validated prospectively at a single institution. The purpose of the study was to assess performance of the MIRACLE-ILD AI tool in: (1) patients receiving routine SABR at an external institution, and (2) patients with diagnosed ILD treated with SABR on the ASPIRE-ILD clinical trial. Materials/
Methods: The performance of the MIRACLE-ILD was assessed on a retrospective cohort of patients who received SABR to lung lesions at an external institution from January 2010 to July 2023. A thoracic radiologist ranked severity of imaging findings (10-point Likert scale) of all cases screened by MIRACLE-ILD as high-risk (model output of >0.55), and a random selection of ten low-risk cases (to blind radiologist to the model result). In addition, early-stage lung cancer patients with ILD treated with SABR from 5 institutions enrolled in ASPIRE-ILD were also used to assess the validity of the model, including characteristics of treatment institution, radiological pattern, ILD type, ILD-GAP index and adverse events. Pearson correlation coefficient, Kruskal-Wallis test and Fishers exact test were used for model score comparisons as applicable. Results: In the retrospective cohort, MIRACLE-ILD selected 13 of 308 (4.2%) cases as high risk. Four of 13 (31%) high-risk cases and 1/10 (10%) low-risk cases were identified as having ILD by the radiologist. The correlation between the model score and radiologist ranking was 0.46. Four of the high-risk cases identified by the model developed pneumonitis (= grade 2) following SABR. In the ASPIRE dataset, 16 of 38 (42.1%) patients with ILD were classified by the model as high risk. Mean ± SD model scores were lower at the 4 external institutions (0.54 ± 0.16) compared to the training institution (0.71 ± 0.13) but was not statistically significant (p=0.09). There was no association between radiological type, ILD pattern, ILD-GAP index and adverse event rate and model scores. Conclusion: In an external dataset of patients treated with SABR, a high proportion (1 in 3) of cases screened as high-risk by MIRACLE-ILD were identified as having radiological findings of ILD. As a screening tool, MIRACLE-ILD may have value in identifying patients with undiagnosed ILD prior to radiation. However, MIRACLE-ILD demonstrated low sensitivity in identifying ILD in ASPIRE-ILD patients and lower model scores measured in patients enrolled at external institutions. Further work is required to determine if this is due to heterogeneity in image acquisition or variances in clinical population and practices. Institution-specific calibration and quality assurance will be required for clinical implementation.
