D. Bayona1,2, D. K. Ebner2, T. M. Weiskittle2, B. C. Kamdem Talom2, R. O. Kowalchuk2, W. Breen2, D. M. Routman2, and M. R. Waddle2; 1University of Texas Dallas, Dallas, TX, 2Department of Radiation Oncology, Mayo Clinic, Rochester, MN
Purpose/Objective(s): Cancer databases have historically fallen short in providing information regarding cancer recurrence due to challenges collecting this information longitudinally. Accurate database information is critical both as research tool and to allow identification of patient status in real time. We hypothesized that machine learning with natural language processing would allow for automated classification of cancer pathology reports as recurrent or not, facilitating the process of accurate registry collection. Materials/
Methods: Patients treated with radiation therapy at one tertiary referral center from 2010 to 2018 with a verified cancer status (cancer recurrence vs. no cancer recurrence) were identified. Patients with recurrent disease were initially identified through manual record review, and the associated pathology report collected. Automated machine learning with natural language processing was employed (Alphabet Inc.) to generate a model for binary classification, with comparison to the gold-standard manually-developed dataset. Results: 7054 patients were identified. 3431 (49%) were female, with median age 64 years. Head and neck (1482, 21%), breast (1480, 21%), upper GI (1307, 19%), and lung/thorax (1107, 16%) were the most common disease sites. Recurrence was verified for 1546 (21.9%) and 981 (13.9%) patients using pathology and radiology reports, respectively. Of these recurrences, 973 were local (13.8%), 1121 regional (15.9%), and 1768 distant (25.1%). 1249 confirmed positive pathology reports were paired with 651 negative pathology reports and used for model training using automated delineation of training and test data (training: 1508, validation: 189, testing: 189). The best fitting model produced demonstrated precision of 99%, recall of 98%, and specificity of 97%. On manual comparison, model-derived false positives resulted from abnormal dictation or uncommon wording sequence, including diagnostic language with negation (ie. “Evidence of recurrence or progression is not noted”). Notably rare pathologies were more likely to generate false negatives secondary to limited reports demonstrating these pathologies available during algorithm training. Conclusion: Automated machine learning with natural language processing serves as a promising tool for identifying recurrence from pathology reports, and may accelerate or improve the identification of cancer recurrence information.
