3446 - Multi-Institutional Qualitative Evaluation of Automatic and Manual Segmentations of Organs at Risk on PRE ACT Breast Cancer Cohorts

K. Verhoeven¹, T. Brion², W. R. Green³, M. Balia⁴, A. Webb⁵, T. Rattay⁶, Y. Liang¹, L. G. Assia⁷, I. Hafsa⁷, S. Romdhani⁴, R. Iandolo⁴, A. Bombezin-Domino⁴, B. K. K. Teo⁸, R. McBeth³, I. Koutsopoulos⁹, C. Talbot⁵, N. Paragyos^4,10, and S. Rivera⁷; ¹Department of Radiation Oncology, Maastricht University Medical Centre - GROW (MAASTRO), Maastricht, Netherlands, ²Institut Curie, Paris, France, ³Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, ⁴TheraPanacea, Paris, France, ⁵Department of Genetics and Genome Biology, University of Leicester, Leicester, United Kingdom, ⁶Leicester Cancer Research Centre, University of Leicester, Leicester, United Kingdom, ⁷Gustave Roussy, Cancer Campus, Villejuif, France, ⁸Department of Radiation Oncology, Division of Physics, University of Pennsylvania, Philadelphia, PA, ⁹Athens University of Economics and Business, Athens, Greece, ¹⁰Centrale Supelec, University of Paris-Saclay, Computer Science and Applied Mathematics, Gif-sur-Yvette, France

Purpose/Objective(s): PRE ACT is an EU funded project that aims to predict the risk of radiotherapy side effects, including arm lymphedema in breast cancer patients based on 4 multicenter international studies CANTO, HYPOG-01 and REQUITE. Because of inter-investigators variability and clinical practice diversity across different centers and cohorts, manual contours are not homogeneous enough to extract necessary features to train the predictive model. The aim of this study is to conduct qualitative and comparative evaluation of the automatic and manual contours to showcase the reliability of the former. Materials/

Methods: We have generated Organ-at-Risk (OAR) contours with Annotate module from ART-Plan (from TheraPanacea) for six crucial OARs for dose planning of breast cancer loco regional radiotherapy, i.e. esophagus, thyroid, heart, ipsilateral lung, ipsilateral humeral head and contralateral breast. For this evaluation, four independent international radiation oncologists have been presented blindly with both automatic and manual contours from a randomly selected subset of left and right breast cancer patients from the PRE ACT dataset (7052 patients). 10 automatic and manual cases were presented across two sessions, with a washout of a minimum of 48 hours. Rating of the contours followed an A-B-C grid, with A for satisfactory contours, B requiring minor changes, and C needing major changes or redoing. A and B contours were estimated as acceptable, therefore ratings of A and B contours define the acceptance rate for each organ.
Results: Average qualitative evaluations are presented in table 1. Automatic contours acceptance ranges from 92 to 100%, whilst manual contours acceptance ranges from 72% to 100%. Inter-expert agreement varies a lot across organs. The lowest inter-expert agreement was obtained for humeral head and heart (45 % for both) for automatic contours and thyroid for manual contours (38,3%). The highest inter-expert agreement was for esophagus and left breast (68.3%) for automatic contours and left breast for manual contours (66.7%).
Conclusion: With this study, we demonstrate that automatic contouring is an excellent and reliable tool to obtain segmentation of OARs with a robust a homogeneous clinical acceptability across diverse cohorts, with many applications to clinical trials. Abstract 3446 – Table 1: Qualitative evaluation of automatic vs manual contours for 9 OARs

		Esophagus	Thyroid	Heart	Left breast	Right breast	Left Humeral head	Right Humeral head	Left lung	Right lung
Automatic	A+B	100%	92,5%	97,5%	97,5%	100%	100%	100%	100%	100%
	C	0%	7,5%	2,5%	2,5%	0%	0%	0%	0%	0%
Manual	A+B	100%	92,5%	85,0%	72,5%	77,5%	95,0%	97,5%	97,5%	97,5%
	C	0%	7,5%	15,0%	27,5%	22,5%	5,0%	2,5%	2,5%	2,5%