2214 - Performance Analysis and Knowledge-Based Quality Assurance of Critical Organ Auto-Segmentation for Craniospinal Irradiation in Pediatric Proton Therapy

O. Ates, E. Hanna, E. Sargent, C. H. Hua, and T. E. Merchant; St. Jude Childrens Research Hospital, Memphis, TN

Purpose/Objective(s): Craniospinal irradiation (CSI) is a vital treatment for young patients with seeding tumors such as medulloblastoma. This study aimed to evaluate two different methods for auto-segmentation of organs at risk: atlas and neural network techniques. Additionally, a novel quality assurance (QA) tool was devised to prospectively assess the accuracy of auto-segmentation as a knowledge-based QA method. Materials/

Methods: This study included a group of 100 CSI patients, aged between 2 and 25 years with a median age of 8, and utilized quantitative overlap and distance metrics to compare the two methods of auto-segmentation against ground truth contours, using 13 specific metrics for 16 organs including brain, brainstem, chiasm, esophagus, left and right cochleae, lenses, eyes, optic nerves, kidneys, and lungs. An in-house knowledge-based quality assurance tool was developed by leveraging the distinctive patterns of CT number distributions for each individual organ. The kernel density estimation (KDE) of each ground truth contour was sampled as a baseline distribution, encompassing 100 patients for 16 organs.
Results: The metrics analysis showed that in most instances (60.58%), the neural network approach achieved superior performance when compared to the atlas method. In 31.25% of instances, there was no noticeable difference between the two methods, while the atlas method was more effective in 8.17% of the comparisons. The CT number-based QA tool revealed that the two approaches demonstrated 75% of agreements with baseline KDEs in 65.6% of instances for the atlas method and in 82.5% of instances for the neural network auto-segmentation.
Conclusion: Neural networks offer significant potential for automating segmentation in pediatric radiation oncology, particularly in the labor-intensive contouring process of critical organs during CSI. An in-house knowledge-based QA tool offers a valuable means to quantitatively assess the results of prospective auto-segmentation. Abbreviations: DSC: Dice Similarity Coefficient, JAC: Jaccard Index, TPR: True Positive Rate, TNR: True Negative Rate, RI: Rand Index, PPV: Positive Predictive Value, HD: Hausdorff Distance, MDA: Mean Distance to Agreement. Abstract 2214 – Table 1: 13 quantitative metrics averaged across 16 organs for all 100 patients and compared against ground truth contours for atlas and neural network methods

Metrics	Atlas Method Mean ± SD	Neural Network Method Mean ± SD
DSC	0.66 ± 0.14	0.73 ± 0.04
JAC	0.55 ± 0.12	0.63 ± 0.04
TPR	0.70 ± 0.15	0.72 ± 0.04
TNR	0.90 ± 0.03	0.94 ± 0.02
RI	0.87 ± 0.02	0.87 ± 0.02
PPV	0.65 ± 0.14	0.79 ± 0.04
Std HD [mm]	2.29 ± 1.64	1.63 ± 0.56
Min HD [mm]	3.72 ± 15.22	0.16 ± 0.97
Median HD [mm]	5.63 ± 16.86	1.15 ± 1.03
Mean HD [mm]	6.16 ± 17.21	1.59 ± 1.08
MDA [mm]	5.95 ± 17.16	1.59 ± 1.16
95% HD [mm]	10.62 ± 20.01	4.99 ± 2.00
Max HD [mm]	15.68 ± 20.55	9.26 ± 2.36