200 - Automated Treatment Planning with Deep Reinforcement Learning for Head-and-Neck Cancer Intensity Modulated Radiation Therapy

D. Yang¹, X. Wu¹, X. Li¹, R. Mansfield¹, Q. J. J. Wu², Q. Wu¹, and Y. Sheng¹; ¹Duke University Medical Center, Durham, NC, ²Duke University, Durham, NC

Purpose/Objective(s): Intensity modulated radiation therapy (IMRT) for head-and-neck (HN) cancer presents a unique set of challenges due to the complex, nonconvex geometry of the target volumes and the proximity to multiple organs-at-risk (OARs). Attaining an optimal balance between OAR sparing and target coverage requires substantial expertise and time. In order to improve planning quality and consistency, in this study we proposed a deep reinforcement learning (RL) framework to self-interact with the treatment planning system (TPS) by driving objective functions during inverse optimization to generate high-quality HN IMRT treatment plans. Materials/

Methods: The RL framework was designed to adopt human planning style: intermediate dosimetric endpoints’ value, objective function setting and objective function loss during inverse optimization were collected as the RL state, and actions were defined as modifying constraints’ dose value. Inverse optimization was initiated after each action and dose-volume-histograms (DVHs) were then updated. Reward signal computed from current DVHs and clinical objectives were sent back to the network to update action policy during model training. Based on the soft actor critic (SAC) algorithm, the RL framework utilizes a dual-network actor-critic architecture, enabling efficient data reuse through its off-policy formulation and promoting stability and exploration via entropy maximization. The framework was trained on 20 HN IMRT patients and evaluated with another 100 patients with IRB approval. The RL generated plans (RL plans) were normalized to same planning target volume (PTV) coverage as clinical plans for plan quality comparison. Clinically relevant dosimetric endpoints were extracted and Wilcoxon Signed-Rank test was performed for comparison.
Results: The testing dosimetric endpoints and MU statistics are tabulated in Table 1. The RL plans show improved hot spot control while preserving same dose coverage. Additionally, the RL plans have comparable or improved dose sparing over clinical plans for all OARs except larynx. The lower total MU of RL plans indicates reduced plan complexity and increased efficiency.
Conclusion: We developed a deep reinforcement learning framework to automatically generate high-quality HN IMRT plans with consistency and efficiency.

Abstract 200 – Table 1

	PTV Dmax (%)	Left Parotid D50% (Gy)	Right Parotid D50% (Gy)	Oral Cavity D50% (Gy)	Larynx D50% (Gy)
Clinical (mean±std)	112.3±5.8	17.1±8.5	15.7±7.9	24.8±8.1	23.7±10.4
RL (mean±std)	109.2±7.6	15.6±3.4	12.2±3.9	25.7±7.1	27.3±8.8
P-value	<0.001	0.928	<0.001	0.083	<0.001
	Pharynx D50% (Gy)	Brainstem Dmax (Gy)	Cord+5mm Dmax (Gy)	Mandible Dmax (Gy)	Total MU
Clinical (mean±std)	35.5±6.1	17.5±4.3	31.9±3.9	47.6±1.3	1880±288
RL (mean±std)	32.1±6.4	15.2±2.6	30.5±1.5	46.0±1.4	1638±359
P-value	<0.001	<0.001	0.003	<0.001	<0.001
Dmax: maximum dose value; D50%: median dose; MU: monitor unit