3362 - In-Depth Timing Study of AI-Assisted OAR Contouring with a Single Institution Expert

S. Elguindi¹, J. Jiang¹, H. Veeraraghavan¹, A. Apte¹, E. LoCastro¹, A. Iyer¹, and I. Onochie²; ¹Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY, ²Memorial Sloan Kettering Cancer Center, New York, NY

Purpose/Objective(s): Artificial Intelligent (AI) assisted contouring of organs at risk (OARs) in radiation therapy is quickly becoming the default contouring method of choice due to its demonstrated benefits in consistency and time-savings. This study aims to undergo a detailed and systematic review on a per-contour basis of the comparison in work required to generate clinically usable OARs manually versus AI generated contours combined with human expert review. Materials/

Methods: Our institution employs a full-time radiation anatomist to help contour OARs for all disease sites. The anatomists onboarding involved 8 months of training conducted by radiation oncologists on OAR contouring for radiation therapy planning with the goal of improving quality and consistency, while reducing physicians contouring burden. Our institution released various in-house developed AI models to contour 33 different OARs on CT imaging for use in routine clinical work. We conducted a per-contour timing study of the anatomist for these CT-Based OARs. An observer over a screen-sharing application measured with a stopwatch the total time it took for the anatomist to either review and edit the AI contour or generate the contour manually. A careful effort was taken to only include the time spent reviewing and/or contouring. For each instance we then generated a contour rate (CR) in centimeters per second by measuring the total linear path length of the contour surface divided by the total time measured. The CR Ratio was then defined for each OAR as the mean AI assisted CR (AICR) divided by the mean manual CR (MCR).
Results: 113 manual and 103 AI assisted timings were collected. The overall mean MCR was 7.9 +/- 10.34 cm/s and the mean AICR was 47.3 +/- 77.4 cm/s. The average CR-ratio was 5.0 +/- 2.8, demonstrating our anatomist can, on average, contour OARs 5x faster by reviewing and, if needed, correcting AI contours over generating them manually. At the individual OAR level, the 3 lowest performing structures as dictated by the lowest CR Ratio were the optic nerves and chiasm, with the latter being the only structure to have a ratio less than 1. Conversely the highest performing structures were the lungs, oral cavity, mandible, and spinal cord with contour ratios of over 8. Notably, difficult to contour structures such as large and small bowel have CR ratios near the mean at 6.1 and 5.3 respectively. It was noted that the anatomists proper use of sophisticated contouring tools, such as next slice prediction, aid in rapid editing which greatly improved AI contour review.
Conclusion: This study demonstrated that AI-assisted contouring offers the ability for an anatomist to make a larger impact in the planning process by increasing efficiency 5-fold. The usefulness of AI generated contours is seen across many different OARs and is shown to be particularly impactful for large, clearly defined structures. We aim to improve this work by including more timings to increase the statistical power and benchmark the expert anatomist against less experienced staff.