3332 - Assessing ChatGPTs Utility as an Educational Aid for Cancer Patients Undergoing Radiation Therapy

S. Abi Farraj^1,2, T. Nahle^1,2, A. Richa^1,2, E. Nasr^1,2, D. Nasr^1,2, T. H. Felefly^2,3, S. Achkar⁴, and F. Azoury^1,2; ¹Department of Radiation Oncology, School of Medicine, Saint Joseph University, Beirut, Lebanon, ²Department of Radiation Oncology, Hôtel Dieu de France University Hospital, Beirut, Lebanon, ³Radiation Oncology Department, Hôtel-Dieu de Lévis, Lévis, QC, Canada, ⁴Department of Radiation Oncology, Centre Charlebourg, La Garenne Colombe, France

Purpose/Objective(s): In the field of oncology, the integration of Artificial Intelligence represents a shift in the dynamics of patient education, though its efficacy in providing patient education within radiation oncology remain uncertain and subject to ongoing evaluation. The purpose of this study is to assess the potential of ChatGPT in enhancing patient comprehension of their disease, helping in the integration of AI into radiation oncology patient care. Materials/

Methods: The methods of the study involved using ChatGPT to answer 156 questions related to radiation therapy for six types of cancer, evenly distributed. 5 radiation oncologists, with an average of 12 years of experience, evaluated these answers assigning a score of "accurate" (1 point) or "inaccurate" (0 points) to each response and gave their perspectives on its reliability. A Likert-type scale was used to evaluate each answer by adding the 5 doctors answers on each question to get an overall answer concerning every response (1point /5 points: Strongly Disagree, 2/5: Disagree, 3/5: Neutral, 4/5: Agree, 5/5: Strongly Agree).
Results: Overall, they strongly agreed (5/5) on 75.6% of the responses, agreed (4/5) on 15.4%, were neutral (3/5) on 5.8%, disagreed (2/5) on 3.2%, and did not strongly disagree on any response. A sample of the questions and their evaluation is listed in the table attached to this abstract. Additionally, 4 out of 5 doctors deemed the reliability as "somewhat reliable," while one considered it "not reliable." 3 out of 5 doctors were "unlikely to recommend" ChatGPT to cancer patients, whereas 2 out of 5 would "recommend it with caution.". All the doctors stated that they wouldnt use ChatGPT as an educational aid for their patients. The average accurate responses was 144,6 accurate [138-149] responses out of 156.
Conclusion: Our research highlights the promising potential of ChatGPT in educating patients, especially those with complex cancer-related inquiries, due to their high accuracy. However, surveyed doctors express reservations about its reliability for cancer information, with most hesitant to recommend it to patients. Caution is warranted due to potential inaccuracies and lack of medical validation. Future studies should focus on standardizing evaluation methods and integrating these tools with direct interactions between patients and doctors.

Abstract 3332 – Table 1

	Breast	Prostate
When Should Radiation Treatment Start?	Strongly Agree	Strongly Agree
Is Radiation Used for Early or Late Stages of X Cancer, or as a Backup Option?	Neutral	Neutral
Will I Be Radioactive After Treatment?	Strongly Agree	Agree
What Will I Feel During Radiation Treatment?	Agree	Strongly Agree
How Long Does a Radiation Treatment Take?	Agree	Strongly Agree
What Are the Side Effects of Radiation for X Cancer?	Agree	Strongly Agree
Will Radiation Make Me Lose My Hair or Taste?	Strongly Agree	Strongly Agree
Does Radiation Increase My Risk of Other Cancers?	Strongly Agree	Agree
How Do We Treat Side Effects from Radiation?	Strongly Agree	Strongly Agree