3389 - Improving Intestinal Crypt Detection in Deep Learning Models with Reinhard Method-Based Stain Color Normalization for Evaluating Normal Tissue Radiation Response

M. J. Kim¹, Z. Yang², J. Fu², S. Melemenidis², B. W. Loo Jr³, and X. Gu⁴; ¹Department of Radiation Oncology, Stanford University School of Medicine, Palo Alto, CA, ²Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, ³Department of Radiation Oncology, Stanford University, Stanford, CA, ⁴Stanford University Department of Radiation Oncology, Palo Alto, CA

Purpose/Objective(s): Intestinal regenerating crypts are considered a common endpoint surrogate to evaluate abdominal normal tissue radiation response. Deep learning-based approaches have been used to automatically identify crypts in mice histological images to assist the analysis and improve efficiency. However, large stain variations are commonly present in histological images and often introduce visual inconsistencies that negatively affect the performance of deep learning models (DLMs). To alleviate the influence of stain variations on DLM performance, we implemented a Reinhard Method-based color normalization approach to transfer the images into their reference color characteristics as we aim to improve performance of an in-house crypt auto-detection DLM on varying stain coloration mice abdominal histological images. Materials/

Methods: We collected 350 H&E stained histological images of mice jejunum cross-section from a previous normal tissue radiation response study where all the mice underwent whole abdominal irradiation. An expert reviewed and sorted all the histological images into 2 groups: reference (200 images) or variant (150 images) groups. The reference group contains the histological images with stain color styles preferred by the expert, while the variant group consists of images with suboptimal color style variations that may affect following analysis. Images from both groups were all pre-processed to remove the background. The variant group was further pre-processed by implementing the Reinhard method for stain normalization. In detail, we extracted the color distribution characteristics, such as background luminance and channel-wise color information, across all reference groups. The extracted reference characteristics were then transferred to the variant group images globally through a linear transformation matrix to normalize the color distribution characteristic histograms of the variant group to the reference group histograms. To assess the color normalization effectiveness, we tested both the original and the normalized variant group images with our in-house crypt auto-detection DLM, which was previously trained with the reference group images, to compare the crypt auto-detection performance.
Results: Compared with the original images with stain variations, the crypt detection sensitivity of the DLM was improved by 24% (0/25 to 6/25) using post-normalized images.
Conclusion: Improvements of detection sensitivity indicate improved model performance of using normalized histology slides for previously trained DLMs. Overall, our study demonstrates that this color distribution-based stain normalization approach can improve performance of existing DLMs on histological images with stain variations.