(284h) Machine Learning Enabled Cancer and Immune Cells Segmentation on Clinical Images

Motivation: Cancer Diagnosis typically involves a manual examination of stained tissue samples to capture different disease patterns. This task is often time-consuming, subjective, and limited in observing all the heterogeneous components. Manually utilizing pattern information from multiple cell types can be difficult, and computerized pattern recognition has been reported to be effective^1–4. Additionally, our lack of understanding of the progression of this disease limits the ability to provide timely, holistic, and personalized care to the growing number of cancer patients around the world. There are several molecular and physiological changes that together govern the course of the disease. Currently, many immunohistochemical biomarkers are used to investigate breast tumors and determine their prognosis. Some of these biomarkers, like tumor-infiltrating immune cells, are important targets for new therapies. There is a lack of standardized assessment of tumor-infiltrating lymphocytes (TILs, an important class of immune cells) on stained tissue specimens, as suggested by the International Immuno-Oncology Biomarker Working Group^5,6. Machine learning coupled with clinical images can provide a quantitative and objective platform for digitally segmenting cancer cells and immune cells for accurate diagnosis and comprehensive cancer profiling. Here, we present a machine learning based digital disease segmentation identifying different tissue components on breast cancer biopsy images.

Methods and Results: This work utilizes archival tissue samples from clinics and biobanks, primarily formalin-fixed and paraffin-embedded (FFPE). The samples are stained using Hematoxylin and Eosin (H&E) staining, the most commonly used stain in medical diagnosis. These samples are imaged using a whole slide image scanner that provides a high-resolution image of the entire tissue sample. We have predominantly used datasets from public healthcare repositories in this work, but we will also extend our pipelines to datasets from our clinical collaborators. We use a combinatorial approach of unsupervised learning for data exploration to separate heterogeneous cell components and supervised learning to capture diagnostic classes in the sample. First, a simple deep learning model (previously developed by Mittal et al. ⁷) is utilized to digitally segment the epithelial and stromal compartments on the H&E stained image. The developed pipeline takes an H&E image as an input and divides it into small regions of interest at different zoomed levels. The goal was to mimic the protocol followed by a pathologist for diagnostic decision-making. Next, these compartments are individually segmented using clustering and image segmentation to identify the different populations of the cells. This platform allows the identification of cancer cells and microenvironment sampling, particularly the immune cells, as illustrated in the figure below. The model determines the different tumor subpopulations and the corresponding immune cells in the tumor bed. In the stromal areas, the model captures various pockets of the collagen (altered density and polarization) and the intermixed cellular components like immune cells, fibroblasts, etc.

Outcome: The developed platform provides the ability to conduct quantitative, rapid, and standardized visualization of tumor subpopulations, stromal signatures, and immune cells, allowing clinicians to determine a comprehensive/accurate disease profile.

References:

1. Beck, A. H. et al. Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival. Science Translational Medicine 3, 108ra113-108ra113 (2011).
2. Janowczyk, A. & Madabhushi, A. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. Journal of Pathology Informatics 7, 29 (2016).
3. Litjens, G. et al. A survey on deep learning in medical image analysis. Medical Image Analysis vol. 42 60–88 (2017).
4. Ehteshami Bejnordi, B. et al. Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA 318, 2199 (2017).
5. Kos, Z. et al. Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer. npj Breast Cancer 6, 1–16 (2020).
6. Dieci, M. V. et al. Update on tumor-infiltrating lymphocytes (TILs) in breast cancer, including recommendations to assess TILs in residual disease after neoadjuvant therapy and in carcinoma in situ: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. Seminars in Cancer Biology vol. 52 16–25 (2018).
7. Mittal, S., Stoean, C., Kajdacsy-Balla, A. & Bhargava, R. Digital Assessment of Stained Breast Tissue Images for Comprehensive Tumor and Microenvironment Analysis. Frontiers in Bioengineering and Biotechnology 7, 246 (2019).