Deep Hybrid Learning with CNN and Transformer for Lung Cancer Detection and Grading in Histological Images

Abstract

Author(s): Pragati Patharia, Prabira Kumar Sethy*, Anita Khanna, Neepa Biswas, Santi Kumari Behera, Aziz Nanthaamornphong and Yie Wai Leong*

Histopathological evaluation of biopsy samples plays a crucial role in the diagnosis of lung carcinoma and subsequent treatment decisions. Automated analysis of biopsy samples can assist the pathologists in grading lung cancer using multi- magnification microscopy images acquired at different magnifications, providing both local and global information. In this paper, we introduce a hierarchical end-to-end deep learning framework called Multi-Resolution CNN-Transformer Network (MRCT-Net), for accurate classification and grading of lung cancer using multi-magnification (20× and 40×) histopathology images. To obtain multi-scale representations, we construct image pyramids of three resolutions (224 × 224; 112 × 112; and 56 × 56), which are fed into three lightweight CNN encoders, EfficientNetV2, EfficientNetV2-Tiny, and ConvNet-Tiny architectures, respectively. To train and evaluate our proposed model, we have collected a dataset of 691 histopathology whole-slide images (WSIs) at 20×and 40× magnifications consisting of various subtypes and grades of lung carcinoma. Experimental evaluation shows that the proposed MRCT-Net achieves better performance than the baseline models on both classification and grading tasks. The proposed model achieves 93.48% accuracy and 91.30% accuracy for classification and grading tasks respectively, when trained with combined multi-resolution inputs (20× and 40×) and using MRCT-Net with SVM classifier head. The proposed multi- resolution framework learns local spatial and contextual information from CNN and Transformer backbones respectively, which can be used as a complementary, accurate, and interpretable solution for automated histopathology analysis of lung cancer.

Share this article