Application of H&E images combined with gene expression data-driven multimodal deep learning models in gastric cancer prognosis

Objective Accurate prognostic stratification is critical for personalized treatment of gastric cancer (GC). This study aimed to develop PGC-pro (Pathology + Clinical + Genomic Prognosis), a multimodal deep learning model integrating whole-slide histopathological images (H&E WSI), gene expression profiles, and clinicopathological variables to improve prognostic accuracy and interpretability. Methods A total of 318 GC patients with matched H&E WSI, RNA-seq data, and clinical information were enrolled from the TCGA-STAD cohort. Pathological features were extracted using the CLAM framework with ResNet50 encoder. A three-tiered strategy (univariate Cox regression + Benjamini-Hochberg correction + multivariate Cox regression) was applied to identify 150 independent prognostic genes. PGC-pro adopted a three-stage bidirectional cross-attention architecture: intra-modal feature enhancement → cross-modal bidirectional interaction → precise feature fusion. Model performance was evaluated via case-level stratified 5-fold cross-validation using C-index, time-dependent AUC, Kaplan-Meier analysis, and Log-rank test. Interpretability was assessed using SHAP analysis and CLAM attention heatmaps. Results PGC-pro achieved an average C-index of 0.701 ± 0.060, with time-dependent AUCs of 0.730 ± 0.067 (12 months), 0.724 ± 0.074 (36 months), and 0.726 ± 0.033 (60 months). The high- and low-risk groups stratified by median risk score showed significant survival separation (Log-rank P < 0.0001). Ablation experiments demonstrated that the trimodal model improved C-index by 4.9% compared with the best single-modal model (genomics only, C-index = 0.668) and by 5.1% compared with the pathology-genomics bimodal model. SHAP analysis identified NKAIN4, DOK7, and MISP3 as core prognostic drivers, while CLAM heatmaps localized tumor invasion fronts and necrotic regions. Conclusion The PGC-pro multimodal model effectively integrates multi-dimensional data for precise GC prognosis prediction with excellent interpretability, providing a promising tool for clinical decision support. Trial Registration Clinical Trial Number : Not applicable. This work is a bioinformatics secondary analysis study based on public database data and does not belong to clinical trial research.