ModalTune represents a breakthrough in digital pathology, enabling fine-tuning of slide-level foundation models with multi-modal information to improve cancer prediction tasks.

• Addresses the challenges of massive whole-slide images (WSIs) and weak training signals
• Leverages both image and clinical data through multi-modal information fusion
• Prevents catastrophic forgetting during model fine-tuning
• Demonstrates improved performance across multiple cancer types

This research significantly advances medical diagnostics by creating more accurate and robust AI systems for pathology image analysis, potentially leading to better cancer subtyping, diagnosis, and treatment planning.

ModalTune: Fine-Tuning Slide-Level Foundation Models with Multi-Modal Information for Multi-task Learning in Digital Pathology