This research introduces a holistic benchmark for evaluating histopathology vision-language models across diverse clinical tasks, addressing critical gaps in current assessment approaches.

• Assesses model performance across multiple organs, cancer types, and instruments
• Enables standardized evaluation despite privacy restrictions on patient data
• Provides clearer insights into model generalizability and clinical applicability

This benchmark advances medical AI by establishing rigorous standards for vision-language models in cancer pathology, potentially accelerating development of reliable diagnostic tools for clinical practice.

How Good is my Histopathology Vision-Language Foundation Model? A Holistic Benchmark