This research introduces Visual Prompt Engineering (VPE) to improve CLIP models for radiology by directing attention to relevant pathology regions.

• Adapts CLIP for medical image classification without retraining
• Develops localized attention mechanisms using bounding boxes to focus on pathologies
• Achieves significant performance improvements over standard CLIP in zero-shot chest X-ray classification
• Demonstrates increased clinical reliability through better disease localization

This innovation matters because it enables adaptive, flexible diagnostic systems that can identify new medical conditions without costly retraining, potentially accelerating clinical workflows and improving patient care.

Visual Prompt Engineering for Vision Language Models in Radiology