This research systematically assesses the safety vulnerabilities of seven large language models when deployed in healthcare contexts, with a focus on their susceptibility to jailbreaking techniques.

• Proposes an automated evaluation pipeline specifically adapted for medical contexts
• Tests three advanced black-box jailbreaking techniques against popular LLMs
• Identifies critical security gaps that could lead to harmful information in clinical settings
• Provides insights for developing safer AI clinical assistants

As AI increasingly integrates into healthcare decision-making, this research highlights urgent security concerns that must be addressed before widespread deployment of LLM-based clinical systems.

Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare