This research introduces an open-source benchmark to evaluate how large language models balance safety restrictions with legitimate scientific discourse.

Key findings:

• Provides reproducible datasets to measure both appropriate refusals and over-restriction
• Tests LLM responses to systematically varied prompts around controlled substances
• Reveals distinct safety profiles across major AI models
• Highlights the tension between preventing harmful content and enabling scientific progress

For security professionals, this benchmark offers critical insights into AI safety implementation and potential vulnerabilities, helping organizations deploy LLMs that properly safeguard against misuse without unnecessarily limiting valuable applications.

Forbidden Science: Dual-Use AI Challenge Benchmark and Scientific Refusal Tests