Research reveals a serious flaw in safety metrics for large language models: concatenation vulnerabilities can allow harmful content to bypass safety filters.

• Safety metrics that correctly flag individual harmful responses may fail when those same responses are concatenated
• Multiple established safety metrics demonstrated this vulnerability
• This security gap creates potential for adversarial attacks on content moderation systems
• Findings highlight the need for more robust evaluation methods for LLM safety mechanisms

This research matters because it exposes fundamental weaknesses in current safety infrastructure protecting AI systems from generating harmful content.

How Safe is Your Safety Metric? Automatic Concatenation Tests for Metric Reliability