MMSafeAware introduces the first benchmark specifically designed to evaluate safety awareness in Multimodal Large Language Models (MLLMs) that process both text and images.

Key Findings:

• Evaluates MLLMs across 29 safety categories to identify unsafe content
• Reveals significant gaps in current models' ability to detect multimodal safety issues
• Explores methods to improve safety awareness in these increasingly prevalent AI systems

This research addresses critical security vulnerabilities in multimodal AI systems by providing a structured evaluation framework and highlighting the need for enhanced safety mechanisms before widespread deployment in sensitive applications.

Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs