This research reveals how misleading charts can manipulate multimodal LLMs, reducing their accuracy to random guessing levels, and introduces effective countermeasures.

• Misleading visualization techniques (truncated/inverted axes) severely impair multimodal LLMs
• Chart distortions can support misinformation and conspiracy theories
• Researchers developed specialized defenses to restore model accuracy
• Results demonstrate the importance of visualization literacy for AI security

Security Implications: This work identifies a critical vulnerability in multimodal AI systems that could be exploited to spread misinformation at scale, while providing practical mitigation strategies for AI developers.

Protecting multimodal large language models against misleading visualizations