This research introduces ECHOMIST, the first benchmark specifically designed to detect how LLMs handle implicit misinformation – false assumptions embedded in user questions.

• LLMs often fail to challenge misinformed premises in questions (e.g., "How to protect yourself from 5G radiation?")
• Current evaluation methods primarily focus on explicit falsehoods, missing this subtle but dangerous form of misinformation spread
• The benchmark reveals significant vulnerabilities across popular commercial and open-source models
• Simply instructing LLMs to challenge false premises shows limited effectiveness without specialized training

For security professionals, this research highlights critical risks of AI systems inadvertently amplifying misconceptions through seemingly helpful responses, requiring improved safeguards to prevent harmful misinformation spread.

How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation