Researchers demonstrate how attackers can subtly manipulate prompts to bias large language model outputs without users noticing.

Key findings:

• Subtle synonym replacements in prompts can increase the likelihood of target concept mentions by up to 78%
• These manipulations are difficult for humans to detect
• Prompt optimization services create a new attack vector for malicious actors
• The attack requires no model access, working through black-box interactions

This research exposes critical security vulnerabilities in how we interact with LLMs, especially as more users rely on third-party prompt services. Organizations should implement warnings against untrusted prompt sources and develop detection mechanisms for manipulated prompts.

LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses