This research investigates how Large Vision-Language Models lose visual information during generation, leading to hallucinations, and proposes a novel mitigation method.

Key Findings:

• Visual information gradually diminishes during the generation process, causing tokens to become increasingly ungrounded
• Researchers identified three key patterns in how LVLMs process visual information
• A new approach called Visual Information Steering effectively reduces hallucination without additional training

Security Implications: By reducing AI's tendency to generate false visual descriptions, this research directly addresses security concerns related to misinformation and enhances the reliability of AI systems for critical applications.

The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering