AffectGPT introduces a novel framework for comprehensive emotion understanding using multimodal large language models (MLLMs), addressing critical gaps in existing emotion recognition capabilities.

• Creates a large-scale dataset with intensive, descriptive emotion annotations
• Develops a multimodal-centric framework optimized for MLLMs
• Enables advanced video understanding with natural language descriptions of emotions
• Establishes new benchmarks for emotion recognition performance

Medical Impact: This research has significant implications for mental health assessment, enabling more nuanced patient emotional monitoring, potentially improving diagnostic accuracy, and supporting therapeutic interventions through better emotion recognition.

AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models