EndoChat introduces a groundbreaking Multimodal Large Language Model specifically designed for endoscopic surgery, addressing the critical need for AI-powered surgical scene understanding.

• Enables real-time surgical guidance and training through specialized dialogue capabilities
• Supports multiple surgical understanding subtasks relevant to operating room scenarios
• Adapts advanced LLM technology to the unique challenges of robotic-assisted surgery
• Represents a significant step toward AI-augmented surgical procedures

This research bridges the gap between generalized AI models and specialized medical applications, potentially transforming surgical education, reducing medical errors, and improving patient outcomes through enhanced surgical decision support.

EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery