This research establishes a systematic evaluation framework for Large Language Models in Intelligent Outpatient Referral (IOR) systems, addressing a critical gap in standardized assessment methods.

• Identifies key capabilities and limitations of LLMs in healthcare referral workflows
• Proposes comprehensive evaluation criteria for dynamic, interactive healthcare scenarios
• Benchmarks performance of current LLM implementations in outpatient referral tasks
• Highlights challenges specific to medical referral contexts

This framework enables healthcare organizations to better assess and implement AI solutions that could streamline patient routing, reduce clinical workloads, and improve care coordination across healthcare systems.

Large Language Models for Outpatient Referral: Problem Definition, Benchmarking and Challenges