MedR-Bench introduces a rigorous evaluation framework for measuring reasoning abilities of Large Language Models in medical contexts through 1,453 structured clinical cases.

• Evaluates LLMs across 13 body systems and 10 medical specialties
• Assesses performance on critical tasks: examination recommendations, diagnosis, and treatment planning
• Focuses on both final outputs and quality of reasoning processes

This research provides crucial insights for healthcare organizations considering AI implementation by establishing benchmarks for model reliability in high-stakes medical decision support.

Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases