This research evaluates how effectively modern large language models can support clinical decision-making by testing them on actual medical certification exams.

• Study benchmarked both open-source and closed-source LLMs on the 2024 Portuguese National Medical Exam
• Results demonstrate varying capabilities across models in medical diagnosis accuracy and treatment planning
• Research identifies specific strengths and limitations of AI systems in healthcare applications

Why it matters: As healthcare systems face increasing demands, AI assistants could help clinicians process complex medical information faster, potentially improving diagnostic accuracy and treatment decisions - while highlighting critical areas needing human expertise.

Performance of Large Language Models in Supporting Medical Diagnosis and Treatment