GEMA-Score introduces a comprehensive evaluation framework for AI-generated radiology reports that goes beyond simple information coverage to include critical details like abnormality location and diagnostic certainty.

Key innovations:

• Evaluates granular medical details that previous metrics overlooked, including location and certainty of abnormalities
• Uses a multi-agent approach for more thorough and consistent assessment
• Provides explainable scoring to highlight specific strengths and weaknesses in generated reports
• Improves reliability assessment of AI-generated medical documentation

This research is vital for the medical field as it addresses critical gaps in evaluating AI assistance for radiologists, potentially improving diagnostic consistency while ensuring AI-generated reports meet clinical standards for accuracy and detail.

GEMA-Score: Granular Explainable Multi-Agent Score for Radiology Report Evaluation