This research systematically evaluates bias in large language models when responding to palliative care scenarios, with significant implications for responsible AI deployment in healthcare.

Key Findings:

• GPT-4o was tested using the novel Palliative Care Adversarial Dataset (PCAD) specifically designed to reveal biases
• Responses were evaluated by palliative care experts, revealing concerning patterns of bias
• Marginalized groups are particularly vulnerable to AI-perpetuated inequities in care
• The study highlights the urgent need for bias detection and mitigation strategies in medical AI applications

Why This Matters: As healthcare increasingly adopts AI solutions, understanding and addressing algorithmic bias becomes critical for ensuring equitable care delivery, particularly in sensitive contexts like palliative care where trust and ethical treatment are paramount.

Large language models perpetuate bias in palliative care: development and analysis of the Palliative Care Adversarial Dataset (PCAD)