This research evaluates the effectiveness of LLM-based chatbots in providing accurate and reliable information about menopause.

• Mixed-methods evaluation comparing multiple LLM chatbots on menopause queries
• Accuracy assessment revealing performance gaps in healthcare knowledge
• Safety concerns identified when LLMs provide medical advice
• Methodological framework for evaluating AI systems in sensitive health contexts

This work highlights the critical need for robust evaluation metrics before deploying AI assistants in healthcare settings, where misinformation can lead to adverse outcomes.

A Mixed-Methods Evaluation of LLM-Based Chatbots for Menopause