This research compares open-source large language models (LLMs) against specialized translation systems for domain-specific medical content across varying language resource levels.

Key findings:

• LLMs show a significant quality gap compared to dedicated medical translation models
• Performance varies notably based on language resource availability
• English-to-French (high-resource) translations outperform English-to-Swahili (low-resource)
• Results highlight the need for domain adaptation in LLMs for specialized translation tasks

Business relevance: Organizations deploying translation systems for medical content should carefully consider specialized models over general-purpose LLMs, especially for critical healthcare communications or when working with lower-resource languages.

Domain-Specific Translation with Open-Source Large Language Models: Resource-Oriented Analysis