This research introduces a novel approach that leverages large language models to interpret and describe neural activity patterns in the human visual cortex.

• Employs LLM-assisted visual captioning to explain what specific brain regions are responding to
• Addresses the interpretability challenge of traditional deep neural network models of brain activity
• Creates natural language descriptions of voxel properties in the visual processing system
• Bridges neuroscience and AI to enhance our understanding of human visual perception

This breakthrough offers medical researchers a new tool to understand brain function, potentially leading to improved diagnostics for visual processing disorders and brain-computer interfaces.

LaVCa: LLM-assisted Visual Cortex Captioning