This innovative research uses large language models as proxies to analyze how the human brain represents and processes visual information from natural images.

• LLMs extract rich semantic data from images using a Visual Question Answering approach
• Model-derived representations successfully predict neural activity patterns in the brain
• Creates a more ecologically valid alternative to traditional psychological experiments
• Establishes new methodologies for analyzing brain semantic representation without manual annotation

This breakthrough offers medical researchers a powerful new tool to study neural processing, potentially advancing our understanding of visual perception disorders and improving brain-computer interfaces.

Talking to the brain: Using Large Language Models as Proxies to Model Brain Semantic Representation