CSR-Bench evaluates how effectively LLM agents can deploy complex computer science research repositories, particularly for ML/AI projects.

• Measures performance across 11 real-world repositories with varying complexity
• Evaluates agents built on models like Claude and Llama for code understanding and deployment tasks
• Provides a standardized benchmark for assessing LLM capabilities in research engineering contexts

This research addresses a critical engineering challenge: automating the deployment of increasingly complex research codebases, potentially saving researchers significant time and reducing barriers to research reproducibility.

CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories