SolEval provides the first comprehensive benchmark for evaluating how Large Language Models generate Solidity smart contracts at the repository level.

• Addresses a critical gap in LLM evaluation for blockchain development
• Assesses smart contracts on security vulnerabilities, code correctness, and gas efficiency
• Tests repository-level generation, not just isolated functions
• Enables measuring LLMs' ability to create secure, cost-effective blockchain applications

Why it matters: Smart contracts manage billions in assets, making security-aware code generation essential for preventing costly vulnerabilities in blockchain applications.

SolEval: Benchmarking Large Language Models for Repository-level Solidity Code Generation