This research introduces LLM-based code critics that provide detailed evaluations of code changes without needing to execute the code.

• Addresses limitations of existing metrics like build status and log analysis
• Provides structured, detailed feedback on code quality and correctness
• Enables more effective multi-step LLM-based agentic workflows for software engineering
• Offers a promising approach to automated code review and quality assessment

For engineering teams, this represents a breakthrough in how we evaluate and improve automated coding tools, potentially accelerating development cycles while maintaining quality standards.

Large Language Model Critics for Execution-Free Evaluation of Code Changes