This research evaluates code-generating LLMs through the lens of real-world software practitioners, identifying what developers actually need for effective code generation tools.

• Examines functionality, syntax correctness, and maintainability of LLM-generated code
• Highlights disconnect between academic evaluation metrics and practical developer requirements
• Proposes empirically-grounded evaluation frameworks based on professional developer feedback
• Offers guidance for creating more effective code generation tools aligned with development workflows

For engineering teams, this research provides valuable insights into how to better integrate LLMs into development processes while addressing practitioners' actual needs rather than theoretical performance metrics.

Large Language Models for Code Generation: The Practitioners Perspective