Understanding LLM Code Generation Failures

Understanding LLM Code Generation Failures

Beyond Syntax: Why LLMs Struggle with Context-Dependent Programming

This research examines code defects generated by Large Language Models when dealing with real-world software development contexts and dependencies.

  • LLMs produce non-syntactic mistakes that are harder to detect and potentially introduce security vulnerabilities
  • Previous research focused mostly on standalone functions, overlooking context-dependent programming scenarios
  • The study provides insights for improving code generation tools and preventing defects in automated programming
  • Findings have significant implications for software engineering, security practices, and developer education

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

66 | 323