
Understanding LLM Code Generation Failures
Beyond Syntax: Why LLMs Struggle with Context-Dependent Programming
This research examines code defects generated by Large Language Models when dealing with real-world software development contexts and dependencies.
- LLMs produce non-syntactic mistakes that are harder to detect and potentially introduce security vulnerabilities
- Previous research focused mostly on standalone functions, overlooking context-dependent programming scenarios
- The study provides insights for improving code generation tools and preventing defects in automated programming
- Findings have significant implications for software engineering, security practices, and developer education
A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?