This research investigates how large language models trained for code generation may inadvertently leak sensitive information from their training data, identifying both unintentional and malicious disclosure risks.

Key Findings:

• LLMs can unintentionally generate sensitive content without users explicitly seeking it
• Attackers can intentionally prompt LLMs to extract secrets from training data
• The paper proposes methods for testing LLM security vulnerabilities
• Identifies need for improved privacy safeguards in AI code generation systems

Why It Matters: This research highlights critical security concerns for organizations deploying code-generating AI systems, emphasizing the need for robust privacy testing and protection mechanisms to prevent exposure of sensitive information.

Malicious and Unintentional Disclosure Risks in Large Language Models for Code Generation