AI Code
Generation
Risks
Ramkumar Dilli
CONTENTS
Why Developers Love AI Tools
2.
1. Introduction
Real-World Incidents
The Security Risks
3. 4.
Balancing Innovation with Security
The Business Risk
5. 6.
The Future of Secure AI Coding
Mitigations & Best Practices
7. 8.
Q&A / Discussion
Key Takeaways
9. 10.
02
Why Developers Love AI Tools
Introduction
Copilot, ChatGPT,
CodeWhisperer
Rapid adoption Developer
productivity boost
The Rise of AI Code Assistants
Cuts boilerplate
Quick learning of APIs/libraries
Benefits
Fewer Stack Overflow searches
Faster problem- solving
01
02
03
04
Why Developers Love Them
03
The Security Risks
AI generates code, not security
Developers may trust output blindly
Security review gaps
Security Concerns
01
02
03
But What About Security?
Real-World Example: AI-Generated SQL Injection Risk
Vulnerable code patterns
01
Prompt injection attacks
03
Licensing/IP contamination
02
Sensitive data leakage
04
Key Risks
Core Security Risks
Unsafe library usage
Hardcoded secrets
Examples
Insecure defaults No
validation/sanitization
Vulnerable Code Patterns
40% of Copilot’s security- sensitive code had vulnerabilities
ExamplesSQL Injection, XSS
NYU 2021 Study
01
02
Copilot Vulnerability Study
Risk of GPL contamination
AI trained on public repos
Concerns
Unclear legal standing
Licensing and IP Risks
AI completion can be
manipulated
Hidden instructions in
prompts
New attack surface
Risks
Prompt Injection Attacks
Proprietary logic
exposure
Developers pasting code
into AI tools
Incidents
Samsung engineer's
incident
Sensitive Data Leakage
04
Real-World Incidents
Product Company
ChatGPT ban
Copilot’s vulnerability
study
GitHub’s disclaimers
Key Incidents
Examples
Real-World Example: Product Company ChatGPT Data
Breach
05
The Business Risk
Technical debt & CVEs
Legal/IP liability
Compliance failures
Potential Issues
Reputational harm
Risks
06
Balancing Innovation with
Security
Don’t ban—govern Policy and training Shift- left security approach
Strategies
01 02 03
Approach
07
Mitigations & Best Practices
Recommendations
Mandatory code reviews Static/dynamic analysis Secure coding guidelines Defined AI usage policy
Best Practices for AI Code Use
01
Security fundamentals
02
Threat modeling
Training Focus
03
AI limitations
Developer Education
Best Practices to Make AI Code Safer
Define approved
use cases
Restrict sensitive
inputs
Monitor and log
usage
Policies
01
02
03
Governance and Policy
Secret scanning tools
SAST/DAST integration
Dependency checks
Tools
Tooling Support
08
The Future of Secure AI
Coding – Data Leakage &
Privacy
Directions
01.
Research on secure generation
Embedded security guardrails
AI helping fix vulnerabilities
Summary
02.
AI code tools = power + risk
Security must be intentional
Policy, training, and tools
Balance speed with safety
Research
Enterprise Risks: Data Leakage & Privacy
09
Key Takeaways
Power and Risk
AI code tools bring real power to
developers.
They also introduce real risks related
to security and compliance.
Security must be an intentional
aspect of utilizing AI tools.
AI Code Tools
Implement tooling to automatically
review and catch vulnerabilities.
Regular scans can help maintain
security standards.
Automated Reviews and Scanning
Clear policies are needed to define acceptable
AI tool usage.
Developers must be trained to understand the
risks of AI suggestions.
Proper training includes learning how to
mitigate potential security vulnerabilities.
Clear Policies and Training
Best Practices
A company didn’t
ban AI tools after
discovering
security risks.
They created
clear usage
policies and
trained
developers.
Enforced reviews
and scanning
helped achieve
faster development
while maintaining
security.
Balanced Approach
01
02
03
Product Company Example
AI in coding should be governed, not
feared or banned.
Combine policy, training, and
automation to safely enable
innovation.
Safe and Responsible Innovation
Governance and Enablement
10
Q&A / Discussion
Thanks

AI Code Generation Risks (Ramkumar Dilli, CIO, Myridius)

Editor's Notes

  • #36 “What controls are you using for AI code tools in your team?” “What challenges are you seeing?”