Security Review
AI-generated code can introduce security vulnerabilities. Some are bugs humans write too; some are unique to AI patterns.
AI-specific security concerns
Section titled âAI-specific security concernsâPattern-based vulnerabilities
Section titled âPattern-based vulnerabilitiesâAI learns from patternsâincluding vulnerable ones in training data.
Common issues:
- SQL/command injection
- Insecure deserialization
- Hardcoded credentials
- Missing auth checks
- Insecure defaults
Why: Models optimize for âcode that looks right,â not âcode thatâs secure.â
Hallucinated security
Section titled âHallucinated securityâAI may implement security incorrectly:
- Encryption using weak algorithms
- Authentication that doesnât actually validate
- Authorization checks that can be bypassed
- Input validation missing edge cases
Code looks secure but isnât.
Dependency risks
Section titled âDependency risksâAI may suggest packages that:
- Donât exist (hallucinatedâcould be typosquatted)
- Have known vulnerabilities
- Are unmaintained
Security review checklist
Section titled âSecurity review checklistâInput handling
Section titled âInput handlingâ- All user inputs validated
- Input length limits enforced
- Paths sanitized (no traversal)
- URLs validated
Authentication & authorization
Section titled âAuthentication & authorizationâ- Auth properly implemented
- Tokens validated correctly
- Sessions managed securely
- Access controls on sensitive operations
- Authorization checked server-side
Data protection
Section titled âData protectionâ- Sensitive data encrypted at rest
- TLS enforced
- Secrets not hardcoded
- No sensitive data in logs
Injection prevention
Section titled âInjection preventionâ- Parameterized database queries
- Shell commands properly escaped
- No eval() with user input
- Template injection prevented
Dependencies
Section titled âDependenciesâ- All dependencies verified to exist
- No known vulnerable versions
- Lock files committed
Review by risk level
Section titled âReview by risk levelâ| Code Type | Review Level |
|---|---|
| Any AI change | Skim for patterns, check inputs, verify deps exist |
| Security-sensitive | Full checklist, manual testing of boundaries |
| Auth/authz | Line-by-line, threat modeling, senior security review |
Automated tools
Section titled âAutomated toolsâStatic analysis (SAST): Run on all code. Wonât catch all AI-specific issues, but catches common vulnerabilities.
Dependency scanning: Dependabot, Snyk. Check all deps exist before installing.
Secret scanning: Pre-commit hooks, repository scanning, CI/CD checks.
Team training
Section titled âTeam trainingâAI-specific:
- Common patterns AI gets wrong
- How to spot hallucinated security
- When to be extra suspicious
General security:
- OWASP Top 10
- Language-specific issues
- Secure coding guidelines
Incident response
Section titled âIncident responseâWhen AI-generated code causes a security issue:
- Treat like any security incidentâdonât minimize because âAI did itâ
- Document AI involvement: Tool, prompt, what review happened
- Root cause: Was this AI-specific? Would human have caught it?
- Process improvement: What would have caught this earlier?
Post-incident: Update checklists, share learnings, consider tool-specific mitigations.
Resources
Section titled âResourcesâ- Government Agents â Mark Myshatyn, Los Alamos - Security considerations for AI agents in government