Auditing AI-Generated Code: A Security-Focused Review Framework

Auditing AI-generated code means treating every model-produced diff as untrusted input until proven otherwise: run static analysis and dependency scanning before human review, check for hallucinated packages and missing auth checks, and require the same (or stricter) sign-off as code from a first-week contractor. The framework below covers what to automate, what to inspect manually, and where teams get burned.

Why AI-generated code needs a different review posture

Code review built for human pull requests assumes a baseline: the author understood the codebase's conventions, probably ran the code locally, and has some accountability tied to their name. None of that holds for a Copilot suggestion, a Claude Code agent run, or a Cursor tab-complete. The model has no memory of your last incident postmortem, no awareness that your auth middleware has a known edge case, and no skin in the game if the code ships broken.

The failure modes are also different in kind, not just frequency. Human developers write insecure code out of ignorance or time pressure, and the mistakes tend to cluster around known weak points: missing input validation, off-by-one errors, forgotten edge cases. LLMs produce those same mistakes, but they add a new category: statistically plausible but non-existent APIs, package names, and configuration flags. GitHub's own research on Copilot and secure coding has pushed the ecosystem toward pairing AI suggestions with automated security tooling rather than trusting either the model or the reviewer alone (see GitHub's documentation on Copilot code review).

Treat AI output as a contribution from a fast, well-read, overconfident junior engineer who has never been fired for being wrong. That mental model changes what you check and how much you automate.

What actually breaks in practice

Four patterns show up repeatedly across teams shipping AI-assisted code to production:

Hallucinated dependencies. Models suggest package names that sound right but don't exist, or that exist but are typosquatted malware. This is common enough that it has a name, "slopsquatting," and it is one of the few AI-specific supply chain risks with no human-error equivalent. A pip install or npm install run against a hallucinated name can pull in an attacker-controlled package if that name happens to be registered.

Confidently wrong auth and authorization logic. Ask an assistant to "add an endpoint that lets users fetch their own profile" and you'll often get a working GET /profile/:id with no check that the requesting user matches :id. The code compiles, the happy path works in a demo, and the broken access control ships straight to production. This maps directly onto the OWASP Top 10 category for broken access control, consistently the most reported real-world vulnerability class (see OWASP Top 10).

Secrets and config drift. Generated code frequently hardcodes example values, API keys copied from a public snippet in training data, or default credentials meant as placeholders. It also tends to over-permission: an S3 bucket policy or IAM role generated to "make it work" will often grant wider access than the task needs, because narrow scoping requires context the model doesn't have.

Silent downgrade of security-relevant defaults. Ask for "a JWT verification function" and depending on prompt phrasing you can get one that skips signature verification, doesn't check expiration, or accepts the none algorithm. Each of these is a known, well-documented JWT pitfall, but the model reproduces training data that includes both correct and incorrect examples at roughly the same statistical weight.

Building the review pipeline: what to automate first

Automate before you ask a human to look at anything. The cost of a false negative in automated scanning is far lower than the cost of a human reviewer's attention being wasted on issues a linter should have caught.

A practical pipeline order:

• Static analysis (SAST) on every AI-touched diff, not just human-authored ones. Tools like Semgrep or CodeQL catch injection patterns, unsafe deserialization, and known-bad function usage without needing to understand intent.
• Dependency and SBOM scanning immediately after any new import or package addition. This is the direct countermeasure to hallucinated or typosquatted packages. If the package was added or suggested by an AI tool in the same session, flag it for manual verification against the actual registry, not just a scan for known CVEs.
• Secret scanning on the diff, not the whole repo, so it runs fast enough to sit in CI without adding meaningful latency. GitHub's push protection and secret scanning documentation covers the built-in version most teams already have access to (see GitHub secret scanning docs).
• License and provenance checks if the AI tool's training data or output policy is unclear about license contamination risk. This matters more for larger code blocks (20+ lines lifted near-verbatim) than for boilerplate.

A minimal Semgrep rule to catch one of the more common Copilot-era mistakes, missing ownership checks on ID-based lookups, looks like this:

1rules:
2 - id: missing-owner-check-on-lookup
3 languages: [python]
4 severity: WARNING
5 message: >
6 Endpoint fetches a resource by ID without verifying the
7 requesting user owns or is authorized to access it.
8 patterns:
9 - pattern: |
10 def $FUNC(...):
11 ...
12 $OBJ = $MODEL.objects.get(id=$ID)
13 ...
14 - pattern-not: |
15 def $FUNC(...):
16 ...
17 $OBJ = $MODEL.objects.get(id=$ID, owner=$USER)
18 ...

This won't catch every access control bug, and it will produce false positives on legitimately public resources. That's fine. The goal is to force a human decision point, not to replace the decision.

How do you review code you didn't write and the model didn't test?

This is the actual hard part, and it's where most teams under-invest. Reviewing AI-generated code requires more context reconstruction than reviewing a colleague's PR, because you can't ask the author why they made a choice. There's no "why" to retrieve, only a statistical artifact.

Practical adjustments to a review checklist:

• Trace every external input to its use. Don't assume validation happened upstream just because the code "looks" defensive. Models often add validation-shaped code (a type hint, a docstring claiming sanitization) without the actual check.
• Verify every new dependency against the real registry, by name and by version, before merge. A 30-second npm view <package> or pip index versions <package> check is cheap insurance against slopsquatting.
• Read error handling for what it hides, not what it catches. AI-generated try/except blocks frequently swallow exceptions broadly (except Exception: pass) to make demo code "work," which is exactly the kind of failure mode that turns a minor bug into a silent security gap.
• Check that tests were actually generated and actually assert something. It's common to get a test file that imports the module, calls the function once, and asserts True or checks that no exception was raised, no assertions on actual output. This gives a false sense of coverage in CI dashboards.
• Diff against the framework's documented secure pattern, not against "does this look reasonable." For anything touching auth, crypto, or deserialization, pull up the framework's own security guidance (Django's, Rails', Spring Security's) and compare line by line rather than relying on intuition.

Cost and latency trade-offs teams actually hit

Running full SAST plus dependency scanning plus secret scanning on every commit, including every AI-assisted keystroke-level suggestion accepted in an editor, is not free. Teams that try to scan at the IDE-suggestion level (every Copilot or Cursor completion) rather than at the commit or PR level usually abandon it within weeks because of latency: a 2 to 5 second SAST pass per suggestion breaks flow state fast, and most engineers turn it off if given the option.

The workable middle ground: let the AI tool run unscanned in the editor for speed, then gate at the PR level where a few extra seconds or even a couple of minutes of CI time is acceptable. This means accepting that insecure code will exist transiently on a developer's machine, which is fine, and catching it before merge, which is not optional.

For teams using agentic tools that make multi-file changes autonomously, such as Claude Code or similar agent frameworks, the calculus shifts again: a single agent run might touch 15 files, and reviewing that as one unit rather than file-by-file is usually faster and catches more systemic issues (a auth check missing consistently across five new endpoints, for instance) than a per-file review. Anthropic's documentation on Claude Code's permission and review model is a useful reference for how agent-initiated changes can be scoped and gated before they touch a real branch (see Claude Code documentation).

When this framework is overkill

Not every AI-generated line needs this treatment. A one-off internal script that transforms a CSV and runs once on a laptop doesn't need SAST, SBOM scanning, and a manual auth trace. Apply the full framework to anything that touches: user data, authentication or authorization, payment or financial logic, external network calls, or anything that will run unattended in production. For internal tooling, prototypes, and throwaway scripts, a basic secret scan and a sanity read-through is proportionate. Over-applying heavy process to low-risk code is how security review programs lose credibility and get bypassed.

Adoption path for a team starting from zero

Start with secret scanning and dependency verification in CI this week, since both are cheap, fast, and catch the two failure modes most unique to AI-generated code (hardcoded secrets and hallucinated packages). Add SAST rules targeted at your framework's known weak points (auth checks, deserialization, SSRF) over the next sprint rather than adopting a generic ruleset wholesale, since generic rules generate noise that trains reviewers to ignore warnings. Once the automated layer is stable, update your PR template to require an explicit statement of which parts of a diff were AI-generated or AI-assisted, not to shame contributors but to tell reviewers where to spend the extra five minutes of manual scrutiny. Revisit the pipeline every quarter, because both the models and their failure modes change faster than most internal security processes do, and a ruleset tuned for last year's Copilot behavior may miss what this year's agentic tools get wrong.