HOW TO REVIEW CODE FROM AI AGENTS

Reviewing agent-authored code fails in a predictable way: the reviewer scales their attention to the prompt (“just fix the flaky test”) instead of the diff (nineteen files). Agent diffs are big, plausible, and mostly correct, which is exactly the profile that defeats skim review — the one dangerous change is small and surrounded by legitimate churn. This guide is the workflow that survives that profile. It assumes the agent is a fast, prolific, untrusted contributor, because from a review standpoint that is what it is.

Human PRs tend to be scoped by human energy: people stop when the change feels big. Agents do not get tired; they keep editing until the task appears done. That means review must be re-anchored on mechanical signals — size, sensitive paths, deleted security calls, runtime probes — not on how confident the agent’s summary sounds.

Budget review by diff size, not prompt size

The first discipline is mechanical: decide how long the review takes after you see the diff, never before. A trivial request that produced a 400-line diff gets a 400-line review. Start every review with the shape of the change, not the content:

git diff main...HEAD --stat
git diff main...HEAD --name-status   # A/M/D per file — deletions matter most

Three things to extract before reading a single hunk: files the agent touched that the task didn’t require (scope creep), files it created (new endpoints and handlers had every security decision made from scratch, with no existing pattern to follow), and files or code blocks it deleted (removed code is where security controls silently disappear, and reviewers habitually skip red hunks).

If the diff is too large to review at this standard, that is a finding in itself. Send it back and have the agent split the work — agents re-do work cheaply, and “make this reviewable” is a legitimate iteration. A useful rule of thumb: if you cannot finish a careful review in one sitting, the PR is too big for one merge. Split by layer (schema vs API vs UI) or by feature slice so each PR has a clear security story.

Reading deletions deliberately

Force yourself through the red side of the diff first on any security-relevant file:

git diff main...HEAD -U0 -- 'src/**' | grep '^-' | grep -v '^---' | head -200

Look for removed calls that match your security vocabulary: requireAuth, authorize, csrf, rateLimit, helmet, sanitize, verify, checkOwnership. A single deleted middleware registration can reopen an entire router.

Read the full diff, not the summary

The agent’s summary describes what the agent believes it did. It is generated from the same process that produced any mistake in the diff, so it cannot be evidence about the diff — a session that wrongly disabled a CSRF check will produce a summary that doesn’t mention the CSRF check. The same goes for the commit message and the PR description the agent wrote.

Concretely: review in an actual diff view, hunk by hunk, and never mark an agent PR approved on the basis of its description plus green CI. Tests passing means the code satisfies the assertions that exist — agents optimize for exactly that, sometimes by weakening the assertions. Include test files in the “read fully” set and treat any modified assertion as suspect until explained.

Assertion dilution is a security smell

// Before (agent "fixed" the flaky test)
expect(res.status).toBe(403);

// After
expect([200, 403, 404]).toContain(res.status);

That change can make CI green while deleting the security guarantee. Require the agent (or a human) to explain every loosened assertion in the PR body. If the explanation is “flaky under race,” fix the race or quarantine — do not accept weaker security contracts.

Diff security-sensitive paths against main

Not all files deserve equal attention, and tiering is how a large diff becomes reviewable. Before the general read, isolate the paths where a wrong line is a vulnerability rather than a bug:

git diff main...HEAD -- \
  'src/auth/**' 'src/middleware/**' 'src/api/**' \
  '**/migrations/**' '.github/workflows/**' \
  '.env*' '*.config.*' 'package.json'

If that command prints nothing, the review is a normal code review. If it prints anything, those hunks get line-by-line attention first: every changed condition in an auth path, every middleware ordering change, every workflow edit, every new environment variable. Adapt the path list to your repo once, commit it as a script, and make running it a reflex. Pair it with CODEOWNERS on the same paths so agent edits there always require a named human — the review gate and the merge gate should agree about which files matter.

Suggested path tiers

Agents love to “helpfully” reformat Tier 0 files while doing Tier 3 work. Scope creep into Tier 0 is an automatic request for a split PR.

The two-account BOLA test before merge

The most common serious flaw in agent-generated backends is a CRUD endpoint that authenticates the caller but never checks ownership — BOLA. It is invisible in the diff (the code that’s missing doesn’t appear in red) and invisible to the single-account testing agents and developers naturally do. It takes two accounts and about a minute to catch:

# create a resource as user A
ID=$(curl -s -X POST "$STAGING/api/invoices" \
  -H "Authorization: Bearer $TOKEN_A" \
  -d '{"note":"bola-probe"}' | jq -r .id)

# fetch it as user B — expect 403/404
curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $TOKEN_B" "$STAGING/api/invoices/$ID"

Anything other than 403 or 404 fails the review. Run this against every new or modified :id-style endpoint in the diff — read, update, and delete, not just read. Repeat the write variants (PATCH, DELETE) and, if the API takes foreign keys in request bodies, try submitting user A’s IDs from user B’s session (mass assignment is the sibling flaw). This is the single highest-yield minute in agent-PR review.

Mass assignment sibling check

curl -s -X PATCH "$STAGING/api/users/me" \
  -H "Authorization: Bearer $TOKEN_B" \
  -H "Content-Type: application/json" \
  -d '{"role":"admin","isAdmin":true,"balance":999999}'

If the response or a follow-up GET shows elevated privileges or mutated money fields, reject the PR. Agents frequently spread request bodies into ORM update calls because that matches training examples.

CI gates for disappearing security calls

Human vigilance decays; the gate for removed security controls belongs in CI, where it runs on every PR whether anyone is paying attention or not. The cheapest version is a grep over the removed lines of the diff:

#!/bin/sh
# fail if the PR removes lines containing security-relevant calls
removed=$(git diff origin/main...HEAD -U0 -- . ':!**/*test*' \
  | grep '^-' | grep -v '^---' \
  | grep -icE 'requireauth|authorize|verify\(|csrf|sanitize|ratelimit|helmet|escape\(')
if [ "$removed" -gt 0 ]; then
  echo "Removed security-relevant lines — requires human security sign-off"
  exit 1
fi

Tune the pattern list to your codebase’s actual function names and expect to maintain it. The gate is deliberately dumb: it doesn’t judge whether the removal was correct, it converts a silent removal into a loud conversation. Add a parallel gate for dependency changes (fail the build when package.json gains a dependency, forcing an explicit approval — see dependency risks) and a secret scanner as a required check. Then add a dynamic layer: static gates can’t see missing authorization, so run a runtime probe like the vibe code scanner against the preview deployment before merge.

Example GitHub Actions skeleton

name: agent-pr-gates
on: pull_request
jobs:
  security-diff-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - name: Block silent security removals
        run: ./scripts/check-removed-security-calls.sh
      - name: Secret scan
        run: gitleaks detect --no-git -v || gitleaks detect -v
  preview-dast:
    needs: security-diff-gate
    runs-on: ubuntu-latest
    steps:
      - name: Probe preview
        run: vibeeval scan --url "${{ needs.deploy.outputs.url }}" --fail-on high

Wire the preview URL from your host’s deploy-preview action. The exact CLI may differ; the shape is what matters: static gates then live probe.

When to require human re-implementation

Some diffs should not be repaired in review — they should be rewritten by a person, using the agent’s version as a reference at most. The signal is review economics: when verifying the change costs more than re-implementing it, re-implement. In practice that means:

• Authentication and session logic. Login, password reset, token issuance and verification. The failure modes are subtle (JWT alg-none and kid traversal), and correctness here is worth a human hour.
• Payment and money paths. Anything where a logic bug is a financial event, including webhook handlers and balance updates — agent-generated versions routinely miss signature verification and race protection (race conditions in money paths, Stripe webhooks).
• Cryptography beyond library defaults. If the agent wrote its own token generation, comparison, or encryption wrapper rather than calling the platform primitive, rewrite it.
• Diffs you can’t explain. If after a full read you cannot state what the change does and why each hunk is there, low review confidence is the finding. Don’t merge what you can’t narrate.
• CI and deploy pipeline edits. Agents “fixing” workflows can disable required checks or leak secrets into logs. Prefer human authorship for .github/workflows/**.

The rest — CRUD scaffolding, UI, glue code — is where agents earn their keep, gated by the checks above rather than by rewriting.

Reviewing agent tests and fixtures

Agents generate tests that document the happy path they just built. Extend the review to:

• Whether negative cases exist for auth and validation
• Whether fixtures hardcode secrets or production-like PII
• Whether time-based or random values use secure APIs when security-relevant
• Whether snapshots hide authorization headers or tokens in CI artifacts

Ask the agent explicitly in a follow-up: “Add tests for unauthenticated access, wrong owner, and invalid payload” — then review those tests as carefully as production code. Weak tests are how insecure handlers get a green check.

Dependency and lockfile discipline

When package.json / lockfiles change:

1. Confirm each new package exists on the registry (Package Hallucination Scanner).
2. Prefer known packages already used in the monorepo.
3. Run npm audit / equivalent at high threshold.
4. Reject surprise major upgrades bundled into an unrelated feature PR.
5. Check postinstall scripts on new packages for unexpected network or file access.

Agents sometimes “solve” a type error by adding a utility package from training memory. That is a supply-chain event, not a typing nit.

Secrets in the diff

Before approve:

git diff main...HEAD | grep -E 'sk_(live|test)_|sk-[A-Za-z0-9]{20,}|AKIA[0-9A-Z]{16}|AIza|xox[baprs]-|service_role|BEGIN (RSA |OPENSSH )?PRIVATE KEY' || true

Any hit is rotate-and-rewrite, not “remove from this commit and hope history is fine.” If the secret hit git, treat it as public.

Preview environments as review infrastructure

Static review cannot see:

• Middleware matchers that skip API routes in production-shaped hosts
• RLS that exists in a migration file but was never applied to the linked database
• Env vars that only appear when the host injects NEXT_PUBLIC_ at build time
• Open storage buckets configured in a cloud console

Stand up a preview per agent PR and run the two-account test plus a VibeEval pass against that URL. Approving source while the preview is untested is half a review.

Role-play: the 15-minute agent PR

1. Minute 0–2: --stat / name-status; bounce if huge or unexplained deletions.
2. Minute 2–5: sensitive-path diff; CODEOWNERS awareness.
3. Minute 5–10: full read of remaining hunks including tests.
4. Minute 10–12: two-account BOLA + mass assignment probe on touched routes.
5. Minute 12–15: confirm CI gates green (removed-calls, secrets, deps, preview DAST).

If any step fails, stop — do not “approve with comments” on missing ownership checks. Missing authz is not a follow-up; it is a block.

Common reviewer failure modes

• Anchoring on the prompt: “It only asked for a button” while nineteen files moved.
• Trusting green CI: tests written by the agent about the agent’s code.
• Skipping red hunks: deletions hide removed guards.
• Single-account manual click-through: never surfaces BOLA.
• Rubber-stamping because the agent is usually right: base rates do not protect this PR.
• Deferring security to “we’ll scan later”: later is after the merge when context is gone.

Team policy that makes the workflow stick

Write the rules down so agents and humans share them:

• No merge of agent PRs without sensitive-path review when those paths change
• No production deploy without preview DAST for apps with user data
• Auth/payment/crypto = human implementation or human pair-programming only
• Agents may not disable required status checks
• Task specs for agents must include non-goals and security acceptance checks

Store the policy next to contribution docs. Link it from the PR template:

- [ ] Diff size reviewed (`--stat`)
- [ ] Sensitive paths line-reviewed if present
- [ ] Two-account test on new/changed object routes
- [ ] No weakened security assertions in tests
- [ ] Preview scan attached / CI DAST green

The workflow in one pass

For every agent PR: run the --stat and name-status pass and bounce oversized diffs; read the full diff including tests, ignoring the agent’s summary; isolate and line-review the sensitive-path diff; run the two-account test against every touched object endpoint; let CI enforce the removed-call, dependency, and secret gates; and route auth, payment, and crypto changes to human re-implementation. It’s more process than reviewing a colleague’s two-file PR — because it’s a different thing being reviewed.

Agent speed is only an advantage if review quality scales with it. This workflow is how you keep the speed without importing silent authorization failures into production.

Diff heuristics that catch agent damage

Search every agent PR for:

rg -n "service_role|sk_live|sk-|BEGIN RSA|allow read, write: if true|USING \(true\)|dangerouslySetInnerHTML|eval\(|child_process|cors\(\{ origin: true" 

Also reverse-search for deletions of security controls:

git diff main...HEAD | rg -n "^-.*(requireAuth|verifyJwt|csrf|rateLimit|helmet|row level|RLS)"

If an agent “simplified” middleware away, block the merge even when the feature works.

Review roles

Agents can be authors. They should not be the only reviewer.

Breaking large agent PRs

Require agents to open stacked PRs: schema → API → UI. Mixed mega-diffs hide auth regressions. If the tool only emits one PR, split manually before review.

Negative tests agents must not “fix”

When tests fail on authorization, the wrong fix is loosening the assertion. Protect test files with CODEOWNERS. Prefer property tests: for random user pairs, cross-access returns 403.

Session hygiene for long agent runs

Reset context between unrelated tasks. Pasted production logs in the prompt window become part of the working set. Redact tokens before asking the agent to diagnose production errors.

Related resources

• Security Risks in Agentic AI Coding — the taxonomy this workflow defends against
• Claude Code Security · Cursor Composer Security · Devin Security Practices — per-agent configuration
• BOLA in AI-Generated CRUD — the pattern behind the two-account test
• SAST Tools for AI Code and Between SAST and Pentest — where automated gates end and probing begins
• Secure AI Coding Practices — the prompting layer that reduces what review has to catch
• OWASP Top 10 for AI Code — priority map for what review should hunt first