IMPORTED / AGENTIC-CODING-SECURITY

HOW TO REVIEW CODE FROM AI AGENTS

Reviewing agent-authored code fails in a predictable way: the reviewer scales their attention to the prompt (“just fix the flaky test”) instead of the diff (nineteen files). Agent diffs are big, plausible, and mostly correct, which is exactly the profile that defeats skim review — the one dangerous change is small and surrounded by legitimate churn. This guide is the workflow that survives that profile. It assumes the agent is a fast, prolific, untrusted contributor, because from a review standpoint that is what it is.

Budget review by diff size, not prompt size

The first discipline is mechanical: decide how long the review takes after you see the diff, never before. A trivial request that produced a 400-line diff gets a 400-line review. Start every review with the shape of the change, not the content:

git diff main...HEAD --stat
git diff main...HEAD --name-status   # A/M/D per file — deletions matter most

Three things to extract before reading a single hunk: files the agent touched that the task didn’t require (scope creep), files it created (new endpoints and handlers had every security decision made from scratch, with no existing pattern to follow), and files or code blocks it deleted (removed code is where security controls silently disappear, and reviewers habitually skip red hunks).

If the diff is too large to review at this standard, that is a finding in itself. Send it back and have the agent split the work — agents re-do work cheaply, and “make this reviewable” is a legitimate iteration.

Read the full diff, not the summary

The agent’s summary describes what the agent believes it did. It is generated from the same process that produced any mistake in the diff, so it cannot be evidence about the diff — a session that wrongly disabled a CSRF check will produce a summary that doesn’t mention the CSRF check. The same goes for the commit message and the PR description the agent wrote.

Concretely: review in an actual diff view, hunk by hunk, and never mark an agent PR approved on the basis of its description plus green CI. Tests passing means the code satisfies the assertions that exist — agents optimize for exactly that, sometimes by weakening the assertions. Include test files in the “read fully” set and treat any modified assertion as suspect until explained.

Diff security-sensitive paths against main

Not all files deserve equal attention, and tiering is how a large diff becomes reviewable. Before the general read, isolate the paths where a wrong line is a vulnerability rather than a bug:

git diff main...HEAD -- \
  'src/auth/**' 'src/middleware/**' 'src/api/**' \
  '**/migrations/**' '.github/workflows/**' \
  '.env*' '*.config.*' 'package.json'

If that command prints nothing, the review is a normal code review. If it prints anything, those hunks get line-by-line attention first: every changed condition in an auth path, every middleware ordering change, every workflow edit, every new environment variable. Adapt the path list to your repo once, commit it as a script, and make running it a reflex. Pair it with CODEOWNERS on the same paths so agent edits there always require a named human — the review gate and the merge gate should agree about which files matter.

The two-account BOLA test before merge

The most common serious flaw in agent-generated backends is a CRUD endpoint that authenticates the caller but never checks ownership — BOLA. It is invisible in the diff (the code that’s missing doesn’t appear in red) and invisible to the single-account testing agents and developers naturally do. It takes two accounts and about a minute to catch:

# create a resource as user A
ID=$(curl -s -X POST "$STAGING/api/invoices" \
  -H "Authorization: Bearer $TOKEN_A" \
  -d '{"note":"bola-probe"}' | jq -r .id)

# fetch it as user B — expect 403/404
curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $TOKEN_B" "$STAGING/api/invoices/$ID"

Anything other than 403 or 404 fails the review. Run this against every new or modified :id-style endpoint in the diff — read, update, and delete, not just read. Repeat the write variants (PATCH, DELETE) and, if the API takes foreign keys in request bodies, try submitting user A’s IDs from user B’s session (mass assignment is the sibling flaw). This is the single highest-yield minute in agent-PR review.

CI gates for disappearing security calls

Human vigilance decays; the gate for removed security controls belongs in CI, where it runs on every PR whether anyone is paying attention or not. The cheapest version is a grep over the removed lines of the diff:

#!/bin/sh
# fail if the PR removes lines containing security-relevant calls
removed=$(git diff origin/main...HEAD -U0 -- . ':!**/*test*' \
  | grep '^-' | grep -v '^---' \
  | grep -icE 'requireauth|authorize|verify\(|csrf|sanitize|ratelimit|helmet|escape\(')
if [ "$removed" -gt 0 ]; then
  echo "Removed security-relevant lines — requires human security sign-off"
  exit 1
fi

Tune the pattern list to your codebase’s actual function names and expect to maintain it. The gate is deliberately dumb: it doesn’t judge whether the removal was correct, it converts a silent removal into a loud conversation. Add a parallel gate for dependency changes (fail the build when package.json gains a dependency, forcing an explicit approval — see dependency risks) and a secret scanner as a required check. Then add a dynamic layer: static gates can’t see missing authorization, so run a runtime probe like the vibe code scanner against the preview deployment before merge.

When to require human re-implementation

Some diffs should not be repaired in review — they should be rewritten by a person, using the agent’s version as a reference at most. The signal is review economics: when verifying the change costs more than re-implementing it, re-implement. In practice that means:

  • Authentication and session logic. Login, password reset, token issuance and verification. The failure modes are subtle (JWT alg-none and kid traversal), and correctness here is worth a human hour.
  • Payment and money paths. Anything where a logic bug is a financial event, including webhook handlers and balance updates — agent-generated versions routinely miss signature verification and race protection (race conditions in money paths).
  • Cryptography beyond library defaults. If the agent wrote its own token generation, comparison, or encryption wrapper rather than calling the platform primitive, rewrite it.
  • Diffs you can’t explain. If after a full read you cannot state what the change does and why each hunk is there, low review confidence is the finding. Don’t merge what you can’t narrate.

The rest — CRUD scaffolding, UI, glue code — is where agents earn their keep, gated by the checks above rather than by rewriting.

The workflow in one pass

For every agent PR: run the --stat and name-status pass and bounce oversized diffs; read the full diff including tests, ignoring the agent’s summary; isolate and line-review the sensitive-path diff; run the two-account test against every touched object endpoint; let CI enforce the removed-call, dependency, and secret gates; and route auth, payment, and crypto changes to human re-implementation. It’s more process than reviewing a colleague’s two-file PR — because it’s a different thing being reviewed.

/ NEXT STEP

SCAN YOUR APP

14-day trial. No card. Results in under 60 seconds.

START FREE SCAN →