Sample alert How it works Coverage PENTEST METHODOLOGY DOCS PRICING FAQ MCP CONTACT LOG IN SCAN MY APP →

How to Verify AI-Generated Code Is Secure

GUIDE / AI SECURITY

To verify that AI-generated code is secure, you need evidence from outside the code: a test that proves the denial paths, a probe of the running app, and scans that the model did not write. A review gives an opinion. Verification gives a result that you can show to another person.

Why is a code review not verification?

A review checks what the code appears to do. The reviewer reads the code, builds a mental model, and decides if the model is safe. This method has two weak points with AI-generated code.

First, the code often looks correct. It uses clean names, follows common patterns, and includes comments. A comment that says “validates the user” does not prove that the code validates the user. Second, the reviewer sees only the diff. The diff does not show a database policy that was never created, a header that the CDN removes, or a key that is bundled into the frontend.

Verification changes the question. It does not ask, “Does this look safe?” It asks, “What result proves that this is safe, and did I see that result?” If you cannot name the result, you have not verified the change.

For the reading side of the work, use the AI Code Review Guide for suggestions in the editor. Use the Agentic Code Review Guide for large diffs from autonomous agents. This page starts where those guides stop. It covers the proof you collect after the reading.

What counts as evidence?

Evidence has three properties. It comes from something other than the code author. It can fail. And a second person can repeat it.

Evidence type What it proves Example
Denial test A forbidden request gets refused User B gets a 403 on the order of user A
Runtime probe The deployed app behaves safely An anonymous request to the admin API returns 401
Registry lookup A dependency is real and trusted The package exists, has history, and has the expected owner
Secret scan No credential is in the code or the bundle The scanner finds no key in the diff or the built JavaScript
Diff check The change did not remove a control No deleted line contains an auth, validation, or policy check

Things that are not evidence: a clean read, a passing happy-path test, a model that says “this is secure”, and a green build that contains no security test.

How do I make sure a test proves something?

A security test is only evidence if it fails when the code is wrong. This is the most common gap in AI-written tests. The model writes the code and the test together. Both can share the same wrong idea.

Use this procedure for each security test.

  1. Write the test so that it asserts a refusal, such as a 403 or an empty result.
  2. Run the test against the unsafe version. Remove the check or use the previous commit.
  3. Make sure the test fails.
  4. Restore the check.
  5. Make sure the test passes.

If step 3 does not fail, the test is not evidence. Rewrite it. This one habit finds many tests that only confirm the happy path. See Security Regression Testing for how to keep these tests permanently.

How do I verify each type of change?

The procedures below are short on purpose. Each one names the evidence that you must collect. Pick the rows that match the change in your pull request.

Change type Evidence to collect
New or changed API route Denial test and runtime probe
Change to authentication or session code No-credential probe and wrong-user probe
New database table or migration Anonymous query on the deployed database
New dependency Registry lookup and install-script check
New configuration or environment code Secret scan of the diff and the built bundle
File upload or storage change Probe with a wrong type, a wrong user, and a guessed URL
Refactor or “cleanup” Diff check for removed controls

How do I verify a new or changed API route?

  1. List the roles that may call the route and the roles that may not.
  2. Write one denial test for each role that may not call it. Include no credentials, a wrong user, and a wrong role.
  3. For a route that uses an object ID, write a test where user B requests an object of user A.
  4. Run the tests against the deployed preview, not only against local code.
  5. Merge only if every test passes.

The probe does not need a framework. This curl command shows the idea:

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $USER_B_TOKEN" \
  "$PREVIEW_URL/api/invoices/$USER_A_INVOICE_ID"

If the result is 200, the route has a BOLA flaw. The expected result is 403 or 404. See Test Broken Access Control for one test for each access control class. The OWASP API Security Top 10 ranks this flaw first.

How do I verify a change to authentication?

An AI tool can change login or session code and break a rule that nobody asked it to change. Test the rules, not the feature.

  1. Send a request to a protected route with no credentials. Expect 401.
  2. Send a request with an expired token. Expect 401.
  3. Send a request with a token of a lower role. Expect 403.
  4. Send a request with a token that has a changed signature. Expect 401.
  5. Check that logout makes the old token invalid, if your app promises this.

If you use JWTs, see JWT Security Checker for the common token flaws. Also test the API directly. A frontend route guard does not protect the API behind it.

How do I verify a new database table or migration?

Row Level Security is the typical failure here. A model creates a table and the code works in development, because the developer uses a privileged role. In production, the public key can read every row.

  1. Make sure the migration enables RLS on the new table.
  2. Put one known row in the table, owned by user A.
  3. Query the table through the public REST API with the anon key. Expect no rows.
  4. Query as user B. Expect no row of user A.
  5. Query as user A. Expect the row. This proves the policy is not just blocking everything.

Run steps 3 and 4 on the deployed database. A policy that exists in the migration file but did not run is a common cause of failure. See Supabase RLS Checker for a free scan of the same flaw.

How do I verify a new dependency?

AI tools sometimes suggest packages that do not exist. An attacker can register such a name and put harmful code in it. This attack is often called slopsquatting. Do not install a new package before you check it.

  1. Look up the name in the official registry. For npm, run npm view <name>.
  2. Check the owner, the first publish date, the download count, and the repository link.
  3. Compare the name with the package you expected. A name with one changed letter is a warning.
  4. Check if the package has install scripts. For npm, look at the scripts field in npm view <name> scripts.
  5. If the package is new and has little history, find a better known package that does the same job.
  6. Commit the lockfile, and install from the lockfile in CI.

For an automated check, see the Package Hallucination Scanner. The npm documentation describes the lookup command. A real package can also have a known vulnerability, so run your normal dependency audit as well.

How do I verify that no secret was added?

A model that wires up an API often puts the key in the file to make the example work. Check the diff and the built output.

  1. Run a secret scanner on the diff before you commit. Gitleaks is one open-source option.
  2. Build the app and search the output bundle for key patterns, such as sk_live_, service_role, and AKIA.
  3. Make sure that only public keys use the frontend prefix, such as NEXT_PUBLIC_ or VITE_.
  4. Check that the .env file is in .gitignore.
  5. If a scanner finds a real key, revoke it. Removing the line does not remove the key from the Git history.

Search the deployed bundle too. A secret that is not in the repository can still be in the output. See Token Leak Checker, Env Exposure Checker, and Supabase Service Role Key.

How do I verify a file upload or storage change?

  1. Upload a file with a wrong type, for example an HTML file named as an image. Expect a refusal or safe handling.
  2. Upload a file larger than the limit. Expect a refusal.
  3. Request a file of user A as user B. Expect 403 or 404.
  4. Request the file URL with no credentials. Expect a refusal if the file is private.
  5. Change one character in the file URL. Expect a not-found result.
  6. List the bucket through the public API. Expect no list.

How do I find a removed security control?

AI-generated refactors remove controls. The model sees a check as extra code and deletes it, and all functional tests still pass. Look for deleted lines, not only for added lines.

  1. Run git diff and list the deleted lines.
  2. Search the deleted lines for these words: auth, permission, role, owner, tenant, sanitize, validate, csrf, policy, rate, limit.
  3. For each match, find where the same check now exists. If you cannot find it, the control is gone.
  4. Check the removed lines in config files, middleware, and migrations too.
  5. Run your regression tests. A good test fails when a control is removed.

You can automate step 2 in CI. A simple script can fail the build and ask for a manual approval when a deleted line matches the list. This does not replace the tests, but it points a person to the right lines. For the broader static layer, see SAST Tools for AI Code.

How do I probe the deployed app?

Local tests and the diff cannot see the deployed environment. Many security flaws come from configuration: a policy that did not run, a header that the CDN changed, or a storage bucket with a public setting. A runtime probe finds these.

Use this procedure after each deploy to a preview or staging URL.

  1. Create two test accounts in two different tenants.
  2. Request every changed route with no credentials. Expect a refusal on private routes.
  3. Request the objects of one account with the other account. Expect a refusal.
  4. Read the response headers and make sure the security headers are present. See Security Headers Checker.
  5. Search the deployed JavaScript for secrets.
  6. Check that debug routes and error pages do not show stack traces.

A scanner can run these probes for you. VibeEval tests the live URL with AI agents and attaches the HTTP request and response to each finding. This gives you the evidence for the change without a manual script. See Findings for the format. A free first check is available in the Vibe Code Scanner.

What do I do when verification fails?

A failed check is useful. It means the process worked. Do not ask the model to “fix it” and merge without checking again.

  1. Keep the failing evidence. Save the request, the response, or the scan output.
  2. Tell the model the exact request and the exact result. Do not say only “make it secure”.
  3. Apply the fix.
  4. Run the same check again. The same evidence must now show a safe result.
  5. Add the check to the permanent test suite.

Step 5 is the link between this guide and regression testing. A flaw that you verified once can return in the next AI-generated change.

How much verification does a change need?

Use the risk of the change. A text change does not need a probe. A change that touches data access, identity, payments, files, or dependencies needs the procedures above. When you are not sure, treat the change as high risk. For the types of flaws that AI tools produce most often, see AI Code Vulnerabilities and Secure AI Coding Practices.

Common questions

What is the difference between reviewing and verifying AI-generated code?
A review is an opinion. A person reads the code and decides that it looks safe. Verification is evidence. A test, a probe, or a scan produces a result that you can show to another person. Review can miss a flaw. A failing test cannot.
Can an AI tool verify its own code?
Not as the only check. A model that wrote a flaw can also write a test that does not find it. Make sure the test fails on a known bad version of the code. Use tools that the model did not write, such as a secret scanner, a registry lookup, and a probe of the running app.
How do I find a hallucinated package?
Look up each new dependency in the official registry. Check that the name exists, who owns it, when it was first published, and how many people use it. A new package with a name that matches the model's suggestion and almost no history needs a manual check before you install it.
What is the fastest check for a change that touches authentication?
Send a request with no credentials and a request with the wrong user's credentials to the changed route in the deployed app. If either request returns data, the change is not safe. This takes minutes and needs no code reading.
Do I need to verify every change?
Match the effort to the risk. A change to text or styling needs a normal build. A change to authentication, data access, payments, file handling, or dependencies needs the procedures in this guide.
Where do verification tests run?
Run unit tests in the pull request. Run runtime probes after the deploy, against the live URL. Many flaws, such as a missing database policy, only appear in the deployed environment.

Probe the deployed app

VibeEval tests your live URL with AI agents and shows the HTTP request and response as evidence for each finding. Use it as the runtime proof step.

14-day free trial · Cancel anytime

START FREE SCAN →