Sample alert How it works Coverage PENTEST METHODOLOGY DOCS PRICING FAQ MCP CONTACT LOG IN SCAN MY APP →

AI Agent Security Testing for Web Applications

AI PENTEST / AGENTS

AI agent security testing uses AI agents to attack a running web application the way a human tester does: find the surface, log in, form a guess, try an exploit, and keep the proof. The result is a list of flaws that the agent has shown to be real.

This page explains how an agentic tester works, what it checks well, where it fails, and how to configure one safely. For a view by app architecture (SPA, SSR, edge), see AI pentest for web applications. For a comparison with classic scanners, see DAST vs agentic security testing.

How does an AI agent test a web application?

An agentic tester runs a loop. It observes the app, makes a plan, acts, and reads the result. Then it updates the plan. A human tester does the same thing, but the agent does it at machine speed and on many endpoints at the same time.

The loop has six stages. Each stage gives input to the next one. If a stage is weak, the stages after it are weak too.

Stage Question the agent asks Output
1. Recon and crawl What does this app expose? Routes, forms, API calls, bundle contents
2. Sessions per role Who can the agent be? One logged-in session for each role
3. Hypothesis What can go wrong here? A list of specific tests
4. Exploit attempt Does the flaw work? Requests and responses
5. Proof Is the evidence clear? A finding with a reproducible request
6. Retest Did the fix work? A new result for the same check

The sections below describe each stage in order.

Stage 1: How does the agent find the attack surface?

The agent starts without knowledge of your app. It loads the target in a headless browser and follows links within the scope you set. It records every request the page makes.

The agent also reads what the browser receives. The JavaScript bundle often lists API paths, admin routes, and environment variable names. Source maps can show the original code. A sitemap or an OpenAPI file gives the agent a full list of endpoints.

In VibeEval, a headless Chromium session does this first pass. A Web Reconnaissance Scout agent maps the surface with crawling, fingerprinting, and subdomain enumeration. See AI security agents for the full agent list.

Crawl quality sets the ceiling for the whole test. An agent cannot test a route that it never found. Give the agent a sitemap URL and an OpenAPI URL if you have them.

Stage 2: How does the agent use authenticated sessions per role?

Most serious flaws are behind a login. An agent that tests only the public pages misses them.

To test authorization, the agent needs more than one identity. A typical set is:

  • no session (anonymous)
  • a normal user, “user A”
  • a second normal user, “user B”, in a different account or tenant
  • a higher role, such as admin

With these sessions, the agent can ask real access control questions. Can user B read the data of user A? Can a normal user call an admin endpoint? Can an anonymous visitor reach a page that needs a login?

The agent also tests the login flow itself. It examines password reset, magic links, session cookies, and JWT handling. See SaaS privilege escalation for the role matrix method.

In VibeEval, the Authentication option accepts a Bearer token, a cookie, Basic Auth, or a form login. The scanner attaches the credential to every request. To test more than one role, run one scan for each role with the credential for that role.

Stage 3: How does the agent form a hypothesis?

A hypothesis is a specific, testable guess. The agent builds it from what it saw in stages 1 and 2.

Here are three examples:

  • The URL /api/invoices/412 has a numeric ID. Guess: the server does not check who owns invoice 412.
  • A profile update request has the field name. Guess: the server also accepts the field role.
  • A coupon endpoint returns an error on the second use. Guess: two requests sent at the same time both succeed.

Each guess maps to a known weakness class, such as those in the OWASP Web Security Testing Guide. The agent chooses guesses that fit the app. A numeric ID leads to an ownership test. A GraphQL endpoint leads to an introspection test.

This is the main difference from a fixed scanner. The agent does not send every payload to every parameter. It picks tests that make sense for the endpoint in front of it.

Stage 4: How does the agent try the exploit?

The agent turns the guess into real requests. It sends them with the correct session and reads the response.

For an ownership test, the steps are:

  1. Create a resource as user A and note its ID.
  2. Send a request for that ID with the session of user B.
  3. Compare the response to the response that user A gets.
  4. Check if the body contains the data of user A.

The agent can also change course. If the server returns 403, the agent can try a different HTTP method, a different path form, or a different ID format. If the server returns 200 with an empty body, the agent decides that the data is not exposed.

The agent can chain results. A small leak of an internal URL can become the input for a server-side request test.

In VibeEval, specialised agents split this work. The Injection Vulnerability Hunter tests SQL injection and XSS. The Authentication Analyzer tests session and JWT weaknesses. The API Security Prober tests IDOR and mass assignment. The Business Logic Tester tests flow bypass and race conditions.

Stage 5: What counts as proof?

A finding without proof is an opinion. A good agent reports a flaw only after it has shown the flaw.

Good evidence has four parts:

  • the exact request that the agent sent
  • the exact response that came back
  • the identity used (for example, “user B, normal role”)
  • a short statement of why the response is a flaw

If the request of user B returned the invoice of user A, anyone can repeat it. The developer can run the same request and see the same result.

Each VibeEval finding includes the captured HTTP request and response that showed the issue. See Findings for the severity levels, the CWE and OWASP mapping, and the fix prompt.

Proof also reduces noise. If the agent cannot show the flaw, it does not report it. On paid plans, VibeEval adds a human review: a security researcher confirms each automated finding and removes false positives. See Running scans.

Stage 6: How does the agent retest a fix?

The retest runs the same check again, with the same sessions, after the developer changes the code.

If the retest still returns the data of user A, the fix did not work. If it now returns 403 or 404, the finding can be closed.

In VibeEval, the Retest action re-runs the single check that produced the finding. The finding shows “Retest queued” until the result arrives. Statuses are open, confirmed, false-positive, and fixed.

Many teams skip this stage. Then the same flaw can return in the next release.

What do AI agents verify well?

Agents are strongest where a test needs context and repetition. Three areas stand out.

Broken object level authorization (BOLA)

OWASP ranks BOLA as the first API security risk. The test needs two sessions and a list of object IDs. An agent makes these easily and tries every endpoint that takes an ID. See AI pentest for APIs for the API view. For one test per class, see test broken access control.

Authentication flows

An agent can walk through login, registration, reset, and logout. It checks if a reset link can be reused, if a token still works after logout, and if a JWT accepts the none algorithm.

Business logic

Business rules are not in any signature database. Examples: a user redeems a coupon twice, a user changes a price in the request, or a user skips a payment step. An agent can read the flow, guess which rule the app should enforce, and try to break it. Results are better for common patterns than for rules that are unique to one company.

Where do AI agents fail?

Agents have real limits. Know them before you trust a report.

Limit What happens What to do
False negatives The agent does not find a real flaw Treat a clean report as “no flaw found on tested paths”
Scope gaps Hidden routes and missing roles are not tested Give a sitemap, an OpenAPI file, and an account for each role
Rare state Flaws that need a long, specific sequence are missed Add custom instructions and test by hand
Destructive actions An agent can delete or change real data Use staging, test accounts, and path exclusions
Cost and time Deep tests use more requests and more time Run a quick profile on each deploy, a full profile on a schedule you choose
Non-repeatable runs Two runs can take different paths Keep the evidence and retest the same check

A black-box agent cannot see a flaw that never shows in a response. See methodology for the limits of black-box testing.

The important point: a report from an agent shows what the agent proved. It does not show what the agent did not try. Each VibeEval scan lists what was out of scope for that pass.

How do I configure an AI security agent safely?

Follow these steps before the first run.

  1. Get written permission. Test only systems that you own or have permission to test.
  2. Choose the environment. If you can, deploy a staging copy to a public preview URL and test that first.
  3. Create test accounts. Make one account for each role. Make two accounts for the normal role, in different tenants.
  4. Fill the test accounts with fake data. Do not use real customer data.
  5. Set the scope. Choose single URL, entire domain, or subdomains. Add a sitemap and an OpenAPI URL.
  6. Exclude destructive paths. Add routes such as /logout, account deletion, and bulk delete to the excluded paths.
  7. Lower the request rate. Start with a low value and increase it when the app stays stable.
  8. Run a passive pass first. Passive mode observes and does not inject.
  9. Start the full run. Watch your logs and your error rate.
  10. Review the evidence. Retest each fix.

VibeEval supports these controls in the scan wizard: scope, sitemap, OpenAPI URL, authentication, excluded paths, max requests per second (default 50), custom headers, and a Passive only option. Custom headers let you tag scanner traffic in your logs. See Running scans and Targets for details.

Staging or production?

Factor Staging Production
Risk to real data Low Higher
Match to real config Can differ Exact
Traffic in logs and alerts Separate Mixed with real users
Best use First run, deep tests Regular checks after a clean baseline

If staging differs from production, a flaw can exist in one and not in the other. A cache rule or a CDN setting is a common cause. Test staging first, then run a safer, narrower pass on production.

VibeEval scans are production-safe by default. They probe and do not destroy. The Targets page lists the hosts that the safety filter refuses, such as private addresses and localhost. To test a pre-production build, deploy it to a public preview URL.

How often should I run an agent?

An agent is most useful when it runs often. A flaw that enters in one commit should leave in the next test, not in the next yearly audit.

Use a short profile after each deploy and a deep profile before a major release. For the full release-level approach, see continuous penetration testing. For the always-on view of production, see security monitoring.

What should I do with the results?

Treat the output as a work list, not as a verdict. Fix critical and high findings first. Open the evidence and repeat the request if you want to confirm it. Fix the root cause, because one missing ownership check can cause many findings. Then retest each finding.

The NIST guide to technical security testing (SP 800-115) describes the same cycle: plan, discover, attack, report, and retest. An agent does these steps faster. It does not remove the need for a person to decide what the risk means for the business.

Common questions

What is AI agent security testing?
It is the use of AI agents to test a running web application for security flaws. An agent plans a test, sends requests, reads the responses, and decides what to try next. It reports a finding only when the evidence shows the flaw is real.
Can an AI agent replace a human penetration tester?
No, not for all work. An agent is good at repeated, wide tests, such as access control checks on every endpoint. A human tester is better at new attack ideas and at deep knowledge of your business. Many teams use agents on every release and a human tester once a year.
Will an AI agent damage my production data?
It can, if you give it write access and no limits. Use a staging copy when you can. In production, use test accounts, exclude destructive paths, lower the request rate, and use a passive mode for the first run.
Why does an agent miss some bugs?
An agent only tests what it finds and what it decides to try. If a route is hidden, if a role has no test account, or if a flaw needs a rare state, the agent can miss it. A clean report shows that the tested paths are safe. It does not show that the whole app is safe.
Does the agent need my source code?
No. A black-box agent works from the deployed app, the same view an attacker has. It loads the app in a browser, reads the JavaScript bundle, and sends requests to the endpoints it finds.
How is this different from a DAST scanner?
A DAST scanner sends a fixed set of payloads and matches the responses to known patterns. An agent chooses its next request from the last response and can hold several sessions. See the DAST vs agentic security testing comparison for the full difference.

Test with ai agents

VibeEval runs a team of eight specialised agents against your deployed URL. Each finding includes the HTTP request and response that proved it.

14-day free trial · Cancel anytime

START FREE SCAN →