AI Agent Security Testing for Web Applications
AI agent security testing uses AI agents to attack a running web application the way a human tester does: find the surface, log in, form a guess, try an exploit, and keep the proof. The result is a list of flaws that the agent has shown to be real.
This page explains how an agentic tester works, what it checks well, where it fails, and how to configure one safely. For a view by app architecture (SPA, SSR, edge), see AI pentest for web applications. For a comparison with classic scanners, see DAST vs agentic security testing.
How does an AI agent test a web application?
An agentic tester runs a loop. It observes the app, makes a plan, acts, and reads the result. Then it updates the plan. A human tester does the same thing, but the agent does it at machine speed and on many endpoints at the same time.
The loop has six stages. Each stage gives input to the next one. If a stage is weak, the stages after it are weak too.
| Stage | Question the agent asks | Output |
|---|---|---|
| 1. Recon and crawl | What does this app expose? | Routes, forms, API calls, bundle contents |
| 2. Sessions per role | Who can the agent be? | One logged-in session for each role |
| 3. Hypothesis | What can go wrong here? | A list of specific tests |
| 4. Exploit attempt | Does the flaw work? | Requests and responses |
| 5. Proof | Is the evidence clear? | A finding with a reproducible request |
| 6. Retest | Did the fix work? | A new result for the same check |
The sections below describe each stage in order.
Stage 1: How does the agent find the attack surface?
The agent starts without knowledge of your app. It loads the target in a headless browser and follows links within the scope you set. It records every request the page makes.
The agent also reads what the browser receives. The JavaScript bundle often lists API paths, admin routes, and environment variable names. Source maps can show the original code. A sitemap or an OpenAPI file gives the agent a full list of endpoints.
In VibeEval, a headless Chromium session does this first pass. A Web Reconnaissance Scout agent maps the surface with crawling, fingerprinting, and subdomain enumeration. See AI security agents for the full agent list.
Crawl quality sets the ceiling for the whole test. An agent cannot test a route that it never found. Give the agent a sitemap URL and an OpenAPI URL if you have them.
Stage 2: How does the agent use authenticated sessions per role?
Most serious flaws are behind a login. An agent that tests only the public pages misses them.
To test authorization, the agent needs more than one identity. A typical set is:
- no session (anonymous)
- a normal user, “user A”
- a second normal user, “user B”, in a different account or tenant
- a higher role, such as admin
With these sessions, the agent can ask real access control questions. Can user B read the data of user A? Can a normal user call an admin endpoint? Can an anonymous visitor reach a page that needs a login?
The agent also tests the login flow itself. It examines password reset, magic links, session cookies, and JWT handling. See SaaS privilege escalation for the role matrix method.
In VibeEval, the Authentication option accepts a Bearer token, a cookie, Basic Auth, or a form login. The scanner attaches the credential to every request. To test more than one role, run one scan for each role with the credential for that role.
Stage 3: How does the agent form a hypothesis?
A hypothesis is a specific, testable guess. The agent builds it from what it saw in stages 1 and 2.
Here are three examples:
- The URL
/api/invoices/412has a numeric ID. Guess: the server does not check who owns invoice 412. - A profile update request has the field
name. Guess: the server also accepts the fieldrole. - A coupon endpoint returns an error on the second use. Guess: two requests sent at the same time both succeed.
Each guess maps to a known weakness class, such as those in the OWASP Web Security Testing Guide. The agent chooses guesses that fit the app. A numeric ID leads to an ownership test. A GraphQL endpoint leads to an introspection test.
This is the main difference from a fixed scanner. The agent does not send every payload to every parameter. It picks tests that make sense for the endpoint in front of it.
Stage 4: How does the agent try the exploit?
The agent turns the guess into real requests. It sends them with the correct session and reads the response.
For an ownership test, the steps are:
- Create a resource as user A and note its ID.
- Send a request for that ID with the session of user B.
- Compare the response to the response that user A gets.
- Check if the body contains the data of user A.
The agent can also change course. If the server returns 403, the agent can try a different HTTP method, a different path form, or a different ID format. If the server returns 200 with an empty body, the agent decides that the data is not exposed.
The agent can chain results. A small leak of an internal URL can become the input for a server-side request test.
In VibeEval, specialised agents split this work. The Injection Vulnerability Hunter tests SQL injection and XSS. The Authentication Analyzer tests session and JWT weaknesses. The API Security Prober tests IDOR and mass assignment. The Business Logic Tester tests flow bypass and race conditions.
Stage 5: What counts as proof?
A finding without proof is an opinion. A good agent reports a flaw only after it has shown the flaw.
Good evidence has four parts:
- the exact request that the agent sent
- the exact response that came back
- the identity used (for example, “user B, normal role”)
- a short statement of why the response is a flaw
If the request of user B returned the invoice of user A, anyone can repeat it. The developer can run the same request and see the same result.
Each VibeEval finding includes the captured HTTP request and response that showed the issue. See Findings for the severity levels, the CWE and OWASP mapping, and the fix prompt.
Proof also reduces noise. If the agent cannot show the flaw, it does not report it. On paid plans, VibeEval adds a human review: a security researcher confirms each automated finding and removes false positives. See Running scans.
Stage 6: How does the agent retest a fix?
The retest runs the same check again, with the same sessions, after the developer changes the code.
If the retest still returns the data of user A, the fix did not work. If it now returns 403 or 404, the finding can be closed.
In VibeEval, the Retest action re-runs the single check that produced the finding. The finding shows “Retest queued” until the result arrives. Statuses are open, confirmed, false-positive, and fixed.
Many teams skip this stage. Then the same flaw can return in the next release.
What do AI agents verify well?
Agents are strongest where a test needs context and repetition. Three areas stand out.
Broken object level authorization (BOLA)
OWASP ranks BOLA as the first API security risk. The test needs two sessions and a list of object IDs. An agent makes these easily and tries every endpoint that takes an ID. See AI pentest for APIs for the API view. For one test per class, see test broken access control.
Authentication flows
An agent can walk through login, registration, reset, and logout. It checks if a reset link can be reused, if a token still works after logout, and if a JWT accepts the none algorithm.
Business logic
Business rules are not in any signature database. Examples: a user redeems a coupon twice, a user changes a price in the request, or a user skips a payment step. An agent can read the flow, guess which rule the app should enforce, and try to break it. Results are better for common patterns than for rules that are unique to one company.
Where do AI agents fail?
Agents have real limits. Know them before you trust a report.
| Limit | What happens | What to do |
|---|---|---|
| False negatives | The agent does not find a real flaw | Treat a clean report as “no flaw found on tested paths” |
| Scope gaps | Hidden routes and missing roles are not tested | Give a sitemap, an OpenAPI file, and an account for each role |
| Rare state | Flaws that need a long, specific sequence are missed | Add custom instructions and test by hand |
| Destructive actions | An agent can delete or change real data | Use staging, test accounts, and path exclusions |
| Cost and time | Deep tests use more requests and more time | Run a quick profile on each deploy, a full profile on a schedule you choose |
| Non-repeatable runs | Two runs can take different paths | Keep the evidence and retest the same check |
A black-box agent cannot see a flaw that never shows in a response. See methodology for the limits of black-box testing.
The important point: a report from an agent shows what the agent proved. It does not show what the agent did not try. Each VibeEval scan lists what was out of scope for that pass.
How do I configure an AI security agent safely?
Follow these steps before the first run.
- Get written permission. Test only systems that you own or have permission to test.
- Choose the environment. If you can, deploy a staging copy to a public preview URL and test that first.
- Create test accounts. Make one account for each role. Make two accounts for the normal role, in different tenants.
- Fill the test accounts with fake data. Do not use real customer data.
- Set the scope. Choose single URL, entire domain, or subdomains. Add a sitemap and an OpenAPI URL.
- Exclude destructive paths. Add routes such as
/logout, account deletion, and bulk delete to the excluded paths. - Lower the request rate. Start with a low value and increase it when the app stays stable.
- Run a passive pass first. Passive mode observes and does not inject.
- Start the full run. Watch your logs and your error rate.
- Review the evidence. Retest each fix.
VibeEval supports these controls in the scan wizard: scope, sitemap, OpenAPI URL, authentication, excluded paths, max requests per second (default 50), custom headers, and a Passive only option. Custom headers let you tag scanner traffic in your logs. See Running scans and Targets for details.
Staging or production?
| Factor | Staging | Production |
|---|---|---|
| Risk to real data | Low | Higher |
| Match to real config | Can differ | Exact |
| Traffic in logs and alerts | Separate | Mixed with real users |
| Best use | First run, deep tests | Regular checks after a clean baseline |
If staging differs from production, a flaw can exist in one and not in the other. A cache rule or a CDN setting is a common cause. Test staging first, then run a safer, narrower pass on production.
VibeEval scans are production-safe by default. They probe and do not destroy. The Targets page lists the hosts that the safety filter refuses, such as private addresses and localhost. To test a pre-production build, deploy it to a public preview URL.
How often should I run an agent?
An agent is most useful when it runs often. A flaw that enters in one commit should leave in the next test, not in the next yearly audit.
Use a short profile after each deploy and a deep profile before a major release. For the full release-level approach, see continuous penetration testing. For the always-on view of production, see security monitoring.
What should I do with the results?
Treat the output as a work list, not as a verdict. Fix critical and high findings first. Open the evidence and repeat the request if you want to confirm it. Fix the root cause, because one missing ownership check can cause many findings. Then retest each finding.
The NIST guide to technical security testing (SP 800-115) describes the same cycle: plan, discover, attack, report, and retest. An agent does these steps faster. It does not remove the need for a person to decide what the risk means for the business.
Common questions
What is AI agent security testing?
Can an AI agent replace a human penetration tester?
Will an AI agent damage my production data?
Why does an agent miss some bugs?
Does the agent need my source code?
How is this different from a DAST scanner?
Test with ai agents
VibeEval runs a team of eight specialised agents against your deployed URL. Each finding includes the HTTP request and response that proved it.
14-day free trial · Cancel anytime