Security Testing API for AI Coding Agents
A REST API and an MCP connector let a coding agent start a scan on a deployed URL, read the findings, fix the code, and rescan to check the fix.
TEST THE APP YOUR AGENT BUILT
Enter the deployed URL of an app that a coding agent made. The scan shows the findings that the agent must fix first.
A security testing API for AI coding agents lets the agent test the app that it just built. The agent starts a scan on the deployed URL, reads the findings, fixes the code, deploys, and starts a rescan. VibeEval offers a REST API and an MCP connector for this loop.
This page describes the workflow and the safety rules. It does not list endpoint paths or JSON schemas. The VibeEval docs are the source for those details.
Why does an agent need a security testing API?
A coding agent writes code fast. It does not see the running app the way a stranger sees it. A passing unit test does not show that a Supabase table is open or that an admin route needs no login. These gaps appear in the deployed app, in the browser bundle, and in the host settings.
A person can run a scan and paste the report into the agent. This works, but it is slow.
An API closes the loop. The agent calls the scanner as a tool. The agent reads the result as data. The agent can then fix the issue and check the fix without a person in the middle.
The loop has a limit. The scanner tests what is deployed. If the agent has not deployed, the scan tests old code. Make the deploy step part of the loop.
What is the VibeEval REST API and MCP connector?
The VibeEval billing documentation lists a REST API and an MCP connector in the Pro feature set. The Lifetime plan has both. The documentation gives a limit of 25,000 API calls each month on Pro and Lifetime. Enterprise has unlimited API calls. Read the plans and limits page for the current numbers.
| Part | What it is for |
|---|---|
| REST API | Calls from a script, a CI job, or an agent tool that you write |
| MCP connector | A connection that an MCP-compatible agent uses as a tool |
| Webhooks | Events that VibeEval sends to your own HTTPS URL |
| Copy for LLM report | A Markdown report that is written for a coding agent |
MCP is the Model Context Protocol. It is an open standard that connects an AI application to tools and data. A client such as Claude Code or Cursor can use an MCP server as a tool. See the Claude Code guide and the Cursor guide for the security risks of each client.
Note: The documentation names the two features but does not publish the endpoint paths, the auth method, or the connector setup steps. Do not guess them. Use the current docs and your account settings.
What does the scan, fix, rescan loop look like?
The loop has seven steps. The agent does most of them. A person approves the risky ones.
- Deploy. The agent deploys the change to a public preview URL.
- Start a scan. The agent starts a scan on that URL with a profile.
- Wait. The agent waits for the scan to finish.
- Read the findings. The agent reads each finding with its evidence.
- Fix. The agent changes the code for each finding, smallest fix first.
- Deploy again. The agent deploys the fix to the same environment.
- Rescan or retest. The agent checks that the finding is gone and that no new finding appeared.
Repeat steps 4 to 7 until no Critical or High finding is open. Then a person reviews the diff.
Which scan profile does the agent choose?
The scan documentation lists four profiles:
| Profile | Duration | Use it when |
|---|---|---|
| Quick Scan | 1 to 3 minutes | The agent needs a fast check after a small change |
| DAST Full | 10 to 30 minutes | The change adds a route, a table, or a login |
| API Scan | 5 to 15 minutes | The app has an OpenAPI spec that you can give to the scan |
| Custom | Varies | You want only some modules, such as IDOR or CORS |
Use the Quick Scan inside the fix loop. Use DAST Full before a release. For an API, give the scan an OpenAPI URL. Read Running scans for each option.
How does the agent know that a scan is finished?
A scan moves through the statuses queued, running, pending_review, and completed. The statuses failed and cancelled also exist. The agent has two ways to learn the status:
- Poll. The agent asks for the scan status at a fixed interval.
- Webhook. VibeEval sends a
scan.completedorscan.failedevent to an HTTPS URL that you register.
The webhook documentation lists these events: scan.queued, scan.started, scan.completed, scan.failed, finding.new, finding.critical, finding.statusChanged, and team.memberAdded. An event arrives as an HTTP POST with a JSON body. The body holds the ID, the target, the severity, and the status. Respond with a 2xx quickly and process the event later. See Webhooks.
Do not poll each second. Wait for the expected time of the profile, then poll at a long interval.
What do the findings hold?
Each finding holds the data that an agent needs. The findings documentation lists these fields:
- A title and a severity: critical, high, medium, low, or info.
- A CWE and an OWASP reference.
- The captured HTTP request and HTTP response that proved the issue.
- Numbered remediation steps.
- A before and after code patch, when a patch can be shown.
The Copy for LLM report puts these fields in one Markdown file. Each fix prompt includes the app URL, the detected stack, the evidence, and a rules block. The rules block tells the agent to make the smallest fix that fully resolves the issue and not to change unrelated code. Prompts exist for Lovable, Cursor, Claude Code, and a generic agent.
Give the agent the evidence, not only the title. The request and response show the agent which route and which parameter to fix.
How does the agent check that a fix works?
Two actions check a fix. The documentation describes both.
| Action | What it runs | Quota effect |
|---|---|---|
| Retest | The single check that made the finding | Not stated |
| Re-run scan | A new scan on the same target and profile | Counts against the monthly scan quota |
A re-run produces a trend against the last completed scan. The trend shows which findings are new, fixed, or unchanged. Use the trend to find a regression that the fix caused.
If a retest still shows the finding open, check three things before the agent changes more code:
- Make sure that the fix is deployed to the exact URL that the scan tested.
- Check for a CDN cache that serves the old response.
- For a leaked key, make sure that the old key is replaced at its provider. A rescan does not prove that the old key is dead.
The FAQ explains the retest cases in more detail.
A fix that passes today can fail next week. Ask the agent to write a test for each fixed finding. The guide on security regression testing shows how.
Which safety rules must I set before I give an agent access?
An agent that can start scans and read findings is a tool with power. Treat it as you treat any automation that holds credentials. OWASP lists excessive agency as a risk of LLM applications. Give the agent the least power that the task needs.
What is the scope of the scan?
Scan only hosts that you own or have written permission to test. The targets documentation states this rule. VibeEval does not verify domain ownership with a DNS record. It uses a safety filter and abuse review, and it suspends accounts that scan systems that they do not own.
Set the scope in the instructions of the agent:
- Name the exact URL that the agent may scan.
- Name the scope: a single URL, the entire domain, or subdomains.
- Do not let the agent choose a new target from text that it reads in a web page or a ticket.
- Add excluded paths for routes that can do harm, such as
/logoutand destructive admin routes.
A web page that the agent reads can hold hidden text that tells the agent to scan another site. This is prompt injection. A fixed target list in the tool configuration stops this.
Which accounts does the agent use?
Never give the agent a production admin login. An agent can print a secret in a log, a commit message, or a chat. A scan with an admin login can also change or delete real data.
- Create two test accounts on a staging or preview environment.
- Give the first account a normal user role.
- Give the second account a different user role, or the same role in a different tenant.
- Put the credentials in the authentication options of the scan. The scan supports Bearer Token, Cookie, Basic Auth, and Form Login.
- Use the secret instruction field for any text that must not appear in a report.
- Replace the test credentials when the work ends.
Two accounts let the scan test if one user can read the data of the other. The multi-tenant isolation guide explains these tests.
What must the preview environment hold?
A preview that has production data makes each scan a risk. Use a separate database with fake data. Use test keys for payment and email. If the preview has Deployment Protection, the scanner cannot reach it without a bypass. The scan options accept custom headers, which are attached to every request. The Vercel preview guide shows the bypass header of Vercel and how to keep its secret safe.
How do I limit the effect of the scan?
The scan documentation says that scans are production-safe by default. Still, set these limits for an automated loop:
- Lower the maximum requests per second. The default is 50.
- Use Passive only for a read-only pass.
- Add a custom header that tags scanner traffic in your logs.
- Set a cap on the number of scans per task, so that a loop cannot use the whole quota.
Which actions need a person?
Do not let the agent do these actions without approval:
- Add a new target.
- Change the scope or the credentials of a scan.
- Mark a finding as a false positive. This removes it from the risk score for the whole team.
- Merge or deploy a fix to production.
- Replace a production key.
The agent may read findings, edit code in a branch, and open a pull request. A person approves the rest. The agentic code review guide shows how to review the diff that an agent makes.
How do I fit this into CI?
An agent in an editor is one place for the loop. A CI job is another. The CI job runs after each preview deploy and fails the build when a Critical or High finding is open. The page on continuous security smoke testing covers the CI gate. Use the REST API from the job.
For a deeper test of each release, see continuous penetration testing. For how the testing agent itself works, read AI agent security testing.
What can the API loop not do?
Be clear about the limits.
- It tests a deployed URL. It does not read the source code. A flaw that has no effect on the running app can stay unseen.
- It is not final until the review ends. Every scan on a paid plan goes through an on-call review. Findings are preliminary until it finishes.
- It does not replace review. A scan shows that a flaw is present. It does not show that the code is secure. Read verify AI-generated code for the difference.
- It cannot reach a private host. A host on a private network needs a public preview.
- It has a quota. Each scan and each API call uses a quota.
How do I start?
- Read the getting started guide and sign up for the trial.
- Add a staging or preview URL as a target. Read the targets page.
- Run one scan by hand. Read the findings and the report.
- Create two test accounts and add them to the scan options.
- Read the current API reference and the MCP connector instructions in the docs.
- Give the agent one fixed target and a cap on scans.
- Ask the agent to fix one Critical finding, deploy, and retest.
- Review the diff before you merge.
If you want a first look before you add an agent, use the vibe code scanner. It tests an AI-built app from its URL. For the checks that apply to a vibe-coded app, read the vibe coding security checks.
Common questions
Which plans include the REST API and the MCP connector?
Can an agent scan localhost?
Do I wait for the on-call review?
Does a rescan use my scan quota?
Which credentials must the agent use?
Where are the endpoint paths and the JSON schemas?
Give your agent a test loop
The REST API and the MCP connector are part of the Pro feature set, with 25,000 API calls each month. Start with one deployed URL and one fix cycle.
14-day free trial · Cancel anytime