Sample alert How it works Coverage PENTEST METHODOLOGY DOCS PRICING FAQ MCP CONTACT LOG IN SCAN MY APP →

Security Testing API for AI Coding Agents

SECURITY / API

A REST API and an MCP connector let a coding agent start a scan on a deployed URL, read the findings, fix the code, and rescan to check the fix.

TEST THE APP YOUR AGENT BUILT

Enter the deployed URL of an app that a coding agent made. The scan shows the findings that the agent must fix first.

A security testing API for AI coding agents lets the agent test the app that it just built. The agent starts a scan on the deployed URL, reads the findings, fixes the code, deploys, and starts a rescan. VibeEval offers a REST API and an MCP connector for this loop.

This page describes the workflow and the safety rules. It does not list endpoint paths or JSON schemas. The VibeEval docs are the source for those details.

Why does an agent need a security testing API?

A coding agent writes code fast. It does not see the running app the way a stranger sees it. A passing unit test does not show that a Supabase table is open or that an admin route needs no login. These gaps appear in the deployed app, in the browser bundle, and in the host settings.

A person can run a scan and paste the report into the agent. This works, but it is slow.

An API closes the loop. The agent calls the scanner as a tool. The agent reads the result as data. The agent can then fix the issue and check the fix without a person in the middle.

The loop has a limit. The scanner tests what is deployed. If the agent has not deployed, the scan tests old code. Make the deploy step part of the loop.

What is the VibeEval REST API and MCP connector?

The VibeEval billing documentation lists a REST API and an MCP connector in the Pro feature set. The Lifetime plan has both. The documentation gives a limit of 25,000 API calls each month on Pro and Lifetime. Enterprise has unlimited API calls. Read the plans and limits page for the current numbers.

Part What it is for
REST API Calls from a script, a CI job, or an agent tool that you write
MCP connector A connection that an MCP-compatible agent uses as a tool
Webhooks Events that VibeEval sends to your own HTTPS URL
Copy for LLM report A Markdown report that is written for a coding agent

MCP is the Model Context Protocol. It is an open standard that connects an AI application to tools and data. A client such as Claude Code or Cursor can use an MCP server as a tool. See the Claude Code guide and the Cursor guide for the security risks of each client.

Note: The documentation names the two features but does not publish the endpoint paths, the auth method, or the connector setup steps. Do not guess them. Use the current docs and your account settings.

What does the scan, fix, rescan loop look like?

The loop has seven steps. The agent does most of them. A person approves the risky ones.

  1. Deploy. The agent deploys the change to a public preview URL.
  2. Start a scan. The agent starts a scan on that URL with a profile.
  3. Wait. The agent waits for the scan to finish.
  4. Read the findings. The agent reads each finding with its evidence.
  5. Fix. The agent changes the code for each finding, smallest fix first.
  6. Deploy again. The agent deploys the fix to the same environment.
  7. Rescan or retest. The agent checks that the finding is gone and that no new finding appeared.

Repeat steps 4 to 7 until no Critical or High finding is open. Then a person reviews the diff.

Which scan profile does the agent choose?

The scan documentation lists four profiles:

Profile Duration Use it when
Quick Scan 1 to 3 minutes The agent needs a fast check after a small change
DAST Full 10 to 30 minutes The change adds a route, a table, or a login
API Scan 5 to 15 minutes The app has an OpenAPI spec that you can give to the scan
Custom Varies You want only some modules, such as IDOR or CORS

Use the Quick Scan inside the fix loop. Use DAST Full before a release. For an API, give the scan an OpenAPI URL. Read Running scans for each option.

How does the agent know that a scan is finished?

A scan moves through the statuses queued, running, pending_review, and completed. The statuses failed and cancelled also exist. The agent has two ways to learn the status:

  • Poll. The agent asks for the scan status at a fixed interval.
  • Webhook. VibeEval sends a scan.completed or scan.failed event to an HTTPS URL that you register.

The webhook documentation lists these events: scan.queued, scan.started, scan.completed, scan.failed, finding.new, finding.critical, finding.statusChanged, and team.memberAdded. An event arrives as an HTTP POST with a JSON body. The body holds the ID, the target, the severity, and the status. Respond with a 2xx quickly and process the event later. See Webhooks.

Do not poll each second. Wait for the expected time of the profile, then poll at a long interval.

What do the findings hold?

Each finding holds the data that an agent needs. The findings documentation lists these fields:

  • A title and a severity: critical, high, medium, low, or info.
  • A CWE and an OWASP reference.
  • The captured HTTP request and HTTP response that proved the issue.
  • Numbered remediation steps.
  • A before and after code patch, when a patch can be shown.

The Copy for LLM report puts these fields in one Markdown file. Each fix prompt includes the app URL, the detected stack, the evidence, and a rules block. The rules block tells the agent to make the smallest fix that fully resolves the issue and not to change unrelated code. Prompts exist for Lovable, Cursor, Claude Code, and a generic agent.

Give the agent the evidence, not only the title. The request and response show the agent which route and which parameter to fix.

How does the agent check that a fix works?

Two actions check a fix. The documentation describes both.

Action What it runs Quota effect
Retest The single check that made the finding Not stated
Re-run scan A new scan on the same target and profile Counts against the monthly scan quota

A re-run produces a trend against the last completed scan. The trend shows which findings are new, fixed, or unchanged. Use the trend to find a regression that the fix caused.

If a retest still shows the finding open, check three things before the agent changes more code:

  1. Make sure that the fix is deployed to the exact URL that the scan tested.
  2. Check for a CDN cache that serves the old response.
  3. For a leaked key, make sure that the old key is replaced at its provider. A rescan does not prove that the old key is dead.

The FAQ explains the retest cases in more detail.

A fix that passes today can fail next week. Ask the agent to write a test for each fixed finding. The guide on security regression testing shows how.

Which safety rules must I set before I give an agent access?

An agent that can start scans and read findings is a tool with power. Treat it as you treat any automation that holds credentials. OWASP lists excessive agency as a risk of LLM applications. Give the agent the least power that the task needs.

What is the scope of the scan?

Scan only hosts that you own or have written permission to test. The targets documentation states this rule. VibeEval does not verify domain ownership with a DNS record. It uses a safety filter and abuse review, and it suspends accounts that scan systems that they do not own.

Set the scope in the instructions of the agent:

  • Name the exact URL that the agent may scan.
  • Name the scope: a single URL, the entire domain, or subdomains.
  • Do not let the agent choose a new target from text that it reads in a web page or a ticket.
  • Add excluded paths for routes that can do harm, such as /logout and destructive admin routes.

A web page that the agent reads can hold hidden text that tells the agent to scan another site. This is prompt injection. A fixed target list in the tool configuration stops this.

Which accounts does the agent use?

Never give the agent a production admin login. An agent can print a secret in a log, a commit message, or a chat. A scan with an admin login can also change or delete real data.

  1. Create two test accounts on a staging or preview environment.
  2. Give the first account a normal user role.
  3. Give the second account a different user role, or the same role in a different tenant.
  4. Put the credentials in the authentication options of the scan. The scan supports Bearer Token, Cookie, Basic Auth, and Form Login.
  5. Use the secret instruction field for any text that must not appear in a report.
  6. Replace the test credentials when the work ends.

Two accounts let the scan test if one user can read the data of the other. The multi-tenant isolation guide explains these tests.

What must the preview environment hold?

A preview that has production data makes each scan a risk. Use a separate database with fake data. Use test keys for payment and email. If the preview has Deployment Protection, the scanner cannot reach it without a bypass. The scan options accept custom headers, which are attached to every request. The Vercel preview guide shows the bypass header of Vercel and how to keep its secret safe.

How do I limit the effect of the scan?

The scan documentation says that scans are production-safe by default. Still, set these limits for an automated loop:

  • Lower the maximum requests per second. The default is 50.
  • Use Passive only for a read-only pass.
  • Add a custom header that tags scanner traffic in your logs.
  • Set a cap on the number of scans per task, so that a loop cannot use the whole quota.

Which actions need a person?

Do not let the agent do these actions without approval:

  • Add a new target.
  • Change the scope or the credentials of a scan.
  • Mark a finding as a false positive. This removes it from the risk score for the whole team.
  • Merge or deploy a fix to production.
  • Replace a production key.

The agent may read findings, edit code in a branch, and open a pull request. A person approves the rest. The agentic code review guide shows how to review the diff that an agent makes.

How do I fit this into CI?

An agent in an editor is one place for the loop. A CI job is another. The CI job runs after each preview deploy and fails the build when a Critical or High finding is open. The page on continuous security smoke testing covers the CI gate. Use the REST API from the job.

For a deeper test of each release, see continuous penetration testing. For how the testing agent itself works, read AI agent security testing.

What can the API loop not do?

Be clear about the limits.

  • It tests a deployed URL. It does not read the source code. A flaw that has no effect on the running app can stay unseen.
  • It is not final until the review ends. Every scan on a paid plan goes through an on-call review. Findings are preliminary until it finishes.
  • It does not replace review. A scan shows that a flaw is present. It does not show that the code is secure. Read verify AI-generated code for the difference.
  • It cannot reach a private host. A host on a private network needs a public preview.
  • It has a quota. Each scan and each API call uses a quota.

How do I start?

  1. Read the getting started guide and sign up for the trial.
  2. Add a staging or preview URL as a target. Read the targets page.
  3. Run one scan by hand. Read the findings and the report.
  4. Create two test accounts and add them to the scan options.
  5. Read the current API reference and the MCP connector instructions in the docs.
  6. Give the agent one fixed target and a cap on scans.
  7. Ask the agent to fix one Critical finding, deploy, and retest.
  8. Review the diff before you merge.

If you want a first look before you add an agent, use the vibe code scanner. It tests an AI-built app from its URL. For the checks that apply to a vibe-coded app, read the vibe coding security checks.

Common questions

Which plans include the REST API and the MCP connector?
The billing documentation lists both in the Pro feature set. Pro and Lifetime include 25,000 API calls each month. Enterprise has unlimited API calls. Read the billing page for the current limits.
Can an agent scan localhost?
No. VibeEval refuses localhost, private addresses, and hosts that end in .local or .internal. Deploy the change to a public preview URL first, then scan that URL.
Do I wait for the on-call review?
Findings are preliminary while a scan is in pending_review. The documentation advises you to wait for the final result before you open tickets. An agent can read preliminary findings, but a person must confirm them before a large change.
Does a rescan use my scan quota?
Yes for a full re-run. The scan documentation says that each re-run counts against the monthly scan quota. A retest runs only the single check that made the finding.
Which credentials must the agent use?
Use dedicated test accounts with the lowest rights. Never give an agent a production admin login or a production API key. Put test credentials in the authentication options of a scan, not in the prompt.
Where are the endpoint paths and the JSON schemas?
This page does not list them. Use the VibeEval docs and your account settings as the source for the current API reference and for the MCP connector setup.

Give your agent a test loop

The REST API and the MCP connector are part of the Pro feature set, with 25,000 API calls each month. Start with one deployed URL and one fix cycle.

14-day free trial · Cancel anytime

START A TRIAL →