SCAN YOUR KIRO APP FOR VULNERABILITIES

Kiro is AWS’s agentic IDE — a VS Code-based editor built around spec-driven development. The agent turns a prompt into requirements.md, design.md, and tasks.md before writing code, then implements against those specs; autopilot mode applies the resulting multi-file changes autonomously, and agent hooks and .kiro/steering/ files shape what it does on every session. It ships in whatever stack your project uses.

Specs are a genuine review asset — intent you can read before code exists — but they create a specific illusion: teams review the design and skim the implementation. The gap between “the design says every endpoint verifies ownership” and what the handler actually does is exactly where vulnerabilities live, and an eleven-file autopilot diff for a one-line prompt invites reading the agent’s summary instead of the change. Meanwhile .kiro/ itself is a committed behavior-control surface: a steering-file edit or a poisoned hook changes what the agent does for everyone who pulls.

VibeEval tests the side the spec can’t vouch for: it scans the deployed app black-box and reports what an attacker can actually reach, whatever the design document promised.

Common vulnerabilities we find in Kiro apps

These are the shapes that recur in Kiro-built applications — most of them living in the gap between a reviewed spec and a skimmed implementation.

Spec-to-code gaps on auth and validation

The design spec says “verify the requesting user owns the resource”; the generated handler checks the session and skips the ownership line. Because the requirement is documented, review tends to assume it’s implemented — the gap only shows up when someone tests the running app.

// design.md: "verify the requesting user owns the project"
// generated handler: the ownership line never made it
const project = await db.projects.findById(req.params.id);
return res.json(project);

Exploit: any authenticated user requests another user’s object by ID and gets it. Fix direction: test every generated route three ways — unauthenticated, wrong user, correct user — instead of trusting the spec.

Ownership checks missing from generated CRUD (BOLA)

The same shape independent of specs: Kiro-generated routes confirm a user is logged in, not that this user owns the object. Exploit: sign in as user B, replay user A’s request with A’s record ID; the API returns A’s data. The agent’s happy-path tests never cross accounts, so CI stays green. Fix direction: owner_id === session.user.id on every ID-keyed route, encoded once in a steering file so the agent scaffolds it by default. Pattern: BOLA in AI-generated CRUD.

Autopilot regressions in files nobody read

Autopilot touches eleven files to implement one feature; the reviewer reads three. The recurring finding is file eight: a CORS config widened to silence a dev error, a validation branch deleted, an auth middleware “simplified.” Exploit: a browser on any origin now makes credentialed requests to your API. Fix direction: autopilot on feature branches only, full git diff before merge, and review time budgeted by diff size, not prompt size. Related: CORS credentials misconfig.

Poisoned steering files and hooks redirecting the agent

.kiro/steering/ injects persistent instructions into every session, and agent hooks fire automatically on events like file save with your local permissions — neither appears in your chat. A one-line steering edit in a “docs-only” PR silently changes the agent’s output for the whole team; a hook in a cloned repo executes the first time you save. Fix direction: CODEOWNERS on .kiro/, hooks read before enabling, untrusted repos opened with hooks and autopilot off. This is the repo-content variant of indirect prompt injection.

Secrets surfacing in committed specs

Specs are generated from workspace context — and if .env or a service-account JSON sat in the workspace, design.md can embed the connection string. Specs get committed and shared as documentation, so the leak travels further than the code would. Fix direction: secrets out of the workspace before generating specs, grep every spec for keys and hostnames before commit, and rotate anything that ever appeared in one. The deployed version — keys in the shipped bundle — is what the scan catches; see the Token Leak Checker.

Dependencies suggested by specs, vetted by no one

When tasks.md calls for npm install <package>, the name carries spec-level authority and tends to get installed without a registry check. Hallucinated and typosquatted names are the AI-specific failure — a squatted near-miss is code execution at install time. Fix direction: confirm each suggested package exists, is canonical, and is maintained before installing; the Package Hallucination Scanner automates the AI-specific subset.

How VibeEval works with Kiro

1. Enter the URL of the deployed app — production or the preview built from an autopilot branch. Black-box: no workspace access, no code upload, no dependence on what the specs claim.
2. The agent probes it in a real browser. It maps the app and its API surface, fires unauthenticated and wrong-user requests at every endpoint (auth and BOLA), replays IDs across accounts, checks the bundle for exposed keys, and tests CORS, security headers, and error-response leakage.
3. You get findings with severity and paste-ready fix prompts. Each prompt is written to drop into a Kiro spec or task — the fix flows through the same requirements-design-tasks pipeline as the feature did. Redeploy and re-scan to confirm.

Manual testing vs VibeEval

	Manual review	VibeEval
Spec-vs-implementation verification	Read the spec, then read every handler against it	Tests the running behavior the spec promised
Full autopilot diff review	Hours when the diff is 11 files for a 1-line prompt	Independent of the diff — scans the deployed result
Cross-user BOLA testing	Two accounts, replayed IDs per endpoint — rarely sustained	Every ID-keyed endpoint, every scan
Regression after each session	Full re-review per autopilot run	Re-scan in minutes after every merge
.kiro/ steering and hook audit	Human judgment, per pull	Upstream of the scan; downstream effects surface in findings
Business logic	Irreplaceable	Out of scope

Manual review keeps .kiro/ hygiene, logic, and intent. The scanner wins on repeatability: the same full battery after every autopilot run, immune to summary-reading.

Frequently asked questions

Does spec-driven development make the code more secure?

It makes intent reviewable, which is genuinely more than most agents offer — and it changes nothing about the implementation’s failure modes. The dangerous case is specifically the documented requirement that didn’t make it into code, because documentation creates confidence. Scan the deployed app either way; it’s the only artifact that can’t be out of sync with itself.

Does VibeEval integrate with Kiro?

The scan itself needs no integration — it runs against your deployed URL, which keeps it independent of the tool that wrote the code. Kiro supports MCP servers, so you can also wire VibeEval’s MCP server into your Kiro configuration for security feedback inside the IDE; either way, findings arrive as prompts ready for the next spec.

What about agent hooks and steering files?

They are the highest-leverage risk surface inside the IDE: hooks execute with your local permissions without a prompt, and steering files silently shape every session. Audit .kiro/ like executable code — our Kiro security checklist sequences the checks. VibeEval covers the other side: whatever steered the agent, the shipped app either holds up or it doesn’t.

Can steering files help security instead of hurting it?

Yes — the same mechanism works for you. A vetted steering baseline (“parameterize SQL, no hardcoded secrets, auth plus ownership check on every route”) shifts the default shape of everything the agent generates, across every spec, for every teammate. Ship it once, gate changes with CODEOWNERS, and let the scan verify it held.

Is autopilot safe to use on a production project?

Yes, with the same discipline you’d apply to a fast contractor with commit access: feature branches only, protected main, full diff read before merge, and a scan of the deploy before promotion. The mode isn’t the problem; the skimmed diff is. For a worked example of gating Kiro output in CI, see Kiro IDE compliance gate.

Can VibeEval fix issues Kiro introduces?

It reports each finding with severity, evidence, and a paste-ready fix prompt. Feed the prompt into a Kiro spec, let the agent implement it, review the diff, redeploy, and re-scan — the loop is built to run after every session.

Related Kiro resources

• How to Secure Kiro — step-by-step hardening: .kiro/ review, hook audits, steering guardrails, autopilot scoping.
• Is Kiro Safe? — the trust-boundary analysis: what runs with your permissions, what steering files can do, where the risk concentrates.
• Kiro Security Checklist — pre-session and pre-merge checks ordered by severity, with the attack patterns we see in Kiro repos.
• Kiro IDE compliance gate — gating Kiro-generated changes in CI, worked end to end.
• BOLA in AI-generated CRUD — the most common finding in agent-built APIs, in depth.

Test your Kiro app before launch

Kiro writes the spec, implements it, and summarizes the diff — the one thing it can’t do is attack its own output. Scan the deployed app after every autopilot run and before every launch: findings in minutes, each with a fix prompt ready for the next spec.