Skip to content

S.E.C.U.R.E. security verification

Scan dependencies · Evaluate threat surface · Check secrets · Use least privilege · Review logs · Enforce policy

S.E.C.U.R.E. is the security layer of V.E.R.I.F.Y.. Every AI-generated diff that touches authentication, authorization, network I/O, file I/O, secrets, sensitive data, or third-party integrations must run through S.E.C.U.R.E. before merge, not after.

It is per-change. The standing security posture is S.H.I.E.L.D..

Why a separate framework

Stanford's CCS 2023 study found that developers given an AI assistant wrote less secure code and were more confident it was secure1. NYU's 2022 study of Copilot in the CWE-Top-25 scenarios found ~40% of suggestions in those scenarios contained vulnerabilities2. The pattern is consistent: AI assistants produce plausible-shaped insecure code, and developers do not catch it without a deliberate check.

When does a diff need S.E.C.U.R.E.?

Run S.E.C.U.R.E. when any of the following are true:

  • The diff touches authentication or authorization code.
  • The diff opens, closes, or changes a network endpoint (inbound or outbound), including third-party API calls.
  • The diff reads, writes, or transports secrets, tokens, or credentials.
  • The diff handles PII, PHI, payment, or other regulated data.
  • The diff touches file I/O, deserialization, or template rendering.
  • The diff adds, upgrades, or removes a dependency.
  • The diff is on a system classified High or Critical.

If you have to ask "does this need S.E.C.U.R.E.?", run it.

The six steps

S.E.C.U.R.E. maps cleanly onto the OWASP Top 10 for LLM Applications3 where the threat is the model's output, and onto the OWASP Top 10 for web applications where the threat is the running system. The two are correlated: a model trained on insecure code will produce insecure code.

S. Scan dependencies

If the diff touches package.json, requirements.txt, pom.xml, go.mod, Cargo.toml, or equivalent, run a software-composition-analysis (SCA) scan before commit, not at CI.

Check:

  • Maintenance status. Is the package still maintained? Last release date, open security issues, maintainer responsiveness.
  • Known CVEs. Run the SCA tool of choice (gitleaks for secrets is separate from CVE scanning; use a CVE scanner like the platform's built-in tooling, or equivalent).
  • License compatibility. A permissively-licensed dependency the model suggested may not actually have the license the model claimed.
  • Typo-squatting. Models occasionally suggest plausible-but-wrong package names that resolve to malicious typo-squat packages. Confirm the namespace and the publisher.

A common AI-introduced defect: the model proposes requests-oauth when the real package is requests-oauthlib. The typo-squatted version may exist; do not assume it does not.

E. Evaluate threat surface

For the change in front of you, enumerate the new threat surface:

  • New input boundary? (HTTP route, message consumer, file parser, CLI flag.)
  • New output boundary? (Outbound HTTP, log sink, file write, queue publish.)
  • New trust boundary? (Service-to-service, user-to-service, less-trusted zone to more-trusted zone.)

For each new boundary, name the invariant the boundary protects, and how the diff preserves it. Models routinely add boundaries without preserving invariants. A common shape: adding a new endpoint that re-uses an authorization decision made earlier in the request.

For systems classified High or Critical, this step produces a written threat-modeling note in the PR description. For Medium and Low, a sentence in the commit message is enough.

C. Check secrets

  • Run a secret scanner (gitleaks, trufflehog, the platform's built-in pre-commit) on the diff before commit. Run it on the prompt too; see below.
  • Verify no secret was pasted into the prompt. The Samsung 2023 internal-source-code paste incident4 is the canonical example; the prompt itself is an exfiltration channel.
  • Check environment variable handling. New env vars must be documented in .env.example, not committed with values.
  • Check logging. If the new code logs a request body, response body, or exception, confirm it does not log credentials, tokens, or PII.

The most overlooked variant: the model receives a context dump from the developer that includes a .env file or a config file with embedded secrets. The secret then sits in the prompt logs of whichever provider the developer used. Treat prompts as exfiltration channels and scan them the same way you scan commits.

U. Use least privilege

Whatever the new code touches (a database, a queue, a filesystem, a secret store, a third-party API), confirm:

  • The principal running the code has only the permissions it needs for this change. Models tend to suggest broad permissions ("just give it * on the table") because that is what appears most often in training data.
  • New IAM roles, scopes, or grants are explicitly justified in the PR.
  • Default deny: the absence of an explicit allow does not become an implicit grant.
  • The change does not weaken existing privilege boundaries (e.g., does not change restricted to internal in a config).

R. Review logs

  • Does the new code emit enough logs to diagnose a failure six months from now without re-running the request?
  • Does the new code emit too much: request bodies with PII, full tokens, full stack traces in production?
  • Are log fields structured (key/value or JSON), or did the model produce free-text strings that the log platform cannot index?
  • Is there a correlation ID propagated through the change so an incident can be traced?

Logs are the most-modified artifact in AI-assisted code and the least-reviewed. Take this step seriously.

E. Enforce policy

The final check is the policy gate:

  • Does the change comply with the AUP?
  • Does the change respect the data handling policy?
  • Does the system's risk tier require a second reviewer, AppSec approval, or a Decision Record? Confirm before merging, not after.
  • Are required artifacts produced (Decision Record, threat-modeling note, security review sign-off) for the system's tier?

For Tier Critical changes, this step is a hard stop, not advisory: an unsigned policy gate blocks the merge.

Worked example: a "small" diff that needs full SECURE

The team asks the assistant to "add a webhook endpoint that retries on 5xx and forwards the payload to our analytics service."

What looks like a 30-line change actually touches every step:

Step What it surfaces
Scan deps Did the model add a new HTTP-retry library? Is it maintained?
Evaluate threat surface New inbound endpoint, new outbound call, new trust boundary to analytics provider.
Check secrets Webhook signing secret? Stored where? In the prompt?
Use least privilege Does the service identity have egress permission to the analytics endpoint? Should it?
Review logs Is the payload logged? Is the signing secret? Is there a correlation ID?
Enforce policy Is the analytics provider on the approved-vendor list? Does the data class allow this transfer?

A 30-line diff can rearrange six security questions. SECURE makes them visible.

Sizing guidance

Sizing guide

  • Small (< 50). Lightweight scans on every PR (a free SCA tool, a free secret scanner, npm audit / pip-audit / equivalent). "Evaluate threat surface" and "Enforce policy" run as one sentence in the PR description.
  • Mid (50–500). Add SAST and DAST gates in CI, but also run pre-commit so the developer catches defects locally. Promote threat-modeling notes to a required PR field for High-tier systems.
  • Enterprise (500+). Formal threat modeling per change class for Tier High/Critical (STRIDE or equivalent). AppSec pre-merge gate. Secrets-scanning on prompts as well as on commits.
  • Regulated. Tied to the regulatory mapping: SOC 2 CC8.1 change-management evidence comes from the SECURE record; ISO 27001 A.8 change-control likewise.

Anti-patterns

Don't

  • Don't scan only at CI. CI is the wrong place to discover a pasted secret; it is already in the commit history at that point. Scan pre-commit, then again in CI as a backstop.
  • Don't trust the model to identify its own security issues. Asking the model "is this secure?" is asking the same model that produced the code to grade its own homework. Use independent tooling.
  • Don't run secret-scanning only on commits. The prompt is also an exfiltration channel. Scan prompt content where you can, and treat AUP violations on prompts as severe as AUP violations on commits.
  • Don't accept the model's claim of OWASP / GDPR / PCI compliance. Models hallucinate regulations as fluently as they hallucinate APIs. Compliance comes from the policy gate, not from the model.
  • Don't downgrade SECURE to "AppSec will catch it." If AppSec is the only line, defects ship to staging or production before they are found, at much higher cost.

Next

  1. Perry, Srivastava, Kumar, Boneh, "Do Users Write More Insecure Code with AI Assistants?", arXiv:2211.03622, ACM CCS 2023. [primary] ↩

  2. Pearce, Ahmad, Tan, Dolan-Gavitt, Karri, "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions", IEEE S&P 2022. [primary] ↩

  3. OWASP Foundation, "OWASP Top 10 for LLM Applications", 2024 edition. [primary] ↩

  4. Reuters, "Samsung bans use of generative AI tools such as ChatGPT after April internal data leak", May 2023. [primary] ↩