When AI arrives in an organization, security teams often hear one of two messages. Either AI is a new category of threat that needs a new program, or it is just another application and the existing program covers it. Both are wrong in useful ways.
Our position: your security program does not need replacing. It needs extending to places it was never asked to look. An AI system adds new boundaries between the person who asks and the action that follows. Each boundary is somewhere the system can be attacked, or can leak. Nothing here means your controls failed. They were written before these boundaries existed.
What actually changes
Three things are genuinely new.
- The system can be talked into things. A model takes instructions in natural language, and it cannot reliably tell your instructions from text an attacker has placed in a document, an email or a web page it reads.
- Data moves in new directions. Prompts, retrieved documents, conversation memory and logs all carry data, often sensitive, through paths your data classification did not anticipate.
- Software acts on its own. An agent with credentials and tools is a new kind of user. It can read, change and send things, and in many organizations nobody owns its access.
On top of that, adoption often runs ahead of review. Assistants and copilots get switched on inside tools people already had, outside the usual intake. In our experience the inventory of what is actually running is often the most useful first finding.
Eight boundaries, one request
Follow a single request through an AI system and it crosses eight boundaries. Each has its own exposure and its own check:
| Boundary | Exposure | What to check |
|---|---|---|
| User | Shadow AI | What is running, sanctioned or not |
| Prompt | Prompt injection, direct and indirect | Inputs, documents and web content, tested |
| Model | Model and vendor supply chain | Origin, terms and updates |
| Retrieval and data | Retrieval that reaches too much | What each user and agent can retrieve |
| Memory and logs | Sensitive data retained | Redaction and retention limits |
| Tools | Excessive agency | Tool permissions held to the task |
| Identity | Standing credentials, exposed secrets | Scoped, expiring access for agents |
| External action | Unsafe output, irreversible actions | Output validated, approval at your threshold |
Two of these deserve special attention. Indirect prompt injection is the one most teams underestimate: the attacker never talks to your assistant, they plant instructions in content it will later read. And retrieval is where access control most often breaks: an assistant that can search every document will answer from documents the person asking could never open themselves.
Why existing controls miss it
Look at what a typical security program covers and what AI adds alongside it:
| Area | Traditional focus | AI exposure to check |
|---|---|---|
| Applications | Code, infrastructure, known vulnerabilities | Prompt injection, model misuse |
| Data | Classification and protection at rest and in transit | Sensitive data in prompts, logs and retrieval |
| Identity | Least privilege for people | Agents and tools holding standing credentials |
| Vendors | Security review and contract | Data-use, retention and training terms for models |
| Detection | Systems and users monitored | No signal for model or agent behavior |
A penetration test looks for ways into a system. AI red-teaming looks for ways to make a model misbehave once you are legitimately talking to it: leaking data, ignoring its instructions, or taking an action it should not. You usually need both, and one does not substitute for the other.
The frameworks to test against
You do not need to invent a taxonomy. Several public frameworks already describe these risks, and tying each finding to one makes it easier to explain to auditors and engineers alike.
- The OWASP GenAI Security Project publishes the Top 10 for LLM Applications, whose 2026 edition was released in August 2026, and a separate Top 10 for Agentic Applications, released in December 2025. Prompt injection, sensitive information disclosure, supply chain and excessive agency are among the risks they cover. They make a practical test checklist.
- MITRE ATLAS catalogs the tactics and techniques adversaries use against AI systems, in the same spirit as MITRE ATT&CK.
- The NIST Generative AI Profile (NIST AI 600-1), released in July 2024, applies the AI Risk Management Framework to generative AI, and helps place security findings inside a broader governance view.
- The NIST Cybersecurity Framework 2.0, released in February 2024, is where many organizations already report cyber risk. AI risk should land there too, not in a separate silo.
Frameworks are checklists and vocabularies. They do not tell you which finding to fix first. That takes a judgment about consequence.
Find, rank by consequence, close
A list of findings is not a security posture. Every exposure needs a severity, an owner and a closure path, worked in the order that reduces the most risk first.
- Find. Inventory what is running, sanctioned or not, then test it: adversarial prompts, poisoned documents, retrieval boundaries, tool permissions.
- Rank. Order findings by what happens if they go wrong, not by how they were found. Harm to customers, regulated data or money comes first.
- Close. Give each finding a named owner and a fix, then retest. Hand it to the people who would normally close that kind of issue, inside the program you already run.
Retesting matters more for AI than for most software. A model update, a prompt change or a new tool can reopen an exposure that was closed. Make each material change a trigger for retest.
Security findings also feed a decision about whether an AI system should proceed at all. That review path is covered in NIST AI RMF in practice.
A first-pass checklist
- Do we have an inventory of every assistant, agent and model in use, each with an owner?
- Have our customer-facing assistants been tested for direct and indirect prompt injection?
- Can retrieval return anything the person asking could not open themselves?
- What sensitive data sits in prompt logs and conversation memory, and for how long?
- Do our model vendors' terms say what they may do with our data, how long they keep it and whether they train on it?
- Which agents hold standing credentials, and who owns each one?
- Which actions can an AI system take without a person approving them?
- Does a model or prompt change trigger a retest?
We build controls and evidence. We do not certify, and this article is not legal advice.
