| Prompt injection | A user, a web page, an email, an uploaded file or a tool result slips in instructions, and the model follows them instead of yours. | No model resists this reliably and no screen catches every attack, so we limit what a hijacked model can do. We treat everything a model reads as untrusted and screen it. An agent that reads untrusted content is not also given private data and an open channel to the outside, and any action that reaches outside the system needs a check outside the model or a person’s approval. |
| Sensitive data exposure | Patient, card or customer data ends up in a prompt, an answer, a log file or a vendor’s training set. | We screen for personal, health and card data and mask or tokenize it before a prompt reaches a model, where possible. Detection misses some values, so we also choose model services whose terms exclude training on your data. Logs are redacted by default, and each record is kept for as long as your retention schedule says. |
| Agents with too much power | An agent with broad access deletes records, moves money or emails customers because it misread a request or was manipulated. | We give each agent its own identity and a named owner, read-only access by default, credentials that are limited to the task and expire, limits on amount and volume, and approval gates on high-impact or irreversible actions. |
| Search that ignores permissions | Enterprise search answers with a document the person asking is not allowed to open. | We carry the source system’s access lists into the search index and filter every query by who is asking. Permission changes reach the index within an agreed time, each customer or business unit has its own index, and documents that must never surface are part of every release test. |
| Confident wrong answers | The model invents a policy, a dosage or a contract clause, and someone acts on it. | Answers come from your approved sources and cite them, and the system is built to say when support is missing and hand off to a person. We measure accuracy on expert-built test sets before each release. That lowers the error rate without removing it, so answers with legal, financial or clinical effect go to a person. |
| Poisoned data, models and memory | A tampered document, dataset, tool server or downloaded model quietly changes how the system behaves, or an agent stores a planted instruction and acts on it weeks later. | We index only approved sources, each with an owner and origin on record. Models and libraries come from an approved registry with fixed versions and integrity checks, and an AI bill of materials lists what each release contains. What an agent may keep in memory is limited and expires. |
| Agents that trust each other | In a team of agents, one passes a planted instruction or its own access to the next, and a small error spreads through the whole workflow. | Agents prove who they are to each other, a message from another agent is treated as untrusted input, no agent can hand on more access than it holds, and limits apply to the whole chain as well as to each agent. |
| Unchecked output | A model’s answer is passed straight to a browser, a database or a command line, and carries an attack with it. | We treat model output as untrusted input: checked against a fixed format, encoded for where it is going and never run as code outside a sandbox. Links and images in an answer are blocked or sent through a proxy, because that is a common way a hijacked answer leaks data. |
| Leaked prompts and secrets | System prompts, API keys or internal rules are coaxed out of the model. | We keep secrets in a vault and add them at the tool layer, out of the model’s view. Access rules are enforced in code, not in prompt wording, and we write prompts on the assumption they will be read. |
| Runaway cost and abuse | A script or a looping agent drives thousands of expensive model calls overnight. | We set quotas for each user and agent, limits on steps and run time, alerts on unusual usage and a switch that stops a system at once. |
| Shadow AI | Staff paste company data into AI tools nobody reviewed or approved. | We build the inventory from sign-in, network and spending records, not from a survey, then offer an approved route that is easier than the workaround and a policy people can follow. |
| No evidence | An auditor asks who approved a decision, and nobody can show them. | We keep a record for each decision: who asked, which agent acted, the sources and the model and prompt versions used, the tools called and who approved. Records are stored so that any later change is detectable. They hold masked values, with full prompts in a separate store that has tighter access and a shorter life, so a deletion request can be met without breaking the audit trail. |