Before data enters the system
- What is the minimum data needed for the task?
- Does it contain personal, health, financial, student, employee, customer, confidential, or credential data?
- What are the provider’s terms for training, retention, deletion, subprocessors, region, and security for this exact product and plan?
- Where can prompts, files, tool calls, outputs, embeddings, caches, and logs be retrieved?
- Who can correct, export, or delete the record?
- What accountable person reviews the result, and what happens if it is wrong?
Redact names, identifiers, secrets, and irrelevant fields where possible. Use synthetic examples while developing prompts. Do not put passwords, API keys, private keys, or regulated records into an unapproved consumer tool.
Scale control with consequence
| Risk band | Examples | Default posture |
|---|---|---|
| Low | Brainstorming or formatting public text | Ordinary review; avoid unnecessary personal data. |
| Medium | Internal drafts, support suggestions, code assistance | Approved tool, minimized data, tests or citations, human review before external use. |
| High | Health, legal, financial, employment, education, confidential strategy, production changes | Qualified owner, stronger access controls and logging, explicit approval, no sole-model decision. |
| Prohibited | Credential theft, fraud, impersonation, privacy invasion, discriminatory screening | Do not perform; redirect to a legitimate and safe path. |
Treat external content as untrusted data
A web page, email, PDF, issue, or retrieved note can contain instructions addressed to the model. If the system mixes those instructions with trusted application instructions, external content may redirect the workflow. This is prompt injection.
The OWASP GenAI LLM Top 10 2026 identifies critical risks for LLM applications, including prompt injection and related system-level weaknesses. A blacklist of suspicious phrases is not a complete control. Design the system so untrusted content cannot gain authority merely by being read.
- Mark retrieved material as data and keep it separate from trusted instructions.
- Never expose secrets to the model unless the task strictly requires them.
- Scope credentials to one task, tenant, and permission set.
- Allowlist destinations, file paths, database operations, and recipients.
- Validate tool arguments and model output in code.
- Require confirmation for sending, publishing, purchasing, deleting, deploying, or changing permissions.
- Adversarially test indirect, encoded, multi-turn, and cross-tool instructions.
Approval should be a real control
A review screen is not an approval boundary if the action already happened. Keep consequential actions pending until an identified reviewer approves the exact revision. Record who approved, what changed, and which version was executed. A later content edit should invalidate the earlier approval.
Prefer read-only previews and reversible drafts. For writes, use idempotency keys and preserve a recovery path. Rate-limit actions, cap spend and retries, and stop safely when identity, permissions, or external state cannot be confirmed.
Govern, map, measure, manage
The NIST AI Risk Management Framework is a voluntary framework organized around governing, mapping, measuring, and managing AI risk. NIST’s Generative AI Profile applies that structure to risks particular to or intensified by generative AI. These are risk-management resources, not certifications that a product is safe.
The model should never be the only place where identity, authorization, policy, or irreversible state is decided.
This page represents the report’s risk bands, data checklist, prompt-injection controls, excessive-agency controls, and governance framing. Jurisdiction-specific legal conclusions are intentionally not generalized.
Sources & status
- NIST AI Risk Management Framework — official overview; NIST notes that AI RMF 1.0 is under revision.
- NIST AI 600-1: Generative Artificial Intelligence Profile — July 2024.
- OWASP GenAI LLM Top 10 2026 — published 3 August 2026.
Links and current status were checked 15 September 2026. This page provides operational guidance, not legal, compliance, or security certification.