If you're the person who has to say yes or no to deploying AI agents in your organization, this is the checklist you want printed out before the next vendor call.
It's organized into five sections — Data, Access, Behavior, Audit, and Governance — and totals 25 specific items. Each item has the question to ask, the answer you want to hear, and the red flag that should make you walk.
It's based on what's actually showing up in 2026 procurement, audits, and incident reports — not what was theoretical in 2023. Use it as a worksheet, a vendor-meeting agenda, or an internal policy template.
Section 1 — Data
1. Where does our data live and who can see it?
Ask what regions our data resides in, which subprocessors have access, and whether data is ever transferred across borders. You want specific regions named, a documented subprocessor list with named entities, and contractual data-residency commitments where required. Walk if you get vague answers about "global infrastructure."
2. Is our data used to train the vendor's models?
Ask whether the enterprise agreement prohibits use of customer data for model training, evaluation, or improvement. You want yes, in writing, with no exceptions. Anthropic, OpenAI, Google, and Microsoft enterprise tiers all offer this in 2026. Walk if there's opt-out language instead of opt-in, or carve-outs for aggregated and anonymized use.
3. How long is our data retained?
Ask about default retention for prompts, outputs, and logs and whether you can shorten it. You want a specific retention period, configurable by the customer, with the option for zero-retention modes for sensitive workloads. Walk on indefinite retention or vague "as long as necessary" language.
4. Is sensitive data classified before it reaches the agent?
Ask whether you have a data classification policy that covers AI input and whether users are prevented from pasting regulated data into unauthorized agents. You want yes, with DLP controls integrated into the agent platform or browser extension. Walk if users are on the honor system.
5. How is PII and regulated data handled?
Ask whether the agent platform has specific controls for HIPAA, FERPA, CJIS, GDPR, CCPA, or whatever regulations apply. You want specific compliance certifications matching your regulations, plus documented data-handling procedures for each category. Walk on "we're working on it" or generic SOC 2 references with no regulation-specific detail.
6. What encryption is in place?
Ask about encryption at rest, in transit, and during inference, plus whether customer-managed keys are available. You want TLS 1.3 in transit, AES-256 at rest, with optional customer-managed keys for sensitive deployments. Walk if there's no customer-managed key option or unclear answers about encryption during inference itself.
Section 2 — Access
7. How do users authenticate?
Ask about SSO support, SCIM provisioning, and mandatory MFA. You want SAML/OIDC SSO, SCIM for user lifecycle, MFA enforced, and no shared accounts. Walk on per-user signups outside SSO or password-only auth.
8. What systems can the agent connect to, and how?
Ask about the connector model, OAuth scopes, read-only versus read-write, and whether you can limit which connectors a user can enable. You want granular OAuth scopes, admin-controlled connector allowlists, and separation between read-only and write-enabled tools. Walk on all-or-nothing connector access.
9. Can the agent be limited by role?
Ask about role-based access control and whether different teams can have different connector and tool sets. You want yes — RBAC for users, RBAC for skills, RBAC for connectors. Walk on a flat access model where every user gets every capability.
10. How is privileged access protected?
Ask whether admin actions, billing changes, and skill installation are logged separately and whether privileged access is reviewed regularly. You want yes, with an audit log review cadence built into your standard processes. Walk on no distinction between admin and user actions in logs.
11. What about offboarding?
Ask whether tokens, sessions, and connector authorizations are revoked automatically when an employee leaves, and how fast. You want it tied to SCIM deprovisioning, with tokens revoked within minutes and connector grants pulled. Walk on a manual offboarding process or stale tokens persisting after the user is gone.
12. Are external collaborators handled safely?
Ask whether guest or external users can access agents you've built and what's exposed. You want strict separation between internal agents and any external-facing surfaces. Walk if it's easy to accidentally share an internal agent externally without realizing what data it can read.
Section 3 — Behavior
13. What guardrails are configurable?
Ask whether you can limit what the agent will say, what topics it'll engage on, and what tools it'll use without approval. You want yes, with policy-as-code or a structured policy editor that doesn't require engineering for every change. Walk on hard-coded guardrails you can't see or modify.
14. Are high-risk actions gated by human approval?
Ask whether you can require a human in the loop before the agent sends external email, makes a payment, deletes a record, or takes any irreversible action. You want yes, configurable per action type, with no way for the agent to bypass. Walk on "it usually asks first."
15. How is prompt injection mitigated?
Ask what protections exist against malicious instructions hidden in retrieved web pages, emails, or documents. You want multi-layered defenses — input filtering, restricted tool access for content from untrusted sources, allowlisted destinations for sensitive actions, regular red-team testing. Walk if the answer is just "our model is trained to ignore injected instructions." It isn't enough.
16. Are tool outputs validated before action?
Ask what validation happens when an agent reads from one system and writes to another. You want type checking, schema validation, content filtering for sensitive patterns, and rate limiting on writes. Walk if agents can read anything and write anywhere with no intermediate validation.
17. How are model versions managed?
Ask whether you get notice when the vendor pushes a new model version and whether you can pin a known-good version. You want versioning with migration windows and the ability to pin or test new versions before rolling them out broadly. Walk on silent model updates.
18. What happens when the agent fails?
Ask about failure modes when the agent gets stuck, hits a rate limit, or hallucinates. You want failures surfaced to the user with clear messaging. No silent retries on writes. No automatic escalation of permissions. Walk on silent failures or retry loops on write actions.
Section 4 — Audit
19. Are all interactions logged?
Ask whether user prompts, agent outputs, tool calls, and tool responses are captured in an audit log. You want yes, with timestamp, user, session, input, output, and every tool invocation. Exportable. Tamper-evident. Walk on partial logs or logs the vendor controls and you can't audit independently.
20. Can we export logs to our SIEM?
Ask about native export to Splunk, Elastic, Microsoft Sentinel, or your SIEM of choice. You want standard export formats (JSON, CEF), streaming via webhook or syslog, and a documented schema. Walk on a proprietary log format with no export path.
21. Can we reconstruct a past decision?
Ask whether you can replay the exact context the agent had — the prompt, the retrieved documents, the model version, the tools available. You want full reproducibility for any past interaction, with retention long enough to support your compliance requirements. Walk on "we log the prompt and the answer" — that's not enough.
22. Is there a regular evaluation cadence?
Ask whether agent performance is measured over time — accuracy, hallucination rate, harmful output rate, regression on known cases. You want a documented eval suite, run on a regular cadence, with results tracked over time. Walk on no evals.
Section 5 — Governance
23. Do we have a written AI use policy?
Ask whether your organization has a clear, plain-language policy on what AI agents can and cannot do, what data they can touch, and what approval is required for new uses. You want yes, reviewed annually, owned by a named accountable executive. Walk on no policy or a policy nobody can find.
24. Is there an AI governance committee?
Ask who reviews new AI agent deployments, who handles incidents, and who decides what's in or out of scope. You want a standing committee with security, legal, IT, HR, and a business representative — or a clearly assigned owner if you're too small for a committee. Walk on no cross-functional review.
25. Are we aligned to NIST AI RMF or an equivalent framework?
Ask whether your AI program is mapped to NIST AI RMF, ISO/IEC 42001, or another recognized framework. You want yes, with documented evidence of the mapping. Walk on "we follow best practices" with no specific framework named.
How to use this checklist
- Vendor evaluation. Send the 25 items to any vendor before a formal RFP. The quality of their answers tells you everything.
- Internal audit. Run it against an existing AI agent deployment to find your gaps. Most teams find 5 to 10 items they hadn't fully addressed.
- Policy template. Convert each item into a policy statement and adopt it as your AI agent governance baseline.
This checklist is intentionally vendor-neutral. It will not tell you which platform to buy. It will tell you whether the platform you're considering is mature enough to deploy responsibly.
Frequently asked questions
What's the biggest AI agent security risk in 2026?
Prompt injection — malicious instructions hidden in untrusted content (web pages, emails, documents) that the agent reads and acts on. Mitigation requires multi-layered defenses, not a single setting.
Are AI agents safe for confidential business data?
With the right enterprise tier and controls, yes. Use enterprise contracts that prohibit training on customer data, enforce SSO and MFA, configure connector scopes carefully, and require human approval for high-risk actions.
What certifications should an AI agent vendor have?
At minimum: SOC 2 Type II. For regulated industries, also relevant ISO certifications (27001, 42001), HIPAA business associate agreements, FedRAMP or StateRAMP authorization, and any region-specific certifications.
How do we audit what an AI agent did?
Through the platform's audit logs. Require complete capture of prompts, outputs, tool calls, and model versions. Export to your SIEM. Set retention periods that match your compliance needs.
Should we let agents take autonomous action?
Selectively. Reversible, low-risk actions (drafts, internal summaries, search) are fine. Irreversible actions (sending external email, payments, data deletion) should require human approval until you have strong evidence the agent is reliable on that specific task.