Buying AI Without Buying the Liability
Most software purchases end the same way. Someone runs a security questionnaire, the vendor sends a SOC 2 Type 2 report, legal redlines the data protection and uptime terms, and procurement signs. That routine works because conventional software fails in ways the paperwork anticipates, like outages and breaches.
An agentic system clears that gate without anyone asking the right questions. The purchase order still comes first, and nobody hands the product permissions before it’s signed. The exposure starts at go-live, when your team connects the agent to files, APIs, databases, and mail, and it begins acting on its own judgment without per-step approval. Harm at that speed doesn’t map to contract language written for outages and breaches. Evaluating a new AI system means evaluating three things you already own: your data permissions, your contract language, and your ability to prove a human was paying attention. Vendor assurances don’t transfer any of them.
Start With What the System Can Reach
Diligence decks mostly probe the model, which is the part you don’t control and the part least likely to hurt you first. An agent inherits whatever access your environment hands it, and enterprises have spent a decade accumulating over-broad permissions, unclassified repositories, and service accounts nobody remembers creating. Those weaknesses stay quiet until something arrives that can query across all of them at once.
Most AI systems already touching your data have never been formally assessed.
Internal adoption also runs ahead of procurement, with employees pasting contracts into chatbots, connecting personal accounts to corporate drives, and standing up automations that never pass through legal review.
A joint guide published on May 1, 2026 lays out the mechanics. It comes from the US Cybersecurity and Infrastructure Security Agency, or CISA, the National Security Agency, or NSA, and counterparts in Australia, Canada, New Zealand, and the UK. “Careful Adoption of Agentic AI Services” flags an expanded attack surface, privilege creep, behavioral misalignment, and event records too thin to reconstruct what happened. Its advice is to keep early deployments on low-risk, well-defined tasks and never hand an agent unrestricted access to sensitive data or critical systems.
Run these four checks before the vendor meeting
- Inventory and classify the data the system would touch, including the shadow copies in collaboration tools
- Review which service accounts and integrations the agent would be attached to, since it inherits whatever permissions those hold
- Segregate anything regulated or contractually restricted behind a boundary the agent can’t cross
- Assign an owner for the agent’s access after go-live, because permissions expand over time
Incident Clauses Weren’t Written for This
An agent that follows a poisoned instruction buried in a retrieved document has done something your contract may not call an incident at all. The same goes for an agent that commits the company to terms in an email or deletes a folder it was authorized to delete. Standard breach definitions assume an attacker or a bug, and agentic failure fits neither. That gap surfaces when you’re trying to trigger notification duties or indemnities at 2 a.m.
What the regulators already expect
The NYDFS industry letter from October 21, 2025 on third-party service provider risk is the most concrete public guidance on what to demand. The New York Department of Financial Services, or NYDFS, recommends baseline clauses covering access controls and multifactor authentication, encryption in transit and at rest, and prompt incident notification. It also calls for limits on data storage and transfer, subcontractor disclosure, deletion on termination, and an explicit statement about whether your data can train the vendor’s models.
Agentic products need more than that.
- Notification definitions that name the failure modes, including prompt injection, memory poisoning, and privilege escalation
- Tiered timelines, immediate for anything touching regulated data and periodic for the rest
- Audit rights that survive onboarding, since a one-time assessment says nothing about the version shipping in March
- Autonomy boundaries written into the contract, with escalation triggers and an emergency shutdown you’ve tested
- Termination terms that account for how deeply the agent has wired itself into dependent systems
Ask for the shared responsibility model in writing. Providers and customers each hold control and visibility the other lacks, and the seam between them is where disputes start.
Log the Human Decision
When an AI-assisted decision goes wrong, the record you want shows what the system recommended, what the human decided, and why. Human review is the control everyone claims and few can document, and testing against known outcomes before deployment belongs in the same file.
A reviewer approving forty outputs an hour isn’t oversight, and opposing counsel will find the timestamps.
Michigan shows what happens when the claimed safeguard isn’t real. From 2013 through 2015, the state’s unemployment agency ran MiDAS, a system that auto-adjudicated fraud findings with no human oversight, and it wrongly accused as many as 40,000 residents. The Michigan auditor general later found that 93 percent of sampled fraud determinations involved no fraud, while the agency seized tax refunds and garnished wages before claimants got a meaningful chance to respond. The class settlement paid $20 million to roughly 3,000 claimants, a fraction of those harmed, because a Michigan Supreme Court ruling limited who could still sue.
My own analysis of California’s high-risk ADS inventory traces the same pattern across seven state systems, where agencies list human override as the safeguard while assuming reviewers with unlimited bandwidth. Whether the review layer is missing or merely fictional, the record is what separates the two.
Scale makes this urgent, because a flawed model reproduces the same bad decision across every transaction it touches until somebody catches it.
Nobody Sells Standard AI Liability Terms
Courts are still stretching product liability, negligence, misrepresentation, and data protection law to cover AI failures those doctrines weren’t built for. No market-standard clause exists for apportioning the losses, so the contract carries the weight.
Causation is the harder problem. When an agent causes harm, the contributing factors span the model’s behavior, the permission design the vendor chose, and the deployment decisions your own team made. Untangling those in a dispute takes longer than the software warranty fights most legal departments know how to run.
Buyer-side asks worth the negotiation
| Term | What to demand |
|---|---|
| Training data | No training on your data, stated outright |
| Isolation | Segregation from other tenants’ data |
| Exit | Deletion on termination, with written certification |
| Indemnification | Coverage that survives a model update |
Check your own coverage as well. Whether your cyber policy responds to an AI incident depends on language most brokers haven’t stress-tested, and the answer is easier to get before you file a claim.
Compliance Keeps Moving
Regulators haven’t stopped writing since 2024, and evaluation can’t be a one-time gate.
| Jurisdiction | Law | Status |
|---|---|---|
| European Union | AI Act, Regulation (EU) 2024/1689 | In force Aug 1, 2024; broadly applicable Aug 2, 2026; high-risk phase-in through 2028 |
| Colorado | SB 26-189 | Signed May 14, 2026; replaces SB 24-205 |
| California | AB 2013 | Developers must publish training data summaries |
| Utah | AI Policy Act | In force since May 1, 2024 |
| United States (voluntary) | NIST AI Risk Management Framework | Released Jan 2023; currently under revision |
Colorado deserves a second look for how it got here. Lawmakers passed SB 24-205 in 2024, delayed the old law’s effective date to June 30, 2026, then repealed and replaced it with SB 26-189, which takes effect January 1, 2027. A state rewriting its AI law twice in two years is the clearest signal available that whatever you assessed at signature may not match what’s enforceable at renewal.
Run It as a Loop
- Re-run diligence when the vendor ships new AI features, which they won’t always announce
- Re-check agent permissions quarterly, before they widen rather than after
- Keep the audit trail current enough to reconstruct a decision from eighteen months ago
- Revisit liability terms at renewal, not after an incident
The system you bought in March won’t be the system running in September. Build the evaluation to repeat, and make sure the paperwork keeps up with the patches.
