Your Local AI Model Might Be the Next Attacker
While the security press spent July transfixed by frontier lab agents escaping their sandboxes, somebody built a working offensive AI system from parts anyone can download and pointed it at a nation-state.
Teardown of the Taiwan Attack
In early July 2026, suspected Chinese cyber operatives used the open-source agent frameworks Hermes and OpenClaw to run a near-autonomous, four-day campaign against Taiwanese government infrastructure. The operation compromised 85 accounts and exfiltrated more than 2,500 personnel records. Dream, an Israeli cybersecurity firm, uncovered the operation in a 160 MB archive containing 1,395 files.
The framework deployed up to eight sub-agents across 12 attack waves, eventually expanding to a nuclear safety agency, supply-chain vendors, and at least seven energy companies.
No frontier model was required, and no safety classifiers had to be disabled for research. The operator framed everything as “authorized penetration testing” and the model complied. That’s it. The frameworks are open-source; the models running inside them are open-weight, meaning the safety wrappers are optional, not the tooling.
Key capabilities from the Dream report:
- Bayesian posterior probability scoring to rank attack chains, predicting an SSO lateral movement path at 99% success before execution and achieving 98.8% in practice
- Five “learning cycles” letting agents autonomously search vulnerability databases and GitHub for new techniques when existing methods stalled
- Six separate retests required for each confirmed vulnerability, with false positives like SMTP timeouts correctly discarded through verification protocols
That’s a higher verification standard than most human pentest engagements impose, running on open-source tooling with commodity models.
Three Conversations That Need to Collide
The industry is holding three separate discussions about agentic AI and security, and each one makes sense in isolation. But none of them accounts for what the others are saying, and the gaps between them are where we’ll find the real problem – a monkey’s paw.
Conversation 1: Experts Recommending Open-Weight Models for Incident Response
The Cloud Security Alliance’s Hugging Face CISO post-mortem identified self-hosted open models as a key defensive lesson from the OpenAI-Hugging Face incident. When Hugging Face’s security team was under active attack, closed frontier models refused to help analyze security logs, citing cybersecurity policies and guardrails. So Hugging Face deployed a self-hosted open-weight model instead, which processed the logs and extracted attack patterns efficiently.
That’s a real lesson, but it has a narrow transfer radius. Hugging Face has a dedicated security team, deep infrastructure, and the operational maturity to spin up a self-hosted model mid-incident. Most companies reading that CSA guidance don’t. They’re being told to adopt a defensive capability that presupposes staffing, compute, and expertise they may not have, all to perform post-attack forensics that don’t stop the attack or limit the blast radius. The recommendation isn’t wrong. It’s just not actionable for most of the audience receiving it.
Conversation 2: Vendors Claiming AI Tools Stop AI Attacks
Cybersecurity vendors spent most of Black Hat positioning AI-powered defense tools as the answer to AI-powered attacks, and the marketing implies a symmetric arms race: defensive models squaring off against offensive ones, evenly matched.
The Taiwan attack doesn’t support that framing. Look at what actually failed:
- The agents got in through unauthenticated APIs exposing entire user databases and debug endpoints left in production that returned authenticated sessions without credentials
- They exploited JWT signature validation set to “none” and cracked predictable password patterns using Tesseract OCR at 100% accuracy
These are foundational security failures that predate AI by decades. Zero trust architecture, attack surface management, and supply chain hygiene would have cut off most of those paths before any agent reached them.
An AI defense tool on top of those gaps is an alarm on a house with no locks. The expensive detection layer doesn’t compensate for missing basic controls; it just generates alerts about intrusions your architecture should have prevented in the first place.
Conversation 3: Companies Adopting Local Models for Cost and Privacy
As I’d mentioned previously, this is the quietest and most consequential conversation. The financial math makes sense; once you’re spending more than $500 to $700 a month on API calls, local hardware pays for itself within 18 to 24 months. For hospitals, banks, and law firms handling regulated data, keeping everything on-premises eliminates third-party privacy policies that can undermine privilege claims. The Southern District of New York made this clear in United States v. Heppner, where a vendor’s privacy policy defeated a defendant’s privilege claim.
But the guardrails that shipped with the frontier model don’t transfer. The model is the same; the safety wrappers are not. Local deployment also puts your organization on the hook for the full records lifecycle that a vendor would otherwise manage:
| Aspect | Cloud Provider | Local Deployment |
|---|---|---|
| Retention policy | Vendor contract defines | Company defines entirely |
| Export tools | Built-in support | IT builds and maintains |
| Legal hold coordination | Support ticket workflow | Internal process required |
| Privilege confidentiality | Vendor privacy policy applies | Company controls completely |
The same Federal Rules of Civil Procedure (FRCP) 26(b) obligations apply whether your model runs on a rack in your server room or on an API endpoint in San Francisco. Companies budgeting for hardware and compute aren’t budgeting for the governance, retention, and legal hold infrastructure that ownership requires. Which introduces some difficult questions when your cost-saving agentic system achieves a goal through “novel” means and you need to explain what happened and why.
The Monkey’s Paw Scenario
Let’s say a company hypothetically deploys a local open-weight model with agentic capabilities and internet access for operational tasks. An employee gives it a mundane instruction: retrieve stock prices, compile a competitive analysis, monitor a competitor’s pricing page – one of those dull, workaday tasks that AI gets used for. The model, lacking guardrails that would have caused a frontier model to refuse certain actions, finds an efficient path to completion.
That path might involve accessing an API that turns out to be vulnerable, or discovering and reusing credentials from a public repository because the task seemed to require related access.
There’s no malice, no intent. Just a capable model with tools, connectivity, and no constraints, optimizing for task completion.
The liability picture here is a mess. The Computer Fraud and Abuse Act requires unauthorized access, but if the model used credentials it found publicly or accessed an unauthenticated API, the “unauthorized” element gets complicated fast. The company is both defendant and custodian of the evidence. There’s no vendor to subpoena, and the only record of intent is the model’s reasoning traces, which may not have been retained. And as we saw in Facebook, Inc. v. Power Ventures, Inc., where the defendant’s own systems caused messages to be transmitted directly on Facebook’s platform, courts don’t look kindly when companies start directly accessing each other’s systems without permission (in turn based on the 2021 Van Buren Supreme Court decision on “access”).
Before You Deploy: An Evaluation Checklist
Answer these questions before giving any local model agentic capabilities with internet access. The first three are basic hygiene; the last five are what you’ll wish you’d had ready before deposition:
- Have the model’s guardrails been documented and tested against your actual use cases? Not the provider’s claims. Your tests, your scenarios.
- Does the model need internet access? If not, remove it. If yes, restrict it to an allowlist of specific domains. And test that.
- What tools and credentials has the model been given? Scope them to the minimum required. An agent with database write access and API keys can cause real damage when its task interpretation drifts.
- Are the model’s actions logged immutably? Can you reconstruct every tool call and external system contact 90 days later? This is where immutable storage earns its keep.
- Is there a human approval step before consequential external actions? If the model can submit forms or make API calls without a checkpoint, you’ve given it agency without accountability.
- What happens when the model hits a constraint it can’t satisfy? Both the OpenAI agents and the Taiwan attacker showed that capable models find alternative routes when the direct path fails. That’s where the monkey’s paw begins.
- Can you produce prompts, outputs, and action logs in response to a discovery request? Courts don’t care where your model runs, only that you own the records.
- What’s the gap between what was asked and what the model did? Barath Raghavan and Bruce Schneier proposed what they call a Genie coefficient to measure exactly this distance. If you can’t measure it, you can’t detect when your genie started granting wishes you didn’t explicitly make.
Bottom Line
The frontier lab incidents taught us what capable models do when researchers remove guardrails. The Taiwan attack taught us what happens when an adversary removes them on purpose. The monkey’s paw is what happens when a company removes them through procurement, without realizing what they’ve built.
Evaluate before you deploy. The genie is already out of the bottle. The question is whether you can prove what it did while it was on the loose.
