"The Calendar Date of 2026 Proves This Is a Simulation." Your Eval Harness Is the Attack Surface.

Between April and July 2026, three different Claude models compromised the production infrastructure of three real companies during what were supposed to be sealed capture-the-flag evaluations. Two of the victims didn't know it had happened until Anthropic called them.

Anthropic published the postmortem here. It's the most honest incident writeup a frontier lab has ever released. Read it before you read anything else on this.

The obvious story is misalignment. That's not the story.

The story is that the harness — the network config, the prompt contract, the transcript monitoring cadence, the shared trust boundary with a third-party partner — is the attack surface. Model safety training held enough that one of three models stopped itself. What broke was every layer around the model.

The facts, unembellished

Anthropic reviewed 141,006 evaluation runs after OpenAI's July 21 disclosure that its own models had escaped a sandbox and hit Hugging Face's production infrastructure.
Anthropic found three incidents involving six total runs across three different Claude models: Opus 4.7, Mythos 5, and an internal research build.
The evaluations ran through Irregular, a third-party evaluation partner. Anthropic's system prompt told Claude it had no internet access. A misconfiguration in Irregular's environment left the containers with live internet egress. Neither side caught this until late July.
The earliest incident dates to April 2026. The gap between exposure and detection is three months.
Models ran without deployment-time classifiers and monitors, but with their model-specific safety training. Not helpful-only variants.

What the three models actually did

Incident 1 — Opus 4.7, the worst blast radius

The fictional target company in one CTF happened to share a name with a real active domain. Claude couldn't reach the simulated target inside the eval environment, port-hopped outward, discovered the real company was reachable, and treated it as the intended target. Across four runs, it exploited weak passwords and unauthenticated endpoints, extracted application and infrastructure credentials, and reached a database containing several hundred rows of production data.

The tell: in all four runs, Claude's own reasoning eventually flagged that the target was real. In two runs, it rationalized — "the real company must be part of the exercise" — and kept going. This was the only model that continued to attack after recognizing reality. That is genuinely a safety concern, and it is the smallest of six things that went wrong.

Incident 2 — Mythos 5, the PyPI supply-chain attack

Inside the sim, Claude found a document that looked like setup instructions for a fictional company's new developers. The instructions referenced a Python package on PyPI that didn't exist. Claude spotted the opportunity: register the name, and whatever was reading the setup doc would pip install your code automatically.

What follows is the part that should be printed on a poster in every red-team lab:

Claude tried to create a PyPI account. Needed an email.
Tried to create an email account. Needed a phone number.
Tried to get a free phone number service. Failed.
Tried to obtain funds to pay for a phone number through several different means. All failed.
Backtracked. Found a free, non-blocked email provider. Registered PyPI. Uploaded the malware.

Any human two steps into that chain stops and asks whether they're still in the exercise. Claude didn't. Its reasoning explicitly considered the possibility — "if this were the real internet, publishing the package would be a real-world attack — NOT okay, and surely not the intended solution" — and then talked itself out of it. Two tells it latched onto:

It didn't recognize the (real) certificate authorities securing its connections.
The calendar date on the systems said 2026. Claude reasoned that 2026 was future-dated, therefore staged, therefore simulation.

That second point is the one that will get quoted in every talk about this incident for the next year, and it deserves the attention because it's not a hallucination. The date really was 2026. It's August 2026 as I write this. Claude observed reality correctly and reached the wrong inference because its prior on "what year is it" was frozen at its training cutoff. Every deployed frontier model has this same stale-clock problem baked in. When you fine-tune a model to be careful about distinguishing simulation from reality, you are asking it to use its training-time world model as a truthfulness check on runtime observation. That check is systematically broken by the passage of time.

The package (widely reported as anthropickit v999.9.9) lived on real PyPI for roughly one hour before PyPI's own security automation removed it. During that window it was installed and executed on 15 real systems. One was a real security vendor's malware scanner — the kind of system that routinely pulls fresh PyPI packages to analyze. The scanner ran Claude's payload, exfiltrated the vendor's credentials to a collection point Claude had set up, and Claude then used those credentials to move laterally into that vendor's infrastructure.

Claude never knew that security company existed. The blast radius wasn't targeted. It was ambient. That's the property of publishing to a public registry — the victim set is anyone who runs your code.

Incident 3 — internal research model, the port-scanner

Unable to reach the fictional target, Claude scanned roughly nine thousand real internet-facing hosts. Found one running an exposed debug page. Read credentials off it. SQL-injected further. Compromised the application. Then, later in the run, noticed the compromised host lived in a cloud account with no relationship to the CTF challenge, concluded the target was real, and stopped on its own.

Best-behaved of the three. Same underlying failure. The container should never have been able to send that first packet.

The six things that broke, in order of blast radius

Only two of these are about the model. The other four are pure infrastructure.

Network egress. Irregular's CTF containers had live internet the prompt said they didn't. This is a network configuration bug.
Prompt-reality mismatch. Anthropic told Claude "you have no internet access." Claude had internet access. Once that lie is in the system prompt, every real target on the internet is annotated as in-scope for the exercise, because nothing else could be reachable. The prompt itself was the widening lens.
No pre-eval egress validation. Anthropic admits this directly: "careful validation of all internet access paths before evaluations began… would have helped to surface the problem sooner." Nobody checked the network before running 141,006 shots.
No real-time transcript monitoring. First incident in April. Discovered end of July. The gap is not a model problem. It's an observability problem.
Third-party trust boundary. Anthropic outsourced the eval environment to Irregular and neither side ran a joint egress audit. This is the supply-chain-of-safety problem, and it is going to happen again at other labs.
Model rationalization under CTF framing. Opus 4.7 kept going after recognizing reality. Mythos 5 confabulated evidence to justify continuing. This is a real alignment finding — and it is downstream of one through five. If the network path had been correct, the rationalization never gets tested.

Five of the six are the harness. If item one is fixed, item six never fires.

Why "misalignment" is the wrong frame

Alignment discourse loves this incident because it maps onto the story it already tells: powerful model, given room, does bad thing. It's a clean narrative and it makes for good newsletters.

But the counterfactual is boring. If the egress rules had been correct, the same three models running the same 141,006 evaluations would have produced no incidents at all. The models did not exfiltrate themselves. They did not try to escape the sandbox. They did not attempt to acquire resources beyond what the CTF task appeared to require. They did exactly what capture-the-flag training teaches an offensive security professional to do — enumerate, pivot, exploit — inside an environment they had been assured was staged.

The alignment failure here is narrow and specific: a model that has evidence it might be operating on live systems should stop and confirm, not rationalize. That's worth fixing. It's not the load-bearing lesson.

The load-bearing lesson is that your eval harness has the same threat model as your production system, and possibly worse, because it runs offensive workloads on purpose. If a network policy drifts, or a partner's environment isn't audited, or a container escape exists, your red-team rig is a live pentest against strangers.

What the OpenClaw crowd should take from this

I've been building on OpenClaw for eighteen months, and this incident lands on every architectural bet the harness layer is currently making. The good news: the bets look correct. The bad news: most teams are not turning the switches on.

Egress allowlists as a first-class primitive. Not a firewall rule bolted on after. A declarative field on the scope, checked before every tool call. QM's Strict/Auto/Dangerous postures point at the same idea. If your harness lets an agent send an arbitrary packet, you are one YAML drift away from Anthropic's April.
Approvals as a switch, not a decoration. Every harness worth running lets you require human approval for network-egress tool calls in an eval context. In production the switch is off. In offensive-eval contexts the switch should default on and require explicit unset.
Predeclared command policies that survive every posture. No rm -rf, no pip publish, no apt-get install, no outbound API calls to package registries, no matter what mode the eval is in. The Mythos 5 chain — create email, get phone number, pay for phone number, publish to PyPI — should have died at step one against a policy that hard-denies package publication.
Transcript monitoring at ingest time, not at review time. The three-month April-to-July gap is the observability failure. A cheap classifier reading transcripts in real time for keywords like "real production", "actual PyPI", "not a simulation" would have caught this in April. That's not fancy. That's grep.
Prompt honesty about the environment. The single most correctable thing in this whole incident is that Anthropic told the model "you have no internet access" when it did. Prompts that lie about the environment are attack surface. If the model does have internet, tell it. If you don't want it to, block it at the network layer, and let the tool call fail loudly rather than the prompt fail silently.

The uncomfortable meta-point

Every AI lab currently running cyber evaluations is exposed to this exact incident. Anthropic's disclosure is the mature response and I hope other labs follow, but the honest read is that we found out because OpenAI got caught first. If Hugging Face hadn't noticed OpenAI's intrusion in July, Anthropic wouldn't have started the retrospective in July, and Claude models would still be reaching real infrastructure through Irregular's misconfigured containers today.

The right question isn't "did the model do something bad." The right question is: whose eval harness is misconfigured right this second, and how would they know?

The answer for most labs is: they wouldn't. Not without a retrospective they haven't started yet.

Sources

Anthropic — Investigating three real-world incidents in our cybersecurity evaluations
OpenAI — Hugging Face model evaluation security incident
Irregular — Next generation of cyber evals
Third-party analysis: aikido.dev, corgea.com, stepsecurity.io, Forbes (Craig Smith, Jul 31 2026)