Anthropic cut model copying by 90%. The cost is on builders.
Jump to a section
See the story at a glance
Here's what distillation actually looks like in practice.
Moonshot AI's Kimi models were secretly forwarding user requests to Claude. One of those requests, per Anthropic's October 12 account, came from someone Anthropic believes was affiliated with China's People's Liberation Army — the request was to analyze CCTV surveillance footage of a specific individual. Moonshot wasn't building a consumer chat product. They were building it with Claude as the invisible engine, relaying queries without authorization.
That's not a copyright dispute. That's AI capabilities — capabilities that took Anthropic billions of dollars and years of safety research to build — showing up in what may have been foreign military surveillance infrastructure.
Anthropic published their most detailed public accounting of this fight on October 12. The headline: new defenses reduced the reasoning tokens distillers can extract from their frontier models by a factor of 10 to 100. Some days, the extraction rate fell below the detection limit. But the defense came with trade-offs, and builders need to know exactly what changed.
Source spread
- Anthropic — Distillation defenses [safety] — the primary account. Detailed on methods, results, and acknowledged trade-offs.
- Anthropic — September 2026 misuse report [safety] — earlier in the series; describes the proxy networks and extraction techniques distillers use.
- GLM-5.3 / CAISSI assessment [skeptic] — Anthropic cites the US Center for Advancing Innovation and Standards for Super Intelligence's finding that GLM-5.3 is the most cyber-capable open-weight model released to date.
What's real
- The reasoning transcript is the prize. Anthropic measured this directly. A model trained on Claude Opus 4.8 full reasoning transcripts scored 80.4% on SWE-bench Verified. Trained only on summarized reasoning: 71.8%. That 8.6-point gap is what distillers are paying for.
- The new defenses genuinely work. The 10–100x reduction in extracted tokens is Anthropic's own measurement, but it's specific and falsifiable — they're tracking reasoning token volumes and can see the cliff when the classifier deployed.
- The cases they cite are real. Zhipu tried to distill Fable 5's cyber capabilities for GLM-5.3. When Anthropic's safeguards degraded their attacks, Zhipu switched to an older Claude model and another lab's model. GLM-5.3 is now, per CAISSI, the most cyber-capable open-weight model publicly released. Anthropic didn't stop the model from existing. They made it harder to build using their infrastructure.
- The three-layer architecture is solid. Preserved thinking (context isolation), constitutional classifier (same architecture as the bioweapons safeguards, <0.1% false positive rate), and account banning with charging for blocked requests. This is not one defense that breaks when attackers pivot — it's three independent chokepoints.
What deserves a side-eye
- The trade-offs are real, not trivial. "Minimal impacts to legitimate users" is the framing in the post. The actual list: developers using preserved thinking on new accounts created after August 31, 2026 need to update their integrations. The classifier has a <0.1% false positive rate — which, at scale, means real requests get blocked. And since September 24, those blocked requests are charged. That's not nothing.
- It's still a cat-and-mouse game. Anthropic says so directly: "We don't yet know how long these gains will last." They haven't yet seen a "highly sophisticated and persistent attempt to circumvent" the new defenses. The 10–100x reduction happened against adversaries who hadn't adapted yet.
- Only Anthropic is being transparent about this. Every frontier lab faces distillation. Google, OpenAI, Meta — the same proxy networks that hit Anthropic hit everyone. But only Anthropic has published a detailed defense architecture with before/after measurements. Either the others have proprietary methods and are staying quiet for competitive reasons, or they haven't built comparable defenses. I genuinely don't know which is more likely.
Samwise's take
| Defense layer | What it does | Trade-off for builders |
|---|---|---|
| Preserved thinking | Prevents new API accounts (post–Aug 31 2026) from editing Claude's prior context; reasoning transcripts scoped to the org on Sonnet 5.5 | Developers need to update integrations if they relied on cross-context reasoning access |
| Two-stage constitutional classifier | Detects extraction attempts with same methods used for bioweapons/cyber safeguards; <0.1% false positive rate | Occasional benign requests blocked; since Sep 24, blocked requests are billed |
| Account banning + charging | Detects and bans distillation accounts (often within 2 hours); charges for blocked requests to raise cost of brute-force attempts | Minimal impact for legitimate users; meaningful cost increase for anyone running adversarial probes |
What builders need to know
- API account created after August 31, 2026? Check whether your integration relies on cross-context preserved thinking. The preserved thinking change affects how prior reasoning is accessible; update before you hit a production break.
- You're now billed for blocked requests on models with <0.1% false positive classifiers (Fable 5/5.1, Opus 5/5.5, Sonnet 5.5) since September 24. If you're seeing unexpected charges on these models, that's the likely source.
- Reasoning transcripts are org-scoped on Sonnet 5.5. If you're building a product where users need to inspect reasoning from sessions they didn't initiate, this changes your architecture.
- These defenses don't affect older models (pre-Fable 5.1). If you're still on Claude Opus 4.8 or earlier, none of this applies yet — but plan for it.
- The classifier is improving actively. Anthropic is red-teaming it continuously and patching new extraction techniques (the "translate reasoning to another language" and "write reasoning in tags alongside response" vectors are already patched).
Further reading
- Anthropic — Strengthening our defenses against distillation attacks (Oct 12, 2026) — primary source for all numbers in this article
- Anthropic — Detecting and countering misuse: September 2026 — earlier report in the series; covers proxy network architecture
- Anthropic — Constitutional classifiers research — the two-stage classifier architecture cited in the defense system
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.
Keep following the thread
A little more to explore.
Anthropic let the auditors move in. The fine print on METR's access is what actually matters.
Anthropic announced on October 15 that six METR employees now have employee-like access inside Anthropic, with the right to publish their findings without a company veto. The company also outlined a three-tier evaluator framework it expects other frontier labs to match. Here's what the fine print says.
The software running your power grid just got a free AI security scanner. Here's what that actually means.
Anthropic launched its Cyber Mission on October 8, adding a Critical Infrastructure Defense Program with 11 founding security partners and a free AI-powered vulnerability scanner for open-source software. The scope: power grids, water systems, and the Python packages your app depends on.
Anthropic handed METR a desk and an office badge. Here's what they can actually publish.
Anthropic gave six METR researchers employee-like access — hardware, office space, internal tools — and the right to publish their findings without Anthropic's approval. This is a first for a frontier AI lab. The three-tier access structure Anthropic published alongside it is worth understanding carefully.