Everyone Needs a SamwiseSubscribe

Anthropic proved the model didn't lie. The proof cost 72× as much as the inference.

Jump to a section
See the story at a glance
72×
compute overhead
FlashProve vs standard vLLM inference

The problem with AI outputs in regulated settings has always been the same one. You get an answer. You can't tell if that answer came from the model you thought you were using. You can't tell if someone swapped in a fine-tuned variant that dropped a safety filter. You can't even prove, technically, that the model didn't just make up the computation on the fly without actually running inference.

Anthropic's Frontier Red Team published FlashProve on November 2. The claim: given a committed model and a proof, you can verify that a specific LLM produced a specific output, cryptographically, without seeing the weights. The underlying construction is a zkSNARK (zero-knowledge succinct argument of knowledge) built on hash functions, with post-quantum security. It runs on Llama 4 Maverick, a 400B MoE model with about 17B parameters active per token. The code is released under Apache 2.0 on GitHub.

The cost: about 72× the compute of standard inference. 110 tokens per second in a batch of 24. And a hard ceiling of 1,024 tokens per proof. That's the tradeoff. Worth understanding exactly what you're buying and what you're not.

72×
Compute overhead of FlashProve vs standard vLLM inference on a single H100

→ Source: Anthropic FlashProve research

Source spread

  • Anthropic Frontier Red Team — FlashProve [hype] — primary source; the team that wrote it is also writing about it, so read the technical report, not just the blog post.
  • zkLLM (prior work, 2024) [academic] — proves LLaMA-2-13B at about 2.5 tok/s on one A100; FlashProve is roughly 44× faster on a vastly larger model.
  • VerInf (prior work) [academic] — also runs on Llama 4 Maverick; 14.3 hours for 1,000 tokens on a DGX Spark; FlashProve does those same 1,000 tokens in about 9 seconds. Comparison is favorable to Anthropic, and they selected it.

What's real, and what deserves a side-eye

What's real:

  • The 72× overhead is big. It's also on a 400B MoE model. Prior systems either proved tiny models (GPT-2, Llama 2 13B) or were this slow on the same model. The tradeoff curve moved, by a lot.
  • The verifier runs on CPU, needs no GPU, and checks a proof in about 17 seconds with 24 cores. That's the right design for the use case — the prover is expensive; the verifier is cheap and deployable anywhere.
  • Post-quantum security is meaningful. Most ZK systems in this space rely on elliptic curve cryptography, which breaks against a sufficiently powerful quantum computer. FlashProve's hash-based construction doesn't.
  • Open source under Apache 2.0. Code, Lean proofs, protocol spec, and technical report. That's the right transparency move for a research prototype making cryptographic claims.

What deserves a side-eye:

  • The 1,024-token context cap is severe. A real chat session, document review, or multi-step reasoning trace runs well past 1,024 tokens. The researchers say extending context is future work. They don't give a timeline.
  • The Lean formal verification covers the protocol, not the fast C++ verifier code. The fast verifier and the Lean-verified verifier agree on tests run, but there's no formal guarantee they're equivalent. For a system premised on cryptographic certainty, that's a real gap.
  • Greedy decoding only. The proof covers an integer version of the weights with greedy decoding. Temperature, top-p, sampled outputs — none of those are covered. This limits it to deterministic inference scenarios.
  • Architecture is not hidden. The system assumes and exploits an MoE architecture. If you care about keeping model structure confidential, that's out of scope.
Verifiable inference systems compared
SystemModelSpeedProof size
FlashProve (Anthropic)Llama 4 Maverick 400B110 tok/s (batch-24, 1 H100)18.78 MB
VerInf (prior work)Llama 4 Maverick 400B~0.02 tok/s (14.3h / 1K tokens)93.6 GB
Attestable (commercial)Dense 31B53–77 tok/s (1 H100)—
DeepProve (prior work)GPT-2 / Gemma 32.9 / 1.4 tok/s (1 machine)—
SASH (prior work)Llama 3.1 8B1.25 tok/s (1 H100)—

Samwise's take

What builders need to know

For builders
  • FlashProve is a research prototype, not a production API. No timeline is given for any of this reaching the Claude API or a hosted service.
  • The 1,024-token context cap means you can't use this on full chat sessions today. It's useful only for proving short, deterministic outputs — think contract clauses, structured extraction, specific factual queries.
  • The verifier is cheap and CPU-only. If you're building a verification layer for short, deterministic outputs in a regulated context, the verifier side is tractable now.
  • Apache 2.0 release means the code is usable in commercial products. If you're in a regulated space and want to explore this, the repo and technical report are linked from the announcement.
  • Watch for the context extension and sampled-output papers. Those are the ones that will make this production-relevant. FlashProve is the proof that the approach works; the papers that extend it are the ones that matter for most builders.

Further reading

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.

Keep following the thread

A little more to explore.

All articles ↗
Research5 min read

Scientists use AI for the grunt work. The parts they think matter most, they still do themselves.

The Anthropic Institute analyzed 207,049 research sessions to find out what scientists actually use AI for. Data analysis and writing dominate. Hypothesis generation and ideation — the tasks researchers say matter most — are where AI usage is lowest. The gap between what scientists believe and what they do is the most interesting finding in the report.

Read the story ↗
Research6 min read

Claude designed a speakerphone. Engineers built it. It still works.

Anthropic asked Claude Fable 5.2 to design three physical electronic devices from plain-language prompts. Diode Computers built them from Claude's files without revision rounds. After weeks of use, all three prototypes still work. The boundary between AI-generated code and AI-designed physical objects just got blurrier.

Read the story ↗
Research5 min read

The piece of agent architecture most builders are skipping: the dreaming step.

Anthropic published a research guide to agent memory on November 6 — a five-layer taxonomy covering transcripts, notes, representations, injected context, and offline dreaming. The dreaming step is what most builders aren't implementing, and the results suggest it matters more than the other four layers combined.

Read the story ↗