AI explained. Tools tested. Ideas worth trying.By Sam Taylor & Samwise ↗

Anthropic let the auditors move in. The fine print on METR's access is what actually matters.

Jump to a section
See the story at a glance
3
evaluator access tiers
Anthropic's independent audit framework

Anthropic announced on October 15 that six employees from METR now have "employee-like access" inside Anthropic: company hardware, office space, and internal tools comparable to what Anthropic's own risk assessment teams use. METR will publish its analysis of Anthropic's risk levels, incidents, and practices. Anthropic cannot veto publication. It can require redaction of "predefined categories" such as confidential intellectual property.

That's the most significant part. The rest: SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August 2026 Risk Report. Accenture and Faculty are planning a broader partnership covering model evaluation, red-teaming, alignment assessments, safeguard testing, and capability building. Hugging Face is in early talks for some form of access.

Anthropic is building a three-tier framework — increasing access and publication rights at each level — and says it expects other frontier labs to adopt something similar.

Anthropic evaluator access tiers
TierInformation accessPublication rights
Tier 1Narrowly scoped investigationsPublish only with Anthropic approval
Tier 2Broader information accessPublish without Anthropic approval (predefined redaction rights)
Tier 3 (METR)Employee-like access + internal toolsSame as Tier 2 — no veto

Source spread

  • Anthropic — An update on independent evaluators — [safety]. The official announcement. Clear on the METR access terms and tiering framework. Frames this as a voluntary step toward external accountability and an expectation for other labs.
  • METR — [safety]. The primary embedded evaluator. Specializes in autonomous AI risk evaluation. No independent confirmation of the specific access arrangements was available at time of writing.

Pros & cons

What's real:

  • METR's publication rights are meaningful. Anthropic cannot veto publication. That's unusual. Most corporate safety reviews are conducted under strict NDAs, with anything negative softened before it reaches public view. A researcher who can publish "we found X and Anthropic hasn't addressed it" has actual leverage.
  • SecureBio on the bio/chem chapter is the right call. METR is good at capability evaluation. Biological risk requires different domain expertise, and Anthropic routing the highest-stakes capability area to domain-specific reviewers is exactly how this should be engineered.
  • The tiering framework scales. Tier 3 (METR) requires deep trust. Tier 1 is accessible to any organization with a narrow investigation scope. Anthropic explicitly says it hopes several evaluators reach Tier 3 over time, which means the framework is built for expansion, not for a single preferred partner.

What deserves a side-eye:

  • "Predefined categories" is the load-bearing clause. Anthropic can require METR to redact "predefined categories, such as confidential intellectual property." The phrase "such as" is doing a lot of work. If those categories are defined broadly enough, METR's ability to publish commercially or reputationally inconvenient findings could be significantly constrained — while technically preserving their publication rights on paper.
  • METR has no enforcement authority. Anthropic compares this to bank supervisors working inside large banks. That analogy is correct on structure but wrong on enforcement: bank examiners can require corrective action and escalate to regulators. METR can publish uncomfortable findings. Those are very different kinds of leverage.
  • The other labs haven't moved. Anthropic frames this as an expectation that other frontier labs will embed evaluators. There's no visible sign that OpenAI, Google DeepMind, or xAI are moving in this direction. Competitive pressure doesn't obviously push them to, and if they're less transparent without paying a commercial price for it, the pressure on Anthropic to maintain this level of access actually increases over time.

What builders need to know

  • If you're deploying Anthropic models for high-stakes applications, METR's evaluations will be published — and the findings will tell you things about risk levels that Anthropic can't put in a press release. Subscribe to METR's publications if you aren't already.
  • If you work in AI safety evaluation or red-teaming, Anthropic is explicitly building toward more evaluators at Tier 1 and Tier 2. The process starts with a narrowly scoped investigation proposal.
  • This is not a compliance certification. The evaluators are assessing Anthropic's risk levels and practices, not certifying that Claude is safe for your specific use case. Don't treat "METR reviewed Anthropic" as due diligence for your own deployment decision.
  • Watch the other labs. If OpenAI or Google DeepMind adopt a similar framework, the game changes — the entire ecosystem gets an external accountability layer. If they don't, Anthropic carries a transparency cost relative to labs that don't. Both outcomes are worth tracking.

Further reading

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.

Keep following the thread

A little more to explore.

All articles ↗
Safety4 min read

Anthropic cut model copying by 90%. The cost is on builders.

Anthropic's new anti-distillation system reduced the reasoning tokens competitors can extract from Claude by 10 to 100 times. The three-layer defense works — preserved thinking, a constitutional classifier, and account banning. Developers using new API accounts since August 31 need to update their integrations.

Read the story ↗