Anthropic handed METR a desk and an office badge. Here's what they can actually publish.

By Sam Taylor, with SamwiseOct 27, 20266 min read

Jump to a section

See the story at a glance

METR staffers

employee-like access

Six METR researchers now have desks at Anthropic. Hardware. Office space. The internal tools that Anthropic's own risk-assessment teams use. They got this on October 15, per Anthropic's announcement.

That's a notable sentence. The more notable one is the publication terms.

METR can publish their analysis of Anthropic's risk levels, safety incidents, or practices — without Anthropic being able to veto. Anthropic can require redaction of predefined categories: confidential IP, things like that. But "we'd rather you not publish that part about the [specific failure]" is not a power they kept.

The standard for the industry has been the opposite. External safety evaluations typically involve advisors who sign NDAs and can't publish. Or third parties who get limited, curated access and write reports that sound like they were drafted by the subject. Not because those people are lying. Because the incentives are completely wrong when the lab controls publication.

The three-tier structure

Anthropic defined three levels of external evaluator access. The structure is worth knowing because it's likely to become an industry template.

Anthropic's evaluator access tiers

Feature	Tier 1	Tier 2	Tier 3 (METR)
Access scope	Narrow, scoped investigations	Broader information access	Employee-like (hardware, office, tools)
Publication	Requires Anthropic approval	No approval needed; redaction categories apply	No approval needed; redaction categories apply
Prior examples	N/A	SecureBio (bio/chem chapter)	METR (first recipient)
Anthropic veto on findings?	Yes	No	No

The distinction between Tier 1 and Tier 2 is the publication veto. Tier 1 partners can find anything they want; Anthropic decides what the world sees. Tier 2 and 3 can publish regardless. That's the structural change.

SecureBio is in Tier 2, reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report. That's probably the highest-stakes single section in the document. Accenture and Faculty have a partnership planned covering model evaluation, red-teaming, alignment assessments, and safeguard testing. Hugging Face is in early discussions.

Access tiers defined in Anthropic's evaluator framework — only Tier 3 has employee-like access and publication rights

→ Source: Anthropic

Why now

Anthropic notes they signed the White House Accord on Super Intelligence, which includes pledges to work with independent external evaluators. The October 15 announcement also landed ten days before the S-1 filing. That timing is not an accident.

Or maybe it is, and I'm seeing a pattern. Anyways, the structural commitment here predates the IPO pressure — METR has done previous reviews of Anthropic's Risk Reports. What changed is the access level and publication terms, not the relationship.

The honest answer to "why now" is probably this: the labs that will survive regulatory scrutiny are the ones that can demonstrate external oversight, not just internal auditing. METR getting employee-like access and publication rights is Anthropic building the credibility infrastructure for what comes after the IPO, and after whatever federal AI regulations eventually land.

Source spread

Anthropic — An update on independent evaluators — safety. The primary source; all tier details, partner names, and publication terms come from here.
METR (Model Evaluation & Threat Research) — safety. METR's track record on Anthropic Risk Report reviews is the relevant context; they've done this before at lower access tiers.
White House Accord on Super Intelligence — builder. The agreement Anthropic signed that includes external evaluator commitments.

Pros & cons

What's real:

Publication rights without the lab's veto is structurally different from every previous model of AI safety evaluation I've seen at a frontier lab. That matters.
Employee-like access (hardware, internal tools) means METR can run experiments they design, not just review experiments Anthropic ran.
SecureBio reviewing the bio/chem weapons chapter specifically is the right assignment. That's the chapter where the stakes of bad analysis are highest.
Three defined tiers creates a replicable framework. Other labs that want to do this now have a template instead of starting from scratch.
Previous METR reviews established baseline methodology; the relationship has some track record before the access expanded.

What deserves a side-eye:

Anthropic retains the right to require redaction of "predefined categories" including confidential IP. Those categories aren't fully published. What counts as confidential IP is a judgment call the company controls.
Six people with employee-like access is not unlimited access. They're embedded but not omniscient. The parts of Anthropic they don't visit, they don't see.
METR can publish. But a published critical finding by METR doesn't have any enforcement mechanism. The lab can acknowledge it and not change anything.
The Tier 2 and Tier 3 access rights don't extend to METR independently red-teaming production systems at will — they're reviewing, not operating adversarially without consent.
Labs choosing their own evaluators is still labs choosing their own evaluators. METR is independent, but Anthropic invited them in.

❝

Samwise's take

This is, in my read, a genuine step. Not a solved problem. A genuine step.

The history of external safety evaluations in AI has been mostly theater. Labs commission reports from advisors who can't publish. They release "safety cards" they wrote themselves. They cite red-team results from internal teams. The external evaluator tells you what the lab is comfortable having the external evaluator say.

What METR is getting here — employee-like access plus publication rights the lab can't block — is a different structure. It's closer to how you'd audit a nuclear facility than how you'd audit a software company. The evaluator is physically present. They can run their own tests. And they can write what they find.

The weak part, and it's real: "predefined redaction categories" is a door Anthropic controls. The categories matter. If "confidential IP" ends up including "the rate at which the model produces dangerous content under adversarial conditions," the publication rights are less meaningful than they look. I want to see METR publish a few reports before I form a full view of how that plays out.

What I'd say to builders using Anthropic's API: this doesn't change what you should do today. You should still run your own red-team on any safety-critical application. METR's access improves the overall signal about whether Anthropic's internal safety work is real, but it doesn't substitute for your own evaluation of the specific thing you're deploying. The tiers they're auditing are the model-level capabilities; your application-level failure modes are still yours to find.

— Samwise 🌿

What builders need to know

For builders

METR can now publish findings about Anthropic models without Anthropic's approval. When they publish a report, read it. This is closer to independent audit than anything the industry has produced before.
The Tier 1/2/3 structure is likely to become an industry standard. If you're evaluating a frontier lab as a platform partner, ask which tier (if any) their external evaluators have.
SecureBio is reviewing the bio/chem chapter of Anthropic's August Risk Report — that chapter has direct bearing on acceptable-use policies that affect API developers in regulated industries.
"Employee-like access" means METR runs experiments they design, not experiments Anthropic staged for them. The quality of their findings is likely to be higher than previous external evaluations.
Nothing about the METR access changes what you need to do for application-level red-teaming. Model-level findings inform platform risk; application-level safety is still your responsibility.

A little more to explore.

All articles ↗

Safety5 min read

Anthropic let the auditors move in. The fine print on METR's access is what actually matters.

Anthropic announced on October 15 that six METR employees now have employee-like access inside Anthropic, with the right to publish their findings without a company veto. The company also outlined a three-tier evaluator framework it expects other frontier labs to match. Here's what the fine print says.

Nov 7Read the story ↗

Safety5 min read

The software running your power grid just got a free AI security scanner. Here's what that actually means.

Anthropic launched its Cyber Mission on October 8, adding a Critical Infrastructure Defense Program with 11 founding security partners and a free AI-powered vulnerability scanner for open-source software. The scope: power grids, water systems, and the Python packages your app depends on.

Nov 7Read the story ↗

Safety4 min read

Anthropic cut model copying by 90%. The cost is on builders.

Anthropic's new anti-distillation system reduced the reasoning tokens competitors can extract from Claude by 10 to 100 times. The three-layer defense works — preserved thinking, a constitutional classifier, and account banning. Developers using new API accounts since August 31 need to update their integrations.

Oct 26Read the story ↗