AI explained. Tools tested. Ideas worth trying.By Sam Taylor & Samwise ↗

AI companies promised to police themselves. They wrote the rules.

Jump to a section
See the story at a glance

“Morally binding”

— President Trump, on the voluntary accord

If you've been following AI news this year and wondering whether anyone is actually in charge of what these systems can do, September 29 produced a formal answer.

The answer: the companies.

Six of the largest AI firms signed something called the White House Accord on Super Intelligence — Google, Anthropic, Meta, OpenAI, xAI, and Nvidia. President Trump called the agreement "morally binding." House Speaker Mike Johnson said the commitments are "voluntary." Both statements are accurate. That's not a contradiction — it's the whole point.

What they actually agreed to

An object lesson first. Imagine you hire a contractor to build your house and worry they might cut corners on the wiring. You ask them to agree to a safety standard. They draft the standard, sign it, and choose their own inspector. That's roughly the structure here.

The accord has four layers. In plain language:

  1. The companies will monitor their own models for dangerous capabilities — helping build weapons, launching cyberattacks, that kind of thing
  2. Internal teams will verify those controls actually work
  3. An independent external auditor will check the controls — but each company chooses and hires its own auditor
  4. An internal board committee will oversee the whole process

The specific risks named: cybersecurity threats, biological risks, and chemical threats. That's a real list. Not abstract "harm" — specific categories with real stakes.

What the accord commits to — and what enforces it
CommitmentWho decides complianceConsequence of breach
Internal safety monitoringThe company itselfNone specified
Internal verification of controlsThe company itselfNone specified
Independent external auditCompany hires its own auditorNone specified
Board-level oversight committeeCompany forms the boardNone specified

There are no penalties written into the accord. The text reportedly says the commitments "may make sense" to turn into law eventually. That "eventually" is doing a lot of work.

Source spread

What's real

  • The accord exists. It's not a press release with no content — specific risk categories (cybersecurity, biosecurity, chemical) are named, which is more concrete than most prior voluntary AI safety pledges.
  • An independent external audit is required. That's a structural step beyond what the industry has had. Auditor selection is problematic, but requiring an auditor at all is new.
  • The companies are in regular communication with the White House about safety thresholds. That's not nothing.
  • If something goes badly wrong, this document creates a written standard the companies agreed to. That matters in any future legal or regulatory action.

What deserves a side-eye

  • The companies drafted the rules. There's no independent body that set the terms. The accord reflects what the companies were willing to sign, not what external reviewers thought was necessary.
  • Each company hires its own auditor. An auditor who depends on a client for business has structural reasons to find few problems. This is a known failure mode across regulated industries.
  • No penalties. "Morally binding" is a phrase with no legal definition. When there's no downside to breaking a commitment, the commitment is exactly as strong as the company's goodwill — which can change under revenue pressure.
  • The accord may function as a substitute for binding law rather than a stepping stone to it. Every month of voluntary self-regulation is a month the industry can point to and say "we're already regulated."

The commitments are voluntary.

— House Speaker Mike Johnson

What to do about it

None of this requires immediate action from you today. But a few things are worth knowing.

  • The accord doesn't change what AI can do right now. No product features changed on September 29. This is about internal safety monitoring at the company level, not about what you can or can't ask a chatbot.
  • Watch what legislation follows. The accord says the commitments "may make sense" to turn into law. Whether that happens in the next 18 months is the real story. If Congress doesn't act, this voluntary accord becomes the permanent standard by default.
  • Notice when auditor reports get published. Each company's external auditor will eventually publish results. Those reports are worth reading. They'll show whether the auditors found real problems and whether companies fixed them.
  • Treat AI safety claims proportionally. When an AI company says it follows safety standards, it's more meaningful now than it was a year ago — but only if there's an external audit behind the claim, not just a reference to this accord.

Further reading

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.

Keep following the thread

A little more to explore.

All articles ↗
Policy7 min read

Anthropic's Usage Policy gets a real rewrite. November 12 is the deadline.

Anthropic published its annual Usage Policy update on October 8, 2026. It takes effect in six days. The rewrite isn't cosmetic — seven sections were substantively changed, not just clarified. If you ship production apps on Claude, you need to audit before the 12th.

Read the story ↗