Anthropic lobbied Congress for an AI safety bill. The threshold it proposed deserves a closer look.

If you use an AI tool at work, at home, or on your phone — and essentially everyone does at this point — you've probably assumed somewhere in the back of your mind that someone is responsible for making sure it doesn't do something truly dangerous. The question of who that someone is, and what "dangerous" means in a legal sense, has been murky since the first chatbot launched.

Last week, Anthropic tried to answer it.

Politico reported on October 16 that Anthropic has been quietly pitching Congress a draft federal AI safety bill. The key mechanism: the government could seek to block an AI model only if it poses a foreseeable and imminent risk of more than $1 billion in damage or of killing or seriously injuring more than 50 people. Below that line? The government would wait.

Only the Commerce Department could file for a blocking order. A federal court would have to grant it. The suspension could last up to 45 days. The bill's coverage is narrow: biological weapons development, large-scale cyberattacks, and AI systems that escape their developers' control. Three categories, not a broad mandate.

Mother Jones called the threshold alarming. That's the headline most readers are seeing. But the story is more complicated.

What's real

The $1 billion number sounds high. In cybersecurity terms, it isn't, necessarily. The 2022 MGM Resorts ransomware attack cost the company over $100 million directly — and that was done by people, with conventional tools. A coordinated AI-assisted attack against financial infrastructure could hit $1 billion in damage fast. The category the bill is aimed at — extreme-scale cyberattacks — is precisely the scenario where a billion-dollar threshold is not a free pass. It's a real bar for a real catastrophe.

Fifty deaths, applied to the specific category of AI-assisted bioweapons development, is a sensible floor for intervention. You don't want "50 deaths" to sound like a lot in a bioweapons context, because a successful biological attack could produce casualties orders of magnitude higher. The intent of these thresholds is: reserve blocking power for clearly catastrophic, imminent harms that are provable enough to survive court review.

The procedural structure matters too. Under this draft, the Commerce Department can't unilaterally halt a model. It files a request. A federal court decides. The suspension maxes out at 45 days. That's not a rubber stamp — it's a meaningful check against regulatory overreach, which is a real risk in any industry that's politically charged.

What deserves a side-eye

The thresholds look designed to make blocking an AI very hard, not just possible.

To be blunt: a bill with a $1 billion damage threshold and a 50-death floor does not catch most of the harms people actually worry about AI causing — bad medical advice, discriminatory hiring algorithms, fraud at scale, election disinformation. None of those are in the three covered categories. If your concern is "AI might harm me in a way I'll never see coming," this bill is largely silent on your concern.

There's also the basic process problem. Anthropic wrote a bill that sets the conditions under which Anthropic's own models could be blocked. Even if the bill is well-designed — and parts of it are — that's a conflict of interest that deserves scrutiny regardless of intent. A pharmaceutical company drafting its own FDA approval thresholds would face questions. This should too.

The bill also gives no coverage to cumulative, slow-moving harms: AI displacing jobs, AI making financial markets more fragile, AI eroding trust in institutions. Those harms may be larger than any single billion-dollar event. They're nowhere in the draft.

Source spread

• Politico, via MLQ.ai — builder. Primary reporting on the draft bill. Most detail on the three-category structure and procedural design.
• Mother Jones — skeptic. Calls the threshold alarming and argues the bill sets a bad precedent for industry self-regulation of safety rules.
• TechSpot — builder. Clean summary of the specific numbers: $1B and 50-death floors, Commerce filing, court decision.
• LawCommentary — skeptic. Frames the bill as Anthropic quietly limiting government power; notes the industry-authorship issue.

What to do about it

This bill hasn't passed. It's a draft proposal that Anthropic shared with Congressional staff, not law. Here's what's practical right now:

• Notice who's writing the rules. Every major piece of AI policy drafted in the last two years has been co-written or heavily influenced by the companies that make AI. That's not unique to Anthropic. It's the current state of AI governance. Public input is mostly absent. That's the gap worth watching.
• Understand what law already covers you. This bill would add a government-blocking mechanism for extreme cases. It doesn't replace existing protections. Product liability, consumer protection law, and civil rights law already apply to AI harms in many contexts — they're just hard to invoke and underenforced.
• Watch the bill's trajectory. If it surfaces in actual Congressional hearings, that's the moment for public comment. Your representative's office takes constituent calls on pending legislation. One isn't nothing.
• Push for the categories that aren't in the draft. Bioweapons and cyberattacks got included because they're easy to agree on and politically legible. The harder, more consequential questions — AI in hiring, AI in lending, AI in medicine, AI in elections — still don't have a federal framework. That's the gap worth demanding answers on.

Further reading

• Anthropic pitches Congress a frontier-AI bill with a $1 billion harm threshold — MLQ.ai
• Anthropic's AI Safety Bill Has an Alarming Threshold for Government Action — Mother Jones
• Anthropic wants AI models blocked only if they threaten over $1 billion in damage or more than 50 deaths — TechSpot
• Anthropic Quietly Pushes AI Safety Bill That Would Limit Government Power to Block Dangerous Models — LawCommentary
• Anthropic's Draft AI Safety Bill: What It Would Do — CellCog