Anthropic draft bill would block AI model release over 50 deaths
- Anthropic has drafted proposed federal legislation and presented it to members of Congress, with a spokesperson confirming the text reflects its Advanced AI Framework.
- Distribution could be blocked where release poses a foreseeable, imminent risk of more than 50 deaths or over one billion dollars in property damage from a biological weapon, control evasion or cyberattack.
- Independent evaluators registered with the US Commerce Department would assess risk and could be funded by the labs, provided payment does not depend on findings.
- An evaluator finding imminent catastrophic risk would have 72 hours to notify Commerce, which would seek a release halt from a judge in Washington, D.C.
A federal AI safety bill drafted by Anthropic would let the US government stop a model’s distribution before release. The company has been presenting the text to members of Congress. As TechCrunch’s account of the draft legislation sets out, the trigger is deliberately narrow. Distribution could be halted where release carries a foreseeable, imminent risk of more than 50 deaths or over one billion dollars in property damage. That applies only where the risk arises from a biological weapon, a model evading its developer’s safeguards, or a cyberattack. The thresholds track California’s Senate Bill 53, which treated comparable scenarios as catastrophic risk.
Enforcement would run through private assessors, with a judicial backstop. Independent evaluators registered with the US Commerce Department would carry out the risk assessments. Those evaluators could be paid by the AI labs themselves, on condition that payment does not depend on the findings. The draft offers no stated mechanism for preventing a funder from shaping an outcome. Where an evaluator identifies imminent catastrophic risk, it would have 72 hours to notify Commerce. Commerce would then seek a release halt from a judge in Washington, D.C. No federal agency could move against a model on its own initiative.
For universities, colleges and research groups, the significance sits in the gate rather than the threshold. A licensed evaluation step before distribution would make assessment records part of the infrastructure that teaching and research depend on, at a point where institutions have little standing to inspect them. The draft describes how a finding reaches a court. It says nothing about what a school, library or research consortium running the same model would be told. Independence of the evaluator is the load-bearing claim, and the funding arrangement leaves that claim to be demonstrated rather than assumed.
Whether the text becomes law is another matter. Much of the industry has argued for voluntary oversight, and Nvidia’s Jensen Huang has said no new legislation is needed. The proposal also addresses a different class of harm from the agentic hacking incidents that have dominated industry discussion, which shapes the debate it enters. Watch for a sponsor willing to introduce it, and for revised language on how evaluator payment is firewalled from evaluator conclusions.