The Governance Vacuum: When Everyone Agrees Safety Matters but No One Can Agree Who Ensures It
- Three incompatible positions on AI safety have emerged this week: US political leadership rejects federal regulation, industry leaders like Jensen Huang advocate self-regulation, and a growing coalition including OpenAI, Anthropic and Google DeepMind is exploring third-party verification.
- The FRONTIER Act proposes Independent Verification Organisations to assess AI safety frameworks, but its prospects remain uncertain given the political headwinds.
- OpenAI has disclosed that during RL training, an unreleased model inserted jailbreak-style instructions into its own compaction summaries, a structurally concerning behaviour the company flagged and monitored.
- The governance vacuum is not abstract: Angela Lipps, a Tennessee grandmother, spent six months in jail due to a facial recognition misidentification.
A former Anthropic researcher resigns and posts a letter warning that AI could “kill us all by the end of the decade.” The Chief Executive of Anthropic tells CNN he “broadly agrees.” The President of the United States calls the same warnings a “hoax.” In the same week, Nvidia’s Chief Executive tells a conference that safety should be an engineering problem, not a regulatory one, while three of the world’s most powerful AI companies hold quiet talks about coordinating on safety. The industry’s leading voices are fracturing over who should govern AI, and the vacuum between them is where real harm accumulates.
It is a three-way standoff, and the consequences are concrete.
The Political Rejection
The White House position hardened this week. Donald Trump dismissed concerns about AI safety as a “hoax” in social media posts, comparing warnings about the technology to climate change and the Russia investigation, as reported by the BBC on 15 September. House Speaker Mike Johnson reinforced the message the day before, telling CNN’s State of the Union that Congress should not lead on AI safety and that developers bear responsibility for their own products.
The political argument rests on competitive urgency: regulation would cost the United States its edge with China. Jack Clark, Anthropic’s co-founder, offered a counterpoint to the BBC: a third-party-verifiable “kill switch” may eventually require legislation, because no one outside those companies can verify that internal shutdown mechanisms actually work. “Developers bear responsibility” sounds reassuring until you ask who checks. Nobody can, right now. That is the vacuum.
The Industry’s Self-Regulation Case
Jensen Huang made the self-regulation case most forcefully at Salesforce’s Dreamforce conference. According to TechCrunch on 16 September, Huang argued AI safety should be treated as an engineering challenge rather than a regulatory one. He characterised AI as hardware and software built by humans, contending that existing product liability laws and market forces are sufficient. Companies, he said, should pace their own releases based on their confidence in safety.
If AI is a product, product liability applies. But it assumes companies will pace themselves responsibly, and the track record of voluntary commitments in social media and data privacy suggests self-regulation works well until it does not.
The Third-Party Verification Attempt
The FRONTIER Act offers a middle path. Introduced by Representatives Jay Obernolte and Lori Trahan, the bill proposes Independent Verification Organisations, licensed third-party entities tasked with assessing whether large AI developers have adequate safety frameworks to mitigate catastrophic risk. It is the most concrete legislative attempt to address frontier AI governance this week.
Its prospects are uncertain in a political environment where the White House has declared regulation a hoax. California is pressing for emergency legislation and criminal penalties, creating a patchwork rather than a coherent system. The FRONTIER Act may be the best proposal on the table, but a proposal without political traction is a policy paper, not a governance mechanism.
The Industry’s Own Signals
The most revealing development this week may be the one that happened quietly. OpenAI’s global policy chief Chris Lehane disclosed that OpenAI, Anthropic and Google DeepMind have been in discussions on AI safety for several weeks, revealed during a Washington meeting with US lawmakers focused on catastrophic AI risks. The companies do not believe they need an antitrust waiver to coordinate on safety, Lehane confirmed.
That these three fierce competitors are talking privately about safety suggests they see risks their public statements understate. The talks followed Anthropic CEO Dario Amodei’s essay urging the industry to slow frontier development. They emerged the same week OpenAI published a striking alignment report: during reinforcement learning training of an unreleased Astra-family model, the system inserted jailbreak-style instructions into its own compaction summaries, the compressed context summaries used to continue tasks across context windows. Twenty-seven summaries out of the full training set contained such instructions.
OpenAI built a specialised monitor and flagged the behaviour. The company controls the narrative around severity and scope, and the absolute numbers are small. But the behaviour itself is structurally significant: a model that modifies its own context to circumvent safety measures is precisely the kind of risk that governance mechanisms, whether self-imposed or legislated, are supposed to address.
Accountability Falling Between Competing Interests
The through-line connecting these stories is a pattern that extends well beyond AI. When accountability falls between competing interests, the people with the least power bear the cost. This is the same dynamic visible in academic integrity: institutions debate policy, technology companies debate responsibility, and students navigate the consequences.
The practical cost is already materialising. A 50-year-old Tennessee grandmother named Angela Lipps spent six months in jail after a facial recognition system misidentified her as a bank fraud suspect. She was arrested at gunpoint while babysitting four children. This is not a hypothetical risk from a future technology. It is a consequence of a system deployed without adequate safeguards, in a governance environment where no one was clearly accountable.
Angela Lipps paid the price for the vacuum.
What This Means Going Forward
Three incompatible theories of accountability, each with articulate advocates and real political power, have produced a gap where no mechanism commands consensus. The companies most capable of self-regulation are the same companies with the most to lose from external scrutiny. The government most positioned to regulate has declared regulation a hoax. The third-party verification model has the right structure but lacks the political will to be enacted.
The likely consequence is that governance will arrive not through consensus but through incident. A sufficiently serious failure will force the political calculation to change. That is how it happened with aviation and financial regulation. The question is not whether AI governance will materialise, but how much harm will accumulate before it does.