AI companies seek to regulate the risks they created

Two of the most prominent AI companies have publicly stated that their most advanced systems pose serious risks and require regulation and independent testing. The admission, delivered through essays, social media posts, and speeches at the United Nations including a Security Council session during the 81st General Assembly on 23 September 2026, is notable for its timing as much as its content. Both companies are approaching significant commercial milestones, including anticipated stock market listings, while the United States heads toward midterm elections.

The practical question is what form any regulation might take and who would define its terms. Anthropic and OpenAI are doing more than warning about risk; they are actively advocating for a specific regulatory architecture that would likely grant them substantial influence over the testing and oversight regimes applied to their own products. Self-policing has a mixed record in other industries, and the AI sector has not yet demonstrated that voluntary commitments translate into durable safety improvements. OpenAI has paused training of its most advanced model and has discussed potential development pauses with Anthropic and Google, though experts quoted in the Associated Press’s report on the companies’ risk warnings questioned whether such self-governance would prove sufficient.

Whose risks are being centred

The framing matters. By emphasising existential risk, the companies are drawing attention toward speculative, long-horizon dangers. Former OpenAI geopolitics lead Sarah Shoker has argued that this focus diverts scrutiny from more immediate harms: AI deployment in military applications, surveillance infrastructure, and the environmental costs of the data centres that power these systems. These are not hypothetical. They are happening now, and they affect communities that have little representation in the boardrooms where regulatory preferences are being formed.

For education and academic work specifically, the dynamic carries a particular irony. Universities and schools are being pressed to adopt AI tools rapidly, often on the vendors’ own terms, while the same vendors lobby for regulatory frameworks that would legitimise those tools at a national level. Institutions end up as both the market and the subject of the regulatory conversation, with limited leverage to demand transparency about how these systems were trained, what biases they carry, or how their outputs are validated. Academic integrity depends on verifiable provenance and honest attribution, qualities that current AI systems do not reliably provide and that the companies’ preferred regulatory models may not require.

What comes next

The United States political landscape complicates any near-term regulatory progress. President Trump has dismissed AI risk concerns as a hoax benefiting China, while his administration’s science adviser David Sacks has characterised calls for slowdowns as fearmongering. That vacuum lets companies shape the narrative unchallenged by the executive branch, even as they position themselves as the responsible parties willing to accept guardrails.

The real question is not whether AI companies will accept regulation, but whose interests it will actually serve. Independent testing, if it materialises, must be genuinely independent: funded transparently, conducted by bodies with no financial entanglement with the labs, and empowered to impose consequences. Until that infrastructure exists, the public is asked to trust the very entities whose products are the subject of concern. That circularity should trouble anyone thinking carefully about governance, evidence, and accountability in AI.

← Back to Blog