Full Breakdown
AI Leaders Call for Regulation While Positioning Their Own Safety Frameworks
By Drooid · · How we work
Core Event
In recent weeks, chief executives of OpenAI, Anthropic and other leading labs have publicly declared that their most advanced models pose “existential risk” and should be subject to independent testing before release. Simultaneously, they are promoting self-crafted auditing regimes and selecting preferred evaluators, a strategy analysts say is designed to shape future regulatory standards ahead of U.S. midterm elections and upcoming public listings.
Background & Political Context
The United Nations’ Security Council recently hosted a discussion on artificial-intelligence safety, where OpenAI CEO Sam Altman and other tech CEOs voiced alarm. President Trump has dismissed AI-risk warnings as a “hoax” and has tied his administration’s economic agenda to Silicon Valley successes. The Biden-era U.S. Center for AI Standards and Innovation (CAISI), created in 2023, continues to receive voluntary model submissions, but industry leaders prefer private evaluation firms such as the Berkeley-based nonprofit METR and the AI Evaluator Forum.
Industry Motives and Market Implications
Analysts argue the safety narrative helps AI firms court investors before anticipated IPOs and shields them from smaller competitors. PitchBook senior analyst Harrison Rolfes describes the approach as building a “wall or a moat” that will generate substantial profits for the dominant players. Chipmaker Nvidia’s CEO Jensen Huang, speaking with President Trump, warned that alarmism could be excessive, suggesting a balance between caution and commercial momentum.
Official Statements & Responses
OpenAI spokesperson Liz Bourgeois said the company has paused training its most advanced models and emphasized that “people want to know AI is being developed safely, and that starts with what companies like ours do ourselves.” An Anthropic spokesperson reiterated the firm’s long-standing call for regulation. Former CAISI head Conrad Stosz, now with the AI Evaluator Forum, noted that many evaluators seek deeper lab access but that the parameters for “embedded evaluators” remain ambiguous.
Verbatim Quotes
- “People want to know AI is being developed safely, and that starts with what companies like ours do ourselves,” — Liz Bourgeois
- “Once again we’re talking about existential risk, while deprioritizing a number of other safety-critical risks that exist today. If you look at the use of AI in military tech, you can see that these systems are already used to kill people,” — Sarah Shoker
- “Lots of evaluators are interested in embedding with labs and getting greater access, but it’s a little ambiguous what embedded evaluators means,” — Conrad Stosz
- “They’re creating a wall or a moat within this sector. … It’s genius and they’re all going to make a lot of money,” — Rolfes
- “All of this talk is actually a way to sort of dissipate and redirect this political will that has built up, rather than actually channeling that political will to do something good,” — Daniel Kokotajlo