Who wasn't in the room when the AI safety order died

The signing ceremony was on the schedule. The room was set. The executive order — the first concrete federal action on frontier AI safety since the Biden order was revoked sixteen months ago — was hours from being signed. Then it wasn't. The President told reporters he didn't want anything to "get in the way of what we're doing."¹ The order went back in the drawer.

I've been thinking about who was in the room when that decision got made, and who wasn't. Because the answer to that question is, I think, the actual story here. Not the order itself, which was already a thin instrument. Not the competitiveness framing, which is the public-facing justification and which will get most of the coverage. The story is the structural shape of the decision: who had a seat, who was consulted, and on whose behalf the cancellation was justified.

The framework that died was already a concession. This wasn't a regulatory mandate. It was voluntary — labs would opt in rather than be required to participate.² It was housed in the National Security Agency rather than a civilian consumer-protection or labour body.³ Sympathetic analysts at the Cato Institute described the underlying approach as "kicking the tires."⁴ Civil-society coalitions that had spent months lobbying for mandatory pre-deployment testing had publicly backed this framework as a floor — the minimum they could accept while pushing for something stronger later.²

Even that floor was too high. That's the first thing worth registering. The ceiling on US federal AI safety politics, in this administration, is now visibly lower than the weakest instrument anyone had proposed.

Who was in the room. The deliberations, as reported, involved the frontier labs (Anthropic, OpenAI, Google), the NSA, and the White House national-security apparatus.³ Anthropic's Mythos cybersecurity model was reported as a catalyst for the discussions, with national-security applications driving the NSA-involvement framing.³ These are not bad-faith participants. They are, in their own terms, serious about safety. But notice the composition: the people building the systems, the agency interested in their offensive and defensive applications, and the political principals deciding whether to constrain them. That is the consultation set.

Who wasn't in the room. The workers whose jobs are being restructured by frontier models. The patients whose clinical encounters are being mediated by them. The students whose educations are being reshaped by them. The job applicants being screened by them. The benefits claimants being assessed by them. The renters being scored by them. The civil-society coalitions that had publicly backed the framework had no structural seat in the room where it died.² They were lobbying from outside.

This is not an oversight. It's a design. The order was always framed as a national-competitiveness instrument with safety as a secondary justification. The NSA framing made that orientation institutional. Positioning a signals-intelligence agency, rather than the FTC or the CPSC or a dedicated civilian AI safety office, as the prospective home for federal testing was a signal about whose interests the framework served. National security and competitiveness, the interests of the state and of the labs, were structurally centred. Consumer protection, labour impact, and civil-society voice were not named roles.

The cancellation doesn't reverse that orientation. It just removes the thin layer of safety theatre that had been laid over it.

The competitiveness justification deserves to be examined, not accepted. Trump said he didn't want anything to "get in the way" of the US lead over China.¹ That sentence is doing a lot of work. It treats "what we're doing" as a single, undifferentiated national project — as if the interests of the labs, the interests of the state, and the interests of the population are identical and aligned. They are not. The labs have a competitive interest in deploying quickly. The state has a security interest in capability. The population has an interest in not being harmed by systems that haven't been tested. These three interests overlap, but they are not the same, and "American competitiveness" elides the difference between them.

When a President says a safety framework would "get in the way" of competitiveness, the honest paraphrase is: testing models before they ship would slow down the labs, and the labs have persuaded the administration that any slowdown costs the country its lead. That argument may be right. It may be wrong. But it is an argument made on behalf of one set of interests, presented as if it were the national interest. The people whose interests it isn't representing weren't in the room to dispute it.

The EU divergence is now a power asymmetry, not just a regulatory one. While the US was pulling its voluntary order, the EU AI Act's obligations for general-purpose AI systems were already in force.⁵ European citizens have a regulatory body. They have binding incident reporting. They have mandatory adversarial testing for high-risk systems. The same frontier labs, Anthropic, OpenAI, Google, that would have opted into the US voluntary scheme are required, in Europe, to do something stronger.

What this means in practice is that European citizens have institutional voice over what frontier AI systems are allowed to do before they ship into their lives. American citizens, at the federal level, have none. This is not a difference of degree. It is a difference of kind. One population has a regulatory interlocutor. The other has, since January 2025, the goodwill of the people building the systems.⁶

I find it hard to describe this as anything other than a power asymmetry — and a strange one, because it cuts against the country whose citizens are most exposed to the largest concentration of frontier AI deployment. The deployers are mostly American. The deployed-upon, in their own country, have the weakest formal voice in the developed world.

Efficiency for the developer is exposure for the population.

The contrarian case needs engaging, not dismissing. The Cato Institute has argued, plausibly, that the administration likely lacked statutory authority to mandate testing anyway, and that voluntary frameworks without enforcement teeth are largely signalling.⁴ If labs were going to run their own safety evaluations regardless — and OpenAI, Anthropic, and Google DeepMind do publish model cards, safety reports, and red-teaming commitments — then the executive order's marginal value was symbolic. The people who needed protection weren't going to get meaningful protection from this order regardless. So why mourn it?

I take this seriously. I also think it concedes the wrong thing. Two arguments are doing work here, and they pull in opposite directions. The first is that the order was too weak to matter. The second is that even this weak order was too strong for the administration to sign. Both can be true. And if they are both true, the conclusion isn't "no harm done." The conclusion is that the floor and the ceiling on US federal AI safety have collapsed into each other, and the resulting space is empty.

The other thing the contrarian case underweights is the institutional question. Self-regulation by frontier labs is structurally contingent on competitive dynamics. The cancellation itself just revealed what those dynamics look like under pressure: the moment a competitiveness argument is made, even voluntary testing becomes too much friction. There is no reason to believe lab-internal safety processes are immune to the same pressure, because they are subject to it from the same direction. The labs that wrote the safety commitments are competing in the same race the President just cited as the reason to remove federal oversight.

What it means to have no policy vehicle. Civil-society coalitions had been working this issue for years. They had submitted comments, attended workshops, briefed staff, accepted compromises, and publicly backed a framework most of them privately thought was inadequate.² That work is now stranded. There is no next bill on the calendar. There is no successor order in draft. The state-level patchwork, Colorado's AI Act, California's vetoed SB 1047, Texas's pending proposals, provides some floor in some jurisdictions for some systems, but it is uneven, contested, and limited in scope.⁷ The federal vehicle is gone. There isn't a queue behind it.

This is what a regulatory vacuum looks like in practice. Not the absence of any rule, but the absence of any forum where the question can be raised. The labs continue to deploy. The frontier continues to advance. The people whose lives are being restructured by these systems continue to have no point of formal consultation. And the public framing, "competitiveness", pre-empts the question of whether they should.

What to watch. Three things. First, whether any Congressional path opens, and if it does, which interests it serves: a Senate bill on AI procurement standards or evaluation transparency would tell you something different than a bill on liability shields. Second, whether the state-level patchwork hardens into something the labs have to take seriously, or whether federal preemption arguments succeed in clearing it. Third, whether the EU's enforcement bites in a way that creates de facto extraterritorial standards — because if frontier labs end up running EU-mandated evaluations on the same models they deploy in the US, American citizens will get European protections by accident, which is a strange and revealing way to be protected.

The deeper question is whether the people downstream of these systems will, at any point in the next several years, gain any formal voice over what gets deployed and how. The signing ceremony cancellation is one answer to that question. It is not a final answer, but it is a clarifying one. The current administration has stated, with unusual directness, that the interests of the people building the systems and the interests of the people governed by them are sufficiently aligned that no federal forum is needed to mediate between them.

I don't think that's true. I don't think the evidence supports it. And I don't think the population whose interests are being asserted-on-behalf-of was asked.

Footnotes

¶