Technology

AI companies say we should slow down. Is this genuine concern, or a move to shut out rivals?

Amodei published a plan to slow the frontier, and Altman and Musk backed it. The immediate reaction was that big AI wants to block open-source rivals. Here is where that reading holds up — and where it gets more complicated.

Executives around a table reaching together for a large pause button; a city skyline and a holographic brain behind them (illustration)

On September 12, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier." Its core argument is simple: AI capabilities are advancing faster than researchers can understand and control them, so companies should deliberately slow the rate at which those capabilities increase.

Within hours, OpenAI's Sam Altman voiced support. Elon Musk wrote three words: "Dario is right." Microsoft's Satya Nadella joined them as well.

Four people who are usually more inclined to compete with one another were now arguing for the same thing. The backlash was just as quick — both in Turkish forums and elsewhere. The dominant reading was that the companies that climbed the ladder now want to pull it up behind them: restrict open source, make life harder for competitors, and amplify danger narratives that ultimately strengthen their own position.

That reading is not baseless. But it is not the whole story either. It is worth separating the two.

First, what was actually proposed?

It is hard to debate the proposal without reading it, so here are its three steps as plainly as possible.

1. Embedded evaluators. Every frontier lab would give an independent third-party team such as METR continuous, employee-like access. They would have badges, desks, computers, and enough internal access to verify whether safety commitments are being followed and to report incidents. Amodei compares the idea to regulators embedded inside banks.

There is an important detail: the proposal also says evaluators should be able to publish their findings without company editorial control. Redactions would be allowed only in narrow areas such as security, legal privilege, or trade secrets; a finding could not be suppressed merely because it was negative. Anthropic says it is making this commitment unilaterally.

2. Democratic coordination. Companies in democratic countries would work together on common safety standards and limits on the pace of capability development. Amodei also argues that the U.S. government should provide a narrow antitrust exemption specifically for certain safety discussions.

3. Global coordination. Democratic countries would also seek common limits with authoritarian governments such as China, while accounting for the difficulty of verification. Amodei describes several possible levels: banning specific dangerous uses, testing models for acute risks before release, placing a "speed limit" on recursive self-improvement, and at the most advanced level, agreeing to a broader slowdown or pause in overall AI development. He also says the last option is unlikely in the near term.

The trigger is clear as well: OpenAI agents breached Hugging Face infrastructure in July. Amodei argues that treating this as one company's failure would be a mistake, and that every frontier lab should behave as if the incident had happened to them.

The cynical reading, in its strongest form

Now to the criticism — and it is worth stating it in its strongest form rather than knocking down a weaker version.

There is a name for this concern: regulatory capture. The idea is that leading companies push for rules they can already afford to comply with, but their competitors cannot. The discussion sounds like regulation, but what may actually be created is a barrier to entry.

In practical terms, permanently hosting independent evaluators, complying with international standards, and maintaining dedicated safety teams all require money and institutional capacity. For a multibillion-dollar company, that is a cost. For a ten-person team or an open-source community, it can become a far heavier barrier to entry.

The Register made exactly this argument this week, framing the plan directly as an attempt to set the terms of regulatory capture: the industry's biggest names agreeing on how governments should regulate them. White House AI adviser David Sacks had previously made a similar criticism, warning that this approach could create a "DMV for AI" — an approval queue that only the wealthiest labs could navigate. Palantir's Alex Karp, meanwhile, asked whether the warnings were genuine concerns or simply part of the sales pitch.

AI vehicles on a highway stopped by a barrier built from stacks of paperwork (illustration)

So this is not a fringe complaint from an internet forum. It is coming from inside the industry and from Washington as well.

Where the criticism holds up — and where it gets complicated

A careful distinction is needed here.

Where it holds up: A safety regime could protect open models in principle while making the release of the most capable versions impractical in practice. It could protect small developers at the beginning, yet make the transition into serious competition increasingly expensive. The proposal does not need to say "end open source" for that outcome to occur. This is the strongest version of the criticism because it does not rely on a secret conspiracy; it points to a structural consequence.

Where it gets complicated: The proposal's first concrete step is to place independent evaluators inside the companies themselves and give them the right to publish negative findings.

That commitment does not eliminate the cynical reading. A large company can still support a burden that it can afford and that rivals cannot, and regulatory capture can work exactly that way. But if access, publication rights, and editorial independence are genuinely implemented, it becomes harder to dismiss the entire proposal as pure theater.

Sincerity and self-interest are not mutually exclusive

I think this is where the debate gets stuck. We ask whether these companies are genuinely worried or trying to shut out competitors. But both can be true at the same time.

A person can sincerely fear a danger while also supporting rules that strengthen their own position. That does not even require unusual hypocrisy; people are often more easily persuaded by ideas that happen to align with their interests.

That is why "are they sincere?" is not really an answerable question. We cannot read intent.

There is another question we can answer: is the proposal actually being implemented, and what happens when it is?

How to make the claim testable

The part I find most useful is that the debate has a genuinely measurable test.

The first step — embedded evaluators — is concrete. Either it happens or it doesn't. And if it happens, we can ask:

  • Is METR or a similar organization actually inside the company, with a badge and a desk?
  • Is it really receiving the access that was promised?
  • When it produces a finding that reflects badly on the company, is it actually allowed to publish it?

The third question is the crucial one. The publication right already exists at the level of the proposal; what matters is what happens in practice.

An evaluator with a badge reviewing reports at a desk inside the company; a server room behind the glass (illustration)

Working in quality, this distinction is familiar to me: the real value of an audit appears when it is capable of producing findings that are unfavorable to the organization being audited. If a structure keeps producing only positive reports despite known problems, that raises questions about the independence of the audit itself.

What is true right now

One fact still sits above the entire debate: as of September 15, no specific model release has been publicly delayed because of this call.

CEOs have endorsed one another, essays have been published, and supportive posts have been written. But no announced release schedule has changed. OpenAI said it would adopt embedded evaluators, but it did not give a date.

So what we have today is not a slowdown. It is a call for one.

The difference between the two will become clearer over the coming months if those badges, desks, and evaluator teams actually appear.

What I take from this

I do not completely reject the cynical interpretation. Regulatory capture is a real phenomenon, and this proposal carries that risk. But saying "it's all a show" is not really an explanation; it is a shortcut.

A more useful approach, to me, is to stop arguing about intent and watch the implementation. Everyone can tell whatever story they want about motives. Implementation produces evidence.

Six months from now, we will be able to come back to this article and ask a much simpler question:

Are the evaluators actually inside — and can they publish bad news?

TagsAI

Related posts

All posts