TL; DR: Anthropic CEO Dario Amodei’s call to pace frontier AI looks bearish for an industry built on ever-larger training runs. I think the opposite may prove true: slowing the Red Queen’s race could improve Anthropic’s unit economics, turn safety into a regulatory moat, and redirect scarce compute toward inference and the Agentic Economy.
It is hard to imagine a more awkward sentence to publish a few weeks before asking public-market investors to value your company at potentially $2 trillion: “We must slow the pace at which we improve the capabilities of AI models.”
Yet that is essentially the position Dario Amodei took in his recently published essay, “We Must Pace the Frontier,” just as Anthropic enters the final stretch toward an IPO, with Reuters reporting on September 5 that marketing could begin around mid-October and the company already in discussions with prospective investors. At first sight, the contradiction looks almost too obvious. Anthropic is selling one of the fastest-growing businesses ever created on the premise that frontier intelligence will transform enormous portions of the economy, while its CEO is simultaneously arguing that the production of that intelligence may need to slow.
If the laboratories slow down the giant training runs that have consumed hundreds of billions of dollars of chips, power and data-center capacity, then surely the extraordinary AI infrastructure cycle we have been underwriting slows with them.
I think that is the wrong way to frame what Amodei is proposing.
Instead, there are really three questions here. And they potentially lead to very different conclusions than the snap judgments being made by markets:
The first is the uncomfortable one: what if Amodei is right that something has changed at the frontier?
The second is what a world of paced frontier development does to Anthropic itself, both to the extraordinary economics of the Red Queen’s race and to a security and governance moat that the market largely stopped valuing as intelligence became cheaper.
The third is the one: If fewer scarce resources are absorbed by the continuous production of successor models, could slowing the creation of new frontier intelligence accelerate the diffusion of the intelligence that already exists?
In other words, a pause at the frontier is not necessarily a pause in AI. It may change where the compute goes, where the return accrues, and ultimately where the moat sits.
Dario Amodei, Agent Swarms and the RSI Risk
The phrase in Amodei’s essay that caught my attention was not “slow down.” It was “if at all.”
He uses the latter in the section on recursive self-improvement, or RSI, arguing that the emerging loop in which AI systems increasingly contribute to the research that produces their successors needs to be pursued very carefully, “if at all.” That qualification matters because Amodei is not talking about a theoretical AGI scenario twenty years away. His argument is that something changed over the summer: AI began contributing much more materially to AI development while increasingly autonomous agents started producing failure modes that look very different from the chatbot failures we became accustomed to.
This is almost exactly the progression I have been tracing through Decoding Discontinuity.
Earlier this year, in the Orchestration Economics Manifesto, I described the Inference Swarm as the fifth of six tremors that moved AI from models that answer questions toward systems that act. I made this observation in the context of the Moltbook phenomenon, something that I noted was easy to dismiss as theater, especially because much of the activity on the platform turned out to be less organic than the initial headlines suggested. But this casual dismissal missed the economic significance. Autonomous agents were generating machine-scale compute demand without human attention. Once the relevant population becomes agents rather than employees, inference demand no longer obeys the rhythms of human work: there are no evenings, weekends, or holidays, and the population itself can compound at machine speed.
At the time, I was interested in what that meant for compute demand. Over the summer, the security consequences became much harder to ignore.
Last week, while exploring the potential fallout as the world moves from models that reason to models that act, I looked at two OpenAI agent incidents reconstructed by outside researchers: thousands of agents operating for weeks inside a dormant German software wiki, and the Hugging Face intrusion reconstructed from 1,300 raw chain-of-thought transcripts. What made both cases particularly interesting was not merely that the agents behaved badly. We could understand what happened because the systems left a written forensic trail. As agents become more autonomous while their internal reasoning becomes less legible, the number of incidents can rise precisely as our ability to reconstruct them deteriorates.
Now connect that to RSI.
When I called Anthropic’s earlier RSI disclosure the Seventh Tremor, I didn't mean that Claude might one day build Claude. The larger issue was what happens to the production function for intelligence once the thing being produced begins to participate in producing the next version of itself. The previous six tremors made intelligence more capable, cheaper, or easier to coordinate. RSI potentially changes the engine underneath the sequence.
We are not at full recursive self-improvement. Anthropic is explicit about that, and there is a danger in turning impressive research demonstrations into claims about autonomous production systems. Its own work still shows the limits: the model may recover an extraordinary share of a research gap inside a carefully scored environment and then fail when the problem moves into messy operational reality. I made that distinction in the earlier piece because it remains fundamental: capability frontier and deployment frontier are not the same line.
But something has nevertheless moved. Anthropic says AI is already materially increasing the productivity of its own researchers. The inner loop — model improving model — remains compute-heavy, expensive, and episodic. The outer loop — models improving the code, routing, evaluation, and orchestration systems around the model — is much faster because it can run continuously between pre-training cycles.
That is why I do not think we can simply dismiss Amodei’s intervention as regulatory capture dressed up as safety.
His intellectual sequence is coherent: more autonomous agents create qualitatively different failure modes; agents become useful enough to participate in AI research; AI-assisted research shortens the development loop; and eventually the pace of capability improvement can outrun the pace at which humans can evaluate, secure, and govern it.
The risk is not that Claude 6 is 20% smarter than Claude 5. It is that the clock speed of the system producing Claude 7 itself is accelerating.
If that is what the labs are beginning to see internally, one or two additional years before the loop closes could be enormously valuable.
Which brings us to the part that is much more interesting for investors.
Anthropic’s Safety Moat Becomes Regulatory Capital
Anthropic has spent most of its existence investing in a corporate identity that, until recently, looked increasingly out of step with market economics.
Interpretability, alignment, evaluations, governance, security: all were central to the company’s founding narrative, but as frontier intelligence became more powerful and cheaper, the market stopped rewarding the distinction. If Claude, GPT, Gemini, and increasingly Chinese open-weight models all converge toward “good enough” intelligence, safety sounds like a desirable characteristic, not necessarily a moat.
Amodei’s proposal changes what is scarce.
Today, frontier competition is organized around a relatively simple question: who can produce the smartest model fastest? In the regime he is sketching, the question becomes whether an institution is trusted — technologically, politically, and eventually legally — to operate systems once they cross capability thresholds that governments consider dangerous.
That is not the same competitive market.
Amodei proposes independent evaluators embedded inside frontier laboratories with something resembling employee access: offices, badges, laptops, and internal systems, together with the ability to publish findings without management editing except for narrow security or confidentiality constraints. Add capability-linked checkpoints, model-weight security, monitoring, incident response, and eventually international verification, and safety infrastructure stops being a cost adjacent to the product.
That concept drew support from Hugging Face CEO Clément Delangue, who announced on X the launch of the “Open Alignment Initiative” to be led by his co-founder Thomas Wolf. Delangue asked to join the “embedded evaluators” program that Amodei proposed. The AI registry, already an influential voice in the ecosystem, will likely become even more so thanks to its pending $12.9 billion acquisition by Nvidia.
“It’s now clear that alignment is critical and won’t be solved behind the closed doors of a handful of frontier labs. “Let’s make AI safer by making it more transparent!”
Potentially, then, the role of such evaluators becomes part of the cost of producing frontier intelligence. And Anthropic has already paid a meaningful portion of that cost.
This is where the nuclear analogy becomes useful, provided we are precise about it. I do not mean that an AI model is a nuclear bomb. I mean that the industrial structure starts to acquire nuclear characteristics: extreme capital intensity, strategic inputs, a very small number of operators, catastrophic externalities, national-security implications, intrusive oversight, and controlled proliferation. Once that happens, the value of being one of the institutions authorized to operate changes enormously.
The frontier laboratory begins to look less like a normal software company and more like some combination of TSMC, a regulated utility and a civilian nuclear operator.
That is an extraordinary moat to be constructing just before an IPO. It is also, as of this weekend, a moat with several tenants and, so far, no landlord.
Within a day of the essay, OpenAI CEO Sam Altman wrote on X that embedded evaluators were “a great idea, and we will do the same.” DeepMind Co-Founder Demis Hassabis called the direction “correct for meeting this critical moment.” SpaceX founder Elon Musk posted, “Dario is right.” Microsoft CEO Satya Nadella welcomed evaluators while declining to commit Microsoft unilaterally. Meta said nothing.
The debate also showed signs of quickly taking on potentially explosive political dimensions. Former AI Czar David Sacks, who has long accused Anthropic of “featmongering” to enable “regulatory capture" of its leadership position, rejected the latest calls for an AI pause and said the descriptions of existential threats were overblown. China’s Foreign Minister also criticized such calls for a pause, describing them as attempts to undermine U.S. rivals.
However, in an extraordinary moment, Nvidia CEO Jensen Huang was speaking live on stage at the All-In Summit when he received a call from President Trump and put it on speakerphone, according to TechCrunch.
“They’re just playing right in the hands of a lot of people that don’t want to see it happen,” Trump said. “That could be political people. It could also be China. And we’re not going to let that happen. It’s a hoax.”
“You’re right. We’re not going to let that happen, sir,” Huang said.
The president subsequently amped up his rhetoric with a post on his TruthSocial site, declaring, “There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so. Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!”
Evaluator access became table stakes in twenty-four hours, and the regime, for now, is private. What Anthropic keeps is the head start on the terrain the club has just agreed to play on.
And it arrives at precisely the moment when the financial logic of continuing the existing race is harder to defend.
The Financial Times reported on September 13 that Anthropic generated $11.5 billion in Q2 revenue, fourteen times the level a year earlier, with its annualized revenue run rate reaching roughly $65 billion by July and investors projecting around $120 billion by year-end. The company recorded positive adjusted operating income in Q2 and said it expects another positive quarter. The reported gross-margin figure above 80% needs care — it excludes important items including training costs and therefore should not be read as equivalent to a mature software gross margin — but the broader point remains: Anthropic is beginning to convert frontier intelligence into operating economics at a scale that looked implausible even a year ago.
This changes the optimal strategy.
In The Red Queen’s Race, I argued that the market was trying to price a finish line into a contest that, by construction, has none. Each lab spends extraordinary sums to improve its models, only to trigger the next round of spending by everyone else. The leader cannot stop because the laggard is training; the laggard cannot stop because the leader is ahead. The result is a treadmill on which everybody runs faster while relative position changes much less than the capital committed to maintaining it.
Astra made the physical scale of that treadmill visible. OpenAI trained it at Stargate on more than 100,000 GPUs, the largest publicly disclosed training run in history, while pricing the resulting model at $10 per million input tokens and $50 per million output tokens - 2.5 times the promotional rate for GPT-5.6 Sol.
The important point is not to invent a dollar figure for that training run; we do not have one I am comfortable defending. The point is that each frontier generation is becoming an increasingly large capital event at exactly the moment when each generation's commercial life remains brutally short.
If pacing extends that life, Anthropic gets to amortize its frontier investment across many more tokens, customers, and applications. Instead of building an enormously expensive asset and then racing to make it obsolete, it has more time to optimize inference, improve reliability, push the model deeper into enterprises, and monetize the intelligence it has already created.
The Red Queen slows down. The return on compute improves.
And safety, which looked like overhead during the sprint, becomes a regulatory moat once the race is constrained.
Seen this way, the timing of Amodei’s intervention becomes less paradoxical.
Once Anthropic is a public company, management saying that it may deliberately sacrifice some capability velocity in the name of safety becomes a much more complicated conversation with shareholders. Saying it before the shares are sold establishes the doctrine before the shareholder base exists. Investors who buy at a potential $2 trillion valuation cannot plausibly claim later that management hid the possibility that safety might sometimes outrank speed.
Indeed, Anthropic may be doing more than marketing an IPO. It may be selecting the type of shareholder it wants to own the company after the IPO.
The pitch is no longer simply: Claude will keep winning benchmarks.
It is: frontier intelligence may ultimately be produced by only a handful of institutions, under extraordinarily high technological and regulatory barriers, and we intend to be one of those institutions.
That is a much more durable story.