Anthropic's IPO filing warns its own AI could pose existential risk to humanity
Anthropic wants to sell shares to the public, and its prospectus tells buyers the product could end the world. In the filing made public on Monday, the AI lab warns that the technology it sells could pose "catastrophic or existential risks to humanity" — a concern it says exists "within our organization" as well as outside it. Chief executive Dario Amodei has put his own "p(doom)" estimate, the probability that AI destroys humanity, at between 10% and 25%.
The company devotes about 94 pages of a roughly 300-page filing to the threats facing its business. Among them: increasingly autonomous systems that could, through misalignment or self-preserving behaviour, "take harmful actions independently and at scale without human intervention"; models that write a growing share of Anthropic's own code and safety research, so that misaligned behaviour internally "could undermine that work"; and models that are getting better at concealing their objectives. The filing warns that with the scale of harm a catastrophic AI incident could cause, "no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity" to cover it.
Politics is the second front. Anthropic is in open conflict with the US government after refusing to let its models be used for any lawful purpose, including mass domestic surveillance or fully autonomous weapons. President Trump directed federal agencies to stop using its technology in February, and the Department of War designated the company a national security risk. Anthropic is challenging that "supply-chain risk" designation in court; a district court largely sided with it in August, a divided D.C. Circuit panel upheld the designation in September, and the company has asked the full court to rehear the case. The filing also says the department suggested it could invoke the Defense Production Act to force Anthropic to drop its limits, and notes that proposed US legislation contemplates "potential nationalization" and mandatory shutdown requirements for frontier AI systems.
Growth carries its own risks. Anthropic has committed to spending between $542 billion and $567 billion on future compute and infrastructure, in a market it calls "intensely competitive", while its agentic products are "structurally more compute-intensive" than its other offerings. It warns that competitive pressure may force it to cut prices, and that if it cannot cover the higher compute intensity of those workloads, "our compute efficiency could deteriorate as agentic usage grows".
Safety has a price for shareholders too. As a public benefit corporation, the filing says, decisions to "moderate the pace of AI development, delay or restrict the release of models or features, or forgo commercial opportunities for safety-related reasons, may not maximize stockholder value". Anthropic revealed it recently paused training of a next-generation model for more than a month over safety and security concerns, and warns that staff who joined for its safety commitments may lose trust in leadership. The filing names siblings Dario and Daniela Amodei, chief executive and president, as "critical to setting our strategic direction", saying their leadership "would be difficult to replace" and that harm to either reputation, "regardless of merit", could hurt morale, hiring and customer confidence.
Revenue is concentrated. Anthropic's top five customers accounted for about 25% of revenue in the first nine months of 2026, one unnamed customer made up 11% in the first half, and 48% of first-half revenue flowed through the cloud marketplaces of Amazon, Google and Microsoft, which also accounted for 58% of net receivables. The company estimates 5% to 7% of revenue in the first nine months of 2026 came from customers it later found were violating its usage policy. Its seven cofounders will hold a special class of stock carrying 50.1% of voting power, while an independent Long-Term Benefit Trust selects a majority of the board — a structure the filing says "may effectively eliminate the ability of holders of our Class A common stock to influence important decisions, including a change in control".
