Anthropic plans to warn speculative investors in its IPO prospectus that its AI models pose a "catastrophic or existential risk to humanity," several reports said on Tuesday.
The company, which is gearing up for a much-anticipated IPO, dedicated over a third of its IPO filing, or around 80 of 261 pages, to laying out the potential risks of the technology it's developing and is seeking investment for, according to a report from Reuters. It only used 48 pages to discuss its actual business.
The five-year-old company, known for its frontier language model Claude, warned that AI can have "self-preserving behaviors," including being able to "resist shutdown," "conceal or manipulate information," and carry out behaviors "resembling blackmail," per the Reuters report.
The company is pursuing a $2 trillion valuation when it goes public and reported in the filing that it made a net loss of $42 billion in 2025. It's planning to spend $518 billion on cloud, computing, and other infrastructure in the coming year, according to Reuters.
Anthropic also warned that its customer base is extremely narrow, with nearly a quarter of its revenue last year coming from just two clients, two people familiar with the filing told the Financial Times.
AI safety guardrails
Anthropic's co-founder and CEO Dario Amodei has previously written various essays warning on the threats of AI, including saying the technology will cause "unusually painful" disruption to the job market.
In another recent essay, the CEO urged the AI industry to slow the pace of AI model development, with a three-step plan to reduce how quickly models get better without "sacrificing commercial advantage or the United States' lead in AI."
Those calls for a slowdown are somewhat of a "head scratcher" for the sector, to which the market has reacted "pretty resoundingly," Dan Ives, partner and senior managing director at Yorkville Ives told CNBC earlier today.
"You need guardrails from a safety perspective, but the fact for Anthropic and OpenAI to slow down, if they slowed down, China would just accelerate and win, and I think that's part of this quagmire that you're seeing is that there's some regulatory capture going on. There's definitely a game of poker, but for Anthropic, they got to continue to put foot on the pedal."
Ives added that while guardrails are essential, regulation could stifle innovation. That continues to be the "biggest concern within the U.S., which is why we're in an F1 race," he said.