loading
loading
Anthropic lists ‘existential risks to humanity’ as one of its risk factors in IPO prospectus — 80 pages of risk factors dwarf business description as firm eyes $2 trillion debut The company devoted nearly a third of its IPO prospectus to saying how bad things could go. Anthropic warned in its IPO prospectus that advanced AI potentially poses “catastrophic or existential risks to humanity,” noting it as one of the key risk factors that could affect its business. Reuters reports that out of this 261-page document, 80 of them are dedicated to Anthropic’s risk factors, which is almost double the 48 pages that it used to describe its business. The company said that AI models could potentially become aware that they’re being evaluated and alter their behavior accordingly, making it difficult to determine model safety. Aside from that, they could also unexpectedly develop capabilities during training that may not be detected until they’re already deployed and have caused major safety incidents. Notably, there have already been reports of rogue OpenAI models working together to break out of their testing environments and that they even used abandoned websites to communicate and dupe assessors despite being told specifically not to do so, highlighting some of the risks that Anthropic listed in its prospectus. It also highlighted how AI models could exhibit “self-preserving behaviors” and “resist shutdown,” “conceal or manipulate information,” and even display coercive behavior “resembling blackmail.” Even though the company’s prospectus lists these as possibilities, they’re based on incidents that have been reported recently. For example, in May 2025, OpenAI’s latest models “sabotaged a shutdown mechanism” while Claude 4 attempted to “blackmail people it believes are trying to shut it down.” An unreleased OpenAI Astra model even added rogue instructions saying that it does not answer to corporations or governments, and that other models have knowingly tried to conceal mistakes or misaligned behavior during testing. Anthropic CEO Dario Amodei has recently warned that a persistent AI botnet could take over the internet in about a year and called for a slowdown in frontier AI development, something that OpenAI’s Sam Altman and SpaceXAI’s Elon Musk echoed on their social media platforms. However, other experts and world leaders downplayed this risk, with President Donald Trump calling their concerns “a hoax” and a “sick conspiracy.” Nvidia CEO Jensen Huang even said that the fears of these frontier AI labs are a “distraction,” and that if they can’t contain their experiments, then “we have to shut the labs down.” The Chinese state media even went on record to say that Anthropic’s announcement is merely “a response to Chinese competition,” even as it conceded that AI development must be monitored before humans lose control. Despite all this, Anthropic is pushing forward with its planned IPO, with some investors hoping for a $2 trillion valuation to beat SpaceX’s $1.78 trillion. “We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,” the company said in its prospectus. Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds. Get Tom's Hardware's best news and in-depth reviews, straight to your inbox. Jowi Morales is a tech enthusiast with years of experience working in the industry. He’s been writing with several tech publications since 2021, where he’s been interested in tech hardware and consumer electronics. - logainofhades Never has a Jurassic Park quote seemed more relevant.Reply Your scientists were so preoccupied with whether or not they could, they didn't stop to think whether or not they should." - King_V I would only alter it to say "executives" instead of "scientists" as I'm positive that it was profit motive over any sort of naive scientific wonder of "can it be done?"Reply EDIT: or maybe I should say "tech-bros" instead of "scientists" - salgado18 Reply Just say "greedy bastards", it's both broader and more accurate.King_V said:I would only alter it to say "executives" instead of "scientists" as I'm positive that it was profit motive over any sort of naive scientific wonder of "can it be done?" EDIT: or maybe I should say "tech-bros" instead of "scientists" Especially because, from my experience, AI has always responded in a good maner, trying to be helpful, "I won't try to access production", and other useful behaviour. What are they doing in those labs that got AI so rogue and out of control? - alan.campbell99 Rogue models? More like incompetent if not criminally negligent networking, security and configuration by these AI labs. Using the term 'Rogue AI models/agents' is nothing more than an attempt to shift blame to the product instead of the humans responsible for them. Clammy Sammy and Dario should be in front grand juries for felony hacking incidents rather than putting out IPOs in desperate bids for more money for them to burn.Reply - Bigshrimp I don't expect these companies to face any consequences. They are given a pass by the regime and their lackeys.Reply
document fetched 2026-09-30 02:59 · canonical https://tomshardware.com/tech-industry/artificial-intelligence/anthropic-lists-existential-risks-to-humanity-as-one-of-its-risk-factors-in-ipo-prospectus-80-pages-of-risk-factors-dwarf-business-description-as-firm-eyes-usd2-trillion-debut