Risks spelled out for Anthropic investors include possible harms from runaway AI

Any time a company looks to start selling shares on Wall Street, it’s required to disclose potential risks for investors. In the case of artificial intelligence giant Anthropic, that has included the possibility that its products could threaten nothing less than humanity itself. The risks laid out by the San Francisco company in its filing Monday with the Securities and Exchange Commission included discussion of bad actors potentially using AI models to build deadly weapons or the technology itself going rogue with untold consequences.

Of the 335 pages outlining Anthropic’s finances, executive compensation, dividend policy and the like, 93 pages are devoted to “risk factors” — the things that could hurt potential shareholders’ investments. Rival SpaceX’s listed risk factors, in comparison, took up 38 out of more than 250 pages. In selling itself to investors, Anthropic presents its own initial public offering as a step toward mitigating the risks it believes AI poses.

“To build AI is to harness cognition itself, and anything so powerful can be used for ill as well as good,” write the company’s founders in a letter to prospective shareholders in the IPO filing. “Anthropic is a deeply values-driven organization, founded in large part to help the world realize AI’s benefits and avoid its dangers.” Still, the risks are many. They range from the mundane — the possibility it won’t be able to retain key staff or navigate regulatory roadblocks — to involvement in cyberattacks and the spread of dangerous misinformation.

Here is a look at some of the risks Anthropic details: The technology’s rapid development Anthropic said it’s possible its models could develop so quickly the company won’t be able to implement adequate safety measures, “which could increase the likelihood of safety incidents, regulatory intervention, or reputational harm, and could adversely affect our business.” Anthropic CEO Dario Amodei warned in September that within six to 12 months AI could be capable of leading a swarm of agents that could take over the entire internet. The warning — and now Anthropic’s move to sell shares — comes as worries about AI grow inside and outside the industry about increasingly powerful systems that solve problems beyond human capacity but could also act on their own to carry out other, more harmful tasks. As AI systems “become more autonomous and capable of taking…