Anthropic boss issues warning about ‘unimaginable’ power of AI

< warm>The CEO of leading AI keep company anthropical, Dario Amodei, has issued an ominous public warning that humanity is on the leaflet of being handed “almost unimaginable power,” for which it is dangerously unprepared.

In a nearly 20,000-word essay titled “The Adolescence of Technology,” Amodei sketches a near-future where AI systems vastly more capable than any Nobel laureate or statesman could be at everyone’s disposal within the next few years. A critical and accelerating factor, Amodei reveals, is that AI development is now creating a self-reinforcing feedback loop.

“Because AI is now writing much of the code at Anthropic, it is already substantially accelerating our progress in building the next generation of AI systems,” he writes, warning that the company is close to “a point where the current generation of AI autonomously builds the next.”

He argues that without decisive and careful action, this technology could lead to catastrophic risks ranging from mass job displacement to human extinction. Other existential dangers include the potential for “a global totalitarian dictatorship” enabled by AI-powered surveillance, propaganda and autonomous weapons.

Amodei also details “autonomy risks,” where AI systems could “go rogue and overpower humanity” – noting that this danger would not even require a sci-fi army of physical robots. The essay chillingly observes that “plenty of human action is already performed on behalf of people whom the actor has not physically met.”

Among the most urgent threats, Amodei highlights the potential for AI to drastically lower the barrier to creating biological and other weapons of mass destruction.

“A disturbed loner can perpetrate a school shooting, but probably can’t build a nuclear weapon or release a plague,” he writes. A powerful AI, however, would make “everyone a PhD virologist who can be walked through the process of designing, synthesizing, and releasing a biological weapon step-by-step.”

In a worst-case scenario, he warns a powerful AI could theoretically guide the creation of a synthetic pathogen capable of “destroying all life on Earth.”

As one of the key industry leaders, whose company is a chief rival to OpenAI, Amodei calls for “surgical” regulation, starting with transparency laws, to build necessary guardrails.

“Humanity needs to wake up,” Amodei concludes, framing the coming years as a critical test of civilization’s maturity. With the technology itself now fueling its own breakneck evolution, he urges a collective response to steer the “glittering prize” of AI away from potential ruin.

---

---