Anthropic CEO Amodei calls to 'slow the pace' of AI model improvements

Editor's note: This article was updated at 1150a ET with from Elon Musk. It was updated again at 4p with from OpenAI CEO Sam Altman.

In the face of potential risks, Anthropic ANTHROPIC CEO Dario Amodei said that the progress of improving AI model capabilities should be slowed down.

"But over the last few months, I have become convinced that fully addressing the risks requires even more prudence—not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up," he wrote in an essay on his personal website on Saturday.

Amodei expressed two concerns. The first is rapid advancements. "This dynamic is called recursive self-improvement….Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all."

His other concern had to do with OpenAI's OPENAI July Hugging Face incident . "Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails."

Amodei also introduced a plan with the aim of "pacing the frontier" with three tenets: embedded evaluators; democratic coordination; and global coordination.

In the first step, AI companies would allow third-party evaluators employee-like access "to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes." He added that Anthropic is now committing to this.

In step two, AI firms would "establish common safety standards as well as limits on the rate of unchecked AI progress." In the final step, "The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance."

OpenAI CEO Sam Altman responded to Amodei's essay on X. "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to soon."

In his own X post, Elon Musk, founder of SpaceXAI (previously xAI), wrote, "Dario is right." Disclaimer Content provided by Seeking Alpha is intended for information purposes only, and that Seeking Alpha does not offer any personalist investment advice and is not a licensed securities dealer, broker, US investment adviser or investment bank.

More news from Seeking Alpha