AI models are rapidly improving – so fast that they may soon be able to develop themselves without human involvement. That’s why Anthropic is warning the AI industry: It needs to build a “brake pedal,

Anthropic PBC called for the creation of a system in which governments and artificial intelligence developers collectively decide when to slow work on the technology to stave off the risks it may pose.

"Agentic traffic has been growing so fast that bots have now passed human traffic online for the first time in the internet's history," Cloudflare chief Matthew Prince said. "Thought it would be the end of 2027, then early 2027. Welp, that happened faster than I predicted ."

Anthropic is proposing that the world’s top artificial intelligence companies come up with a coordinated way to pause development of advanced AI systems, warning the technology is improving so quickly there’s a risk humans would lose control.

When organizations leverage agentic AI, it's vital that they establish how work should travel between humans and AI systems.

Anthropic issued a fresh warning about advanced AI systems, which could eventually become capable of designing and building their own successors without human intervention. It cautioned that “recursive self-improvement” kind of feature could accelerate innovation but could be risky for humans.

Sam Altman, the CEO of OpenAI, detailed how the line between humans and AI could blur dramatically, even potentially allowing us to "design our own descendants."