In a blog post, Anthropic says there are signs that "the role of humans is diminishing with each step in the development process of AI."
Anthropic is one of the largest AI companies and is behind the chatbot Claude, among other products. But the company says it is now time to slow the pursuit of more advanced AI.
"We believe it would be good for the world to have the ability to slow down or temporarily suspend the development of groundbreaking AI, so that societal structures and research on ethical adaptation have time to keep pace with technological advances," the company writes.
The company adds that such a decision must be made jointly by the entire industry so that countries and companies do not have to choose between security and falling behind in development.
“Lots of nonsense”
Marcus Liwicki, professor of machine learning at Luleå University of Technology, does not see any such break coming.
"I don't think it's possible to pause development. Instead, we need to invest more in people so they learn to control the technology," he says.
How big is the risk that AI will start to develop itself?
"What we've seen so far when we've let go of control is that AI systems just produce a lot of nonsense. The risk is that people don't understand it's nonsense; then it becomes dangerous," he says.
Similar warning
The alarm from Anthropic has encountered criticism from both other companies and the US government, which believes that the San Francisco company is being too alarmist.
In response, Anthropic says it is planning a meeting of companies, researchers and government officials to discuss security. The company warns that AI is already accelerating its own development and that this could eventually lead to systems learning to become smarter on their own - without human help.
"We are not there yet, and it can be avoided."
In March 2023, a similar warning came from a number of AI researchers and investors, including Elon Musk. They called for the development of AI to be paused for six months for reasons similar to those now stated by Anthropic.
