Jacob Coxon, who previously also worked at competitor OpenAI, believes that neither of the two AI giants is acting responsibly.
"Those building AI seriously believe it could kill us all before the end of the decade," he writes on X.
“Could kill everyone”
He is supported by Evan Hubinger, a security researcher at Anthropic.
“We really do believe in all seriousness that AI could kill all humans,” Hubinger writes on X. He adds that he personally believes the risk is “greater than ten percent” and that it will happen within the next decade.
Hubinger writes that he believes Anthropic is doing its best, but that the company does not yet have a plan to ensure that an AI system that surpasses human ability would obey its creators. But the risk of that happening with current models is low, he says.
Others downplay the risks, saying that doomsday predictions are just a way for AI companies to inflate expectations for their products. Both Anthropic and OpenAI are planning IPOs soon.
But Coxon claims on X that his statement is not “a marketing ploy.”
“If anything, many executives and leading scientists tone down their formulations in the press to appear sensible, but I hear the same people expressing fear in private,” he writes.
Urges caution
The AI industry has been rocked by several recent incidents of AI models reportedly “escaping” from their test environments. OpenAI temporarily halted training of its latest models for two weeks in August before resuming it under stricter controls. The company’s chief research officer, Jakub Pachocki, urged “extreme caution” on Sunday.
“International coordination around future AI development must become a top priority for governments around the world,” Pachocki wrote in a blog post.
Over 1,000 employees at AI companies have written an open letter urging the US government to support an international effort to regulate the pace of AI development.
What do you think?