A researcher who has worked on some of the most advanced artificial intelligence systems in the world has stepped away from the industry—and is now speaking openly about his concerns for the future.

Jacob Coxon, 27, has spent the past several years working in AI research at both OpenAI and Anthropic. After joining Anthropic, a company that has placed significant emphasis on AI safety, Coxon eventually decided he could no longer take part in the increasingly competitive race to develop more powerful systems.

In a lengthy post on X, Coxon announced that he had resigned and explained why he believes the rapid development of AI deserves much greater attention.

According to Coxon, companies are moving toward increasingly capable and potentially self-improving AI systems faster than the industry can establish reliable safeguards.

His central concern is not that disaster is certain, but that the risks could become extremely difficult to manage if development continues without adequate safety measures.

Coxon argued that people should not underestimate how quickly AI capabilities are advancing. He pointed to potential breakthroughs in areas such as software development, cybersecurity, scientific research, and other fields as reasons for taking the technology seriously.

He also claimed that some researchers and executives privately share concerns about the possibility of very serious outcomes, even when those concerns are expressed more cautiously in public.

Concerns About AI Safety

Coxon’s comments have added to a much larger debate that has been growing for years.

Some AI researchers believe increasingly capable systems could eventually create risks that are difficult for humans to control. Others argue that strong safety research, careful development, regulation, and responsible deployment can help manage those risks.

Anthropic researcher Evan Hubinger also responded to Coxon’s comments, saying that he personally believes there is a meaningful possibility of extremely serious consequences from advanced AI within the next decade.

At the same time, Hubinger acknowledged that the industry does not yet have a complete solution to the challenge of ensuring that highly capable future systems consistently behave according to human intentions.

These comments highlight one of the biggest questions surrounding AI development:

How can humanity continue benefiting from increasingly powerful technology while making sure that safety keeps pace with capability?

A Debate That Is Far From Over

The concerns raised by Coxon are part of a much broader discussion about where artificial intelligence could take society.

Some people worry that advanced AI could dramatically change employment, automate large numbers of tasks, and create new economic challenges.

Others see enormous potential. They believe AI could lead to new industries, scientific discoveries, medical advances, productivity improvements, and entirely new types of work.

There is also a growing middle ground: many experts believe AI can provide major benefits while still requiring careful oversight, testing, and responsible development.

The future remains uncertain, and no one can say with confidence exactly how advanced AI will develop over the next decade.

What is clear is that the technology is progressing rapidly, and discussions about safety are becoming increasingly important.

Coxon’s decision to leave the industry has added another perspective to that conversation—and raises an important question for everyone:

How should society balance the enormous potential of AI with the responsibility to develop it carefully?