Anthropic details bad actors' efforts to misuse its AI for bioweapons

Report comes two days after former employee quit claiming company’s models could cause human extinction by 2030

Criminals, state- groups, spyware vendors, scientists and propagandists have attempted to use Anthropic’s powerful artificial intelligence models to design missiles and bombs, create deadly pathogens and surveil dissidents, according to a threat intelligence report the company published on Thursday.

“The cases we here aren’t typical misuse, but rather examples of the most notable and novel threat activity we’ve identified to date,” Anthropic wrote in its 154-page report. “We’re publishing this work because we believe we have a responsibility to disclose malicious misuse of our services.”

In particular, the company detailed five case studies of scientists using its AI models in biological research. In these examples, Anthropic said the researchers circumvented its safeguards meant to prevent users from “unsupported regions”, as well as worked to hide the purpose of their work.

“Biological misuse is one of the most serious risks of frontier AI models,” Anthropic wrote. “Without the correct safeguards, such capabilities could have catastrophic consequences.”

Anthropic said it banned these accounts. The company did not reveal the names of the research institutions or the countries where the misuse took place, saying it was uncertain of the researchers’ intent.

Anthropic gave dozens of other examples of threat actors using its AI models. Those included cyber operations such as Russian espionage and “smash-and-grab” cyberhacks; surveillance operations, including a China-based program targeting Uyghurs in Syria and another targeting internal dissidents; propaganda campaigns in Russia, Malaysia, Iran and Bangladesh; and the use of its Claude AI model in Yemen, China and Russia to develop software for conventional weapons, including firearms, missiles, armed drones, bombs and other munitions.

In one of the examples of biological research, Anthropic found a scientist using Claude to work on a state- grant application to study the virus chikungunya. The virus is a mosquito-borne disease, similar to dengue and malaria, that can cause months of severe pain, fever and other debilitating symptoms.

Such research could be used to develop vaccines, but could also be used to create biological weapons. Anthropic told the New York Times this case was especially concerning because it could see the study was to be done at a military research institute.

The report comes just two days after an Anthropic employee, Jacob Coxon, set off a media firestorm with his resignation. He stated that he quit the company because it was not acting responsibly in creating its technology. Coxon said Anthropic and its rival, OpenAI, were “racing straight to self-improving superintelligence” that would cause human extinction by 2030. Current Anthropic employees posted public agreements with him.

Many AI experts say the real-world threats, like those detailed in Anthropic’s intelligence report, are far more concerning within the three years than an apocalypse brought on by an omnipotent intelligence.

“AI accelerationism and AI doomerism are two sides of the same coin: they both enforce the notion that an AGI superbeing will come into existence,” said Heidy Khlaaf, the chief AI scientist at the AI Now Institute. Khlaaf said AI labs creating technology that can be used for cybersecurity exploitation and weapons of war could be exceedingly more deadly.

In its report, Anthropic said that the potential real-world misuse from AI models is not typically in public view, instead, it’s investigated internally and by academics, governments and international organizations.

The company said that the entire AI industry needs to work together, alongside governments, to address these harms and create defenses.

“As models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer,” Anthropic wrote.

AI (artificial intelligence)

UK lawmakers urge Burnham to back ban on superintelligent AI after chilling warnings

UK lawmakers urge Burnham to back ban on superintelligent AI after chilling warnings

The Guardian view on controlling AI: humanity cannot outsource its survival

The Guardian view on controlling AI: humanity cannot outsource its survival

Tech whistleblowers warn AI could wipe out humanity. Doomspeak or not, we must take these claims seriously

Tech whistleblowers warn AI could wipe out humanity. Doomspeak or not, we must take these claims seriously

UK economy unexpectedly grows 0.4% in July boosted by AI

UK economy unexpectedly grows 0.4% in July boosted by AI

More Anthropic researchers warn of AI’s perils but Musk dismisses ‘psyop’

More Anthropic researchers warn of AI’s perils but Musk dismisses ‘psyop’

OpenAI not on track to reduce risk of ‘catastrophic’ loss of control, says board member

OpenAI not on track to reduce risk of ‘catastrophic’ loss of control, says board member

We have started losing control of AI. It’s time to shut it down

We have started losing control of AI. It’s time to shut it down

Weatherwatch: AI model beats standard methods at predicting cyclones

Weatherwatch: AI model beats standard methods at predicting cyclones

Lawmakers blast AI companies after researcher warns of human extinction by 2030

Lawmakers blast AI companies after researcher warns of human extinction by 2030

AI could kill all humans in decade, warn experts: but how seriously should we take them?

AI could kill all humans in decade, warn experts: but how seriously should we take them?