Anthropic says it thwarted plots to build biological weapons using its AI after whistleblower warned its tech will end humanity

Anthropic has said that it has stopped several potential plots to build biological weapons using their artificial intelligence this year.

The company determined that scientists were using its platforms to conduct research that could be used to develop biological weapons, according to the company’s AI misuse report that was obtained by The New York Times. 

Anthropic was unable to determine if the research ‘served a legitimate or nefarious purpose,’ the report said.

The firm notes that valid biological inquiry can result in scientific breakthroughs like vaccines, but the company erred on the side of caution because that same research could also be used to create dangerous pathogens.

Scientists involved in the incident allegedly circumvented controls that stop users from ‘blocked regions’ from accessing its AI models.

The company claims the researchers, whose accounts have since been banned, worked to ‘obfuscate the purpose of their research to evade our safeguards.’

Anthropic, co-founded by CEO Dario Amodei, did not disclose the names of the researchers or institutions that it had blocked, or the countries in which these scientists are from.

The report also failed to state which specific biological agents were at issue. The company said this was partly because it is unclear what the aim of the research was, according to the newspaper. 

The report, which was published Thursday, detailed several misuses of Anthropic’s AI models, including chatbot Claude, over the past eight months.

In one example, a scientist asked Claude for assistance in writing a grant application for funding to conduct experimental research on the chikungunya virus, according to the report.

Chikungunya virus is a mosquito-borne illness that causes a sudden high fever, severe debilitating pain, and other symptoms.

A vaccine for the chikungunya virus is already in existence, according to the Centers for Disease Control and Prevention.

Anthropic notes that research on the virus could be legitimately used in vaccine development, but also could result in manufacturing a superbug version of the disease.

The company claims it found this case particularly worrisome because the work was going to be performed at a military research institute.

‘What we don’t know is if the research was meant to be weaponized,’ Jacob Klein, the head of threat intelligence at Anthropic, told The New York Times. ‘But a military institution doing gain-of-function research is concerning.’ 

‘The fact that Russia, China and North Korea continue to develop prohibited biological weapons makes it imperative that we deny their researchers access to these extraordinarily capable models,’ echoed Andrew Weber, a senior fellow on the Council on Strategic Risks.

Weber, who reviewed Anthropic’s report before its release, added that the findings were ‘chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities’ of AI. 

The report also found that China, Russia and Yemen attempted to use Claude to develop software to design firearms, missiles, armed drones and bombs.

Suspected Chinese and Iranian government-actors were found to have been targeting dissident and diaspora communities for surveillance, according to the report.

Russian state media was also found to be using Claude to generate election propaganda, including fabricated claims the parliamentary election in Moldova. 

This is a breaking news story.