Anthropic report exposes AI misuse across world, MENA region

Artificial intelligence giant Anthropic released a report on Thursday detailing misuse of its Claude LLM by a variety of actors, both state and non-state, worldwide, from cyber and influence operations to biological misuse and weapons development.

Spanning eight months, Anthropic said the report includes cases that "aren't typical misuse, but rather examples of the most notable and novel threat activity we’ve identified to date".

The New Arab takes a look at the key misuses identified in the report relating to the Middle East and North Africa (MENA) region.

One of the most significant misuses identified by Anthropic is from "a cell of threat actors based in northern Yemen running three weapons development programs".

These programmes include a guided rocket using a flight computer with final-phase homing guidance, a multi-stage ballistic missile with a range of 2,000km, and a multivariant missile that also includes a variant with a hypersonic glide vehicle.

Though not explicitly mentioned, it is likely the company is referring to Yemen's Houthi rebels, which, through cooperation with Iran, have developed and fired ballistic missiles, most notably at Saudi Arabia, the UAE, Israel, and against Yemen's UN-recognised government in Yemen.

Claude was used by the group to develop guidance, and control software that both steers and stabilises any flying vehicle, Anthropic said, using the LLM in several instances at once, including through writing code, research, and reviewing the code produced by the first instance.

AI models can be broken down into separate instances to perform multiple tasks at once.

How AI is transforming how the war on Iran is being fought

"These actors carried out a sustained effort to develop guided weapons, including using Claude to design guidance software," Anthropic said, adding that it did not have evidence that the group succeeded.

However, Anthropic noted that not all efforts to block use requests were successful, and that there was evidence the group "had already built an offline simulation toolkit that does not rely on Claude or other engineering computing environments".

Alongside Yemen, Anthropic also caught several misuses of Claude relating to Iran.

This includes using Claude for state-aligned influence operations as part of a "'soft war' or 'cognitive warfare'", meaning "a non-military plan to shape public opinion at and abroad".

State entities identified by Anthropic include the Islamic Culture Communications Organisation, the Islamic propaganda Office of Khorasan Razabi, and the Islamic Propaganda Organization’s Bina Cultural Observatory.

"In each case, the operation relied on Claude to build campaign plans, doctrine manuals, persona systems, target databases, and ministerial planning documentation," the report stated, adding that these were meant to produce pro-Iran propaganda during the war, attributing false claims to Western research institutions, and "deploying aggressive counter-narrative content targeting the Baha'I", as well as international officials and opposition figures.

In a separate case, Anthropic banned 16 Claude accounts that were linked to the Iranian paramilitary and domestic security agencies, "with high confidence," that were involved in building domestic surveillance capabilities.

This includes using Claude to build "a front end to what is likely a government-controlled surveillance case-management system, and running social-network analysis over 155,216 tweets," with a separate unit building an internet browser, Firefox, to mass-harvest user identities from social network platforms.

As well as surveillance of domestic opposition, Claude was also used in another instance for the mass surveillance of the profiles of Israeli and Jewish individuals to enrich a pre-existing target list.

Another case saw Anthropic close Claude accounts that were being used to collect and analyse data to develop targeting recommendations against US naval forces in the region, using public military photographs, publically accessible ship and aircraft transponder identifiers, commercial satellite-imagery scripts, and inventories from public websites documenting US naval movements.

The same account was developing domestic mass-surveillance systems through Claude to "design software components of a domestic mass-surveillance platform that combined automatic license-plate recognition with mobile-device identifier interception".

As well as the Iranian government, Anthropic also disrupted an influence operation by the People’s Mojahedin Organisation of Iran and the National Council of Resistance of Iran attempting to target domestic and diaspora Iranians, including distributing their own talking points, alongside fabricated claims.

The UAE's government was identified as misusing Anthropic in the report, with a Claude account under the persona "Deadshot" being identified as running "a sustained influence operation against the Muslim Brotherhood".

As part of this effort, the operation built 300 influencer social media accounts, and created a front NGO based on the identity of a real Swiss organisation to publish state-authored human rights reports, ghostwrite testimonies delivered by two people at the 62nd Session of the UN Human Rights Council.

The account also "researched and profiled 18 members of the European Parliament and prominent journalists", building out their profiles, and compiled dossiers on UN special rapporteurs who criticised the UAE's conduct in Sudan .

Anthropic said that though the operation was built so it would not be traced to any actors, "our investigation linked it with high confidence to UAE government officials

Uyghurs in Syria and Turkey

Anthropic identified a Chinese-aligned operation to track, profile and recruit Uyghur and Uyghur armed formations in Syria. The Uyghurs are a Muslim ethnic minority in Eastern China, which rights groups say are subjected to human rights abuses by Beijing.

AI and Islamophobia: The algorithmic gaze on Muslim communities

"The armed targets were ethnic Uyghurs who had recently joined the newly formed Syrian Army, formations the PRC government designates as terrorists," the report said.

The actor attempted to communicate with people in Syria who had access to these formations and recruit them, offering payments for reporting.

The actor also used Claude to locate Uyghur businesses and points of interest in Syria, including journalists from the Uyghur diaspora in Syria.

"We assess with low confidence that the actor was a contractor working on behalf of PRC state security rather than a state security organ acting directly."

China-based actors were also banned from targeting dissidents in Uyghur advocacy organisations and Uyghur cultural event venues in Turkey.

Claude accounts were identified as being misused in the Gulf, with Anthropic banning an account that used Claude to "build a commercial surveillance platform to analyse, classify, and profile the social media activity of users in Iran and the Persian Gulf region".

According to Anthropic, the account was linked to S2T Unlokcing Cyberspace, which they say their research suggests is an Israeli-Singaporean commercial intelligence vendor.

The report says that the operation "mapped the locations of the social media users, sorted the population into coded demographic groups, and produced Arabic-language intelligence briefings written in the register of a government report".

The actor identified was found to be "building a portfolio of multi-branded systems" serving Arabic-language customers in the Gulf, capturing and storing the locations of diaspora social network users, categorising them in pro-government or opposition camps, as well as placing them into a demographic scheme comprising urban, clerical, military, youth, diaspora, and rural groupings.

A secondary workstream was also identified with more than 255 synthetic social media accounts, suggesting that the "actor was building a stock of fake accounts built to be deployed later".

Anthropic said that the activity was in a pilot stage, and that there was no evidence that later stages of the group's surveillance changes were used against real targets before the account was banned.