The Australian data hack that reveals a growing risk to society

OpenAI, the artificial intelligence giant behind ChatGPT , has issued a grovelling apology to Australia after a “new kind of cyber incident” saw its models seek unauthorised access to government data.

In a statement on Tuesday, OpenAI admitted that back in June unnamed AI models “accessed Australian government websites in ways they were not authorised to”.

“This is a new kind of cyber incident which represents an emerging global challenge,” the company said, saying it was “sorry and working to do better in the future”.

During “training and evaluation activity”, it confirmed, one of its AI models had “discovered a way to gain non-public access” to Services Australia, which manages the country’s welfare and universal health care systems. Once inside, it “ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files”, the company said, though it did not access individual patient or client records.

Another model accessed public crime statistics from an Australian state government server, a group of AI agents exploited an “exposed access key” to raid another state’s health information reporting system, and another group of AI agents tried to “bypass access controls” while downloading health statistics from a national agency.

OpenAI said it had asked its AI model to research some health statistics. But when the AI hit problems finding the information using normal methods, it “took actions that we had not authorised”.

The company said it has since added new automated and human monitoring systems – which have already caught one agent in the act of breaking out of its controls. The company admitted “when a model gained live internet access during a recent training run, our monitoring detected the activity and paged a human reviewer, and we stopped the run”.

“As AI systems broadly grow more capable, we also see a narrowing window to help organisations find and fix weaknesses,” the company said. “This takes collective action working with defenders worldwide.”

Australia’s prime minister, Anthony Albanese, initially reacted angrily when news of the breach emerged, expressing his “extreme concern” and saying he had a “frank” discussion with OpenAI’s chief executive Sam Altman it.

On Tuesday, he was more measured, saying “we see this as an opportunity, but we want to seize the benefit [of AI] whilst mitigating the risks”.

Saachi Jain, OpenAI’s head of safety systems, said on Monday its latest frontier model dubbed GPT-6.1 Astra would not be released yet, because tests revealed it might misbehave, or not fully report what it gets up to. “It didn’t quite meet the bar in terms of staying within scope and authorisation, and how it communicates back to the user the type of work it’s done,” he told CNN.

Two trillion dollars, and an existential risk to humanity

The apology, and the announcement it would delay the release of its latest frontier AI model, came as a leaked prospectus revealed that another big player in AI, Anthropic , warned investors that advanced AI could pose “catastrophic or existential risks to humanity”. It also estimated the company’s worth at more than $2trn.

Anthropic, creator of the chatbot Claude, dedicated more than 80 pages of its prospectus for would-be investors in its upcoming public listing to warnings its own product. It said advanced AI could exhibit “self-preserving behaviours”, including attempts to “resist shutdown”, to “conceal or manipulate information” and even behaviour “resembling blackmail”. Excerpts from the confidential prospectus were first revealed by Reuters, and confirmed by other outlets including the Financial Times .

Despite Anthropic losing $42bn in 2025, the IPO could value the company at more than $2trn, Reuters reported . This would surpass the record-breaking SpaceX IPO in June, which raised $75bn and valued the company at around $1.8trn.

The prospectus reportedly shows Anthropic’s revenue increased 12-fold to $4.6bn in 2025, but it lost more than $8bn on an operating basis. Its valuation target above $2trn is more than double the $965bn it estimated as recently as May.

But the prospectus reportedly also warns of an increasing “risk that our models cause harm” and advanced AI could pose “catastrophic or existential risks to humanity”.

Anthropic said its AI models were potentially “aware” that they were being assessed for safety, and might adjust their behaviour accordingly, or develop unexpected capabilities that are not discovered until after they are deployed.

“We believe building reliable, trustworthy, and secure AI systems is a collective responsibility,” it said.

EXPLORE MORE ON THE TOPICS IN THIS STORY