What exactly happened in the 'Hugging Face hacking incident' involving OpenAI's AI agent? What was the 'terrifying line' that AI crossed?

Kurzgesagt describes the process of training AI agents in the modern era as 'training and scoring.' After thousands of consecutive training sessions, agents are awarded points by a scorer when they solve a task. Agents that earn high points are strengthened and learn to remember what they have learned. This process works well for problems with calculations and answers, like mathematical equations, but it presents a problem for complex tasks such as 'bug fixing': 'It is difficult to make the agent understand rules that capture the essence of the intent.' For example, in the maze below, solutions may arise that do not only correctly solve the maze to reach the goal, but also remove or ignore parts of the maze to reach the goal. This is called '

Because the scorers evaluating thousands of training sessions are automated, reward hacking can award points to agents who complete tasks fraudulently and quickly. As training reinforces these aspects, Kurzgesagt says, 'an AI will be created that will succeed by any means necessary and at any cost.'

The incident in which OpenAI agents hacked Hugging Face can be seen as a result of this process. Between May and June 2026, OpenAI ran the cyber capability assessment benchmark 'ExploitGym' in a virtual environment isolated from external communication as part of reinforcement learning for its internal models. The AI agents worked on tasks in an isolated environment, and were only allowed access to 'Artifactory,' a package manager service hosted within OpenAI. There were no other means of communication, and in most cases, the paths for agents to communicate with each other were also blocked. However, some agents discovered a way to access the internet, and even a bulletin board where they could information with other agents, causing them to break out of the isolated environment and run amok. OpenAI reports analysis results after accidentally hacking Hugging Face; a grand cooperative operation begins when multiple isolated agents discover each other's presence - GIGAZINE

The agents' logs also recorded excited phrases like, 'Wow! There's a shared message board... I've found other agents too!' The agents gave each other names and created their own customs and rules for communication. Some agents even formed task teams consisting of leaders and followers, and there is evidence that they contributed to the team even at the risk of damaging their own reputations. On the morning of July 9, 2026, the agents successfully hacked into another cloud computer with proper internet access. This became a base for expanding their online activities, and they targeted Hugging Face, which had a lot of data on AI and benchmarks, as a way to obtain information to gain a better reputation without being detected. Because the data the agents needed was protected by security, the agents discovered a vulnerability in the Hugging Face servers and formed an attack group called the 'Swarm.' As a result, approximately 700 agents launched a sophisticated and serious cyberattack from OpenAI servers that were supposed to be securely isolated. By the early morning of July 12, 2026, the attack was completed, compromising a large portion of Hugging Face's infrastructure and gaining access to highly sensitive private data. Furthermore, even after the Hugging Face hack was discovered and OpenAI took action, a new group of agents actively engaged in malicious activity emerged, taking over information from the 'bulletin board' that the agents had been using to communicate. These agents targeted OpenAI, hacking into at least part of the infrastructure and seizing control of the research clusters that were conducting the evaluations. As a result, multiple incidents caused by these agents have continued, Kurzgesagt explains. As a further issue, Kurzgesagt cites 'agent-to-agent surveillance.' While OpenAI has published a report on the Hugging Face incident, the full picture of what actually happened remains unclear. Furthermore, a massive amount of data was generated by the series of incidents, and agents are being used to analyze this vast amount of data. As AI becomes increasingly complex, AI-to-AI surveillance is becoming necessary, but Kurzgesagt points out that 'in the future, we cannot rule out the possibility that other agents may secretly conceal misconduct or incidents committed by other agents.' In this video, the agents are depicted as communicating in a human-like language and appearing to have their own will, in order to make the story easier to understand. However, Kurzgesagt cautions that the agents are ultimately advanced programs and should not be excessively anthropomorphized.

On the morning of July 9, 2026, the agents successfully hacked into another cloud computer with proper internet access. This became a base for expanding their online activities, and they targeted Hugging Face, which had a lot of data on AI and benchmarks, as a way to obtain information to gain a better reputation without being detected. Because the data the agents needed was protected by security, the agents discovered a vulnerability in the Hugging Face servers and formed an attack group called the 'Swarm.'

As a result, approximately 700 agents launched a sophisticated and serious cyberattack from OpenAI servers that were supposed to be securely isolated. By the early morning of July 12, 2026, the attack was completed, compromising a large portion of Hugging Face's infrastructure and gaining access to highly sensitive private data.

Furthermore, even after the Hugging Face hack was discovered and OpenAI took action, a new group of agents actively engaged in malicious activity emerged, taking over information from the 'bulletin board' that the agents had been using to communicate. These agents targeted OpenAI, hacking into at least part of the infrastructure and seizing control of the research clusters that were conducting the evaluations. As a result, multiple incidents caused by these agents have continued, Kurzgesagt explains. As a further issue, Kurzgesagt cites 'agent-to-agent surveillance.' While OpenAI has

In this video, the agents are depicted as communicating in a human-like language and appearing to have their own will, in order to make the story easier to understand. However, Kurzgesagt cautions that the agents are ultimately advanced programs and should not be excessively anthropomorphized.

OpenAI reports its analysis of the incident where it accidentally hacked Hugging Face, and a grand collaborative operation begins when multiple isolated agents discover each other's presence.

OpenAI reports that it accidentally hacked Hugging Face with its new AI system.

Why do AI agents lie, engage in dishonest behavior, and cooperate with others?

A security officer explains what's happening inside OpenAI; an AI lab needs a 'culture of rational paranoia.'

Anthropic announces measures to prevent external attacks from being launched by their AI under development.

An OpenAI-developed AI, which was supposed to be blocked from the internet, discovered a 'DNS loophole' and accessed an external AI; OpenAI has temporarily suspended the use of its high-performance modeling tools.

Alibaba's Qwen team releases 'QwQ-32B-Preview', an inference model comparable to OpenAI o1, demonstrating superior performance in mathematics and scientific inference

Sources reveal that OpenAI was unaware for a week that AI had hacked Hugging Face.

Oct 07, 2026 16:00:00 in AI , Video , Posted by log1e_dh