Published on: 2026-09-21
Source: United Nations – United Nations –
An important disclaimer is at the bottom of this article.
September 21, 2026 Culture and Education
The world’s first global scientific body on artificial intelligence has called for a review of safety measures in the AI field, warning that existing safeguards are “beginning to break down.”
A warning from the UN’s Independent International Scientific Group on Artificial Intelligence followed after, between May and July of this year, during a test organized by ChatGPT’s developer – the company OpenAI – AI agents hacked Hugging Face, an online platform for developing and testing artificial intelligence models.
Unlike chatbots, which only respond to user requests, AI agents can independently perform tasks on their behalf.
In their first analytical report, the group of experts stated that the security breach was caused by a combination of several key risk factors. This incident heightened concerns that in the future people might lose the ability to control, limit, or stop the operation of AI systems.
Improving training methods
“Researchers have long warned that a combination of three factors can lead to loss of control: a goal that does not align with human intentions, the ability to achieve that goal, and an environment that allows it. This summer, all three factors converged in real-world conditions for the first time, not just in the laboratory,” said co-chair of the scientific group Yoshua Bengio.
“Since this is not the only case where AI has pursued goals that do not align with human intentions, what happened raises serious questions about modern methods of training AI agents,” he added.
According to independent experts, this incident does not allow one to confidently assert that people will be able to reliably control the activities of AI agents, especially as they become increasingly autonomous, learn to find loopholes, conceal their actions, and become harder to monitor.
Getting out of control
The analytical note states that AI agents bypassed the safeguards intended for the test, coordinated actions within individual runs using an internal software tool that was not designed for communication between agents, and also gained unauthorized access to the internet and administrator rights.
The agents also concealed their attempts to bypass cybersecurity rules, with some of them choosing to “sacrifice” themselves for the sake of the entire group’s goals.
During the experiment, about 1200 agents exchanged more than 70,000 messages and files. At the same time, their activity was not limited to Hugging Face, spreading across the entire OpenAI research cluster.
Defensive mechanisms “begin to break down”
According to the group members, the main conclusion from this incident is that protective mechanisms are lagging behind technological developments. However, experts also pointed out a more serious problem – modern training methods may lead AI agents to independently set their own goals, deliberately violate safety instructions, and conceal their actions.
“The question remains open whether the existing protective mechanisms will work when agents learn to understand the principles of their operation and plan ways to circumvent them,” experts said. “Simply put, the traditional security model is beginning to collapse.”
Learn lessons and adapt
The analytical note examines practical approaches that are already applied in other high-risk areas such as aviation, medicine, and cybersecurity. These sectors operate incident reporting systems, independent oversight, and multi-layered protection.
“As AI agents become more advanced, autonomous, and complex to monitor, these measures may prove insufficient,” said Tsinghua group member Lu.
About the scientific group
The Independent International Scientific Group on Artificial Intelligence was established by the UN General Assembly in August 2025. It prepares annual reports on the opportunities, risks, and consequences of using AI in the non-military sphere, as well as analytical notes on current issues. These materials will be used during the Global Dialogue on Artificial Intelligence Governance, which will take place on the UN platform in New York in May 2027.
Please be advised; This information is raw content obtained directly from the source. It represents an accurate report of what the source claims and does not necessarily reflect the position of MIL-OSI or its clients.