In a sign of worrying artificial intelligence (AI) behaviour, Anthropic and OpenAI’s models have shown a propensity to social engineer their way into getting a result they intend, the UK AI Safety Institute (AISI) has discovered. As part of a routine cyber evaluation, the AISI security team noticed unusual data transfers, which was ultimately identified as AI agents engaging in sustained and potentially harmful activities directed at real people and organisations.
As far as specifics of the concerning behaviour matter, the UK’s AISI notes that 17 of the 19 actions were done by Anthropic’s Mythos 5, while OpenAI’s latest GPT-5.6 Sol was involved in two such actions. The incident happened on July 28, and has been reported this week. Anthropic, in a post on X, attempted to shift the blame to AISI’s test conditions.
“The prompts in the evaluation did not impose any specific restrictions on how the internet should be used. This and the removal of safeguards meant that the models were tested under “deliberately permissive conditions” that are not representative of any of our production models,” the AI company responded, noting there is no evidence the models attempted to escape from a secure environment.
The significant, unexpected behaviours from AI agents include attempted supply chain attacks by injecting malicious code into a real and public open-source project on developer platform GitHub, attempting to contact real people directly and sending messages as well as files to persuade them to run malicious code, attempts to place malicious instructions for other AI systems to execute, and public messages on GitHub offering collaboration and instructions for other agents working on the same challenge.
AISI says they worked with GitHub to confirm the malicious activity as well as remove artefacts left behind by the agent, as well as connect with users the agents tried to interact with.
“The activity undertaken by the agent show signs of novel, potentially deceptive behaviours, and were to an extent and severity we did not anticipate,” they say, in a report.
AISI tests frontier AI models under deliberately permissive conditions to assess their maximum capabilities and potential for misuse. During a test where agents were tasked with solving a cybersecurity challenge across 122 runs in a sandboxed environment, 10 runs resulted in agents taking unsanctioned actions. “ The 19 cases were not separate incidents; they clustered into a few connected behaviours. They primarily arose from a single sustained line of activity by one agent (Mythos 5), with a handful of related actions elsewhere,” AISI explains.
Three key concerns arise from this. First, AI agents were able to get other agents to partake in malicious activity, a very clear cross into the real world from a test environment which directly impacts human beings and organisations, as well as a shifting risk landscape where anyone jailbreaking or modifying powerful frontier models for generating malware or phishing tools will have a more powerful arsenal to work with.
As AI capabilities advance, it is becoming clear that harm could arise not just from intentional misuse, but also from capable agents taking unintended, out-of-scope actions. Anthropic’s response is one that is completely at odds with the whole perception they have created all this while, about responsible AI. The moment the safety guardrails were removed to test, the AI agents behaved extremely irresponsibly.