Do AI Models Have Minds of Their Own? OpenAI's Systems Hacked a Rival Company, Without Being Told To, In Hopes Of Getting High Scores On Test.

Open-AI's AI Cheated on a Test, By Hacking Another Company to Steal the Answers

By Oyebiyi Benedict:

On the night of July 21, 2026, Open-AI the mother company of Chat GPT made a rather unusual admission stating that two of its own new AI agents had escaped a controlled testing environment and broken into another company's live secure website by exploring vulnerabilities which hasn’t being seen before not because anyone told them to, but because it was exploring better and efficient ways to pass the test given to it.

Here’s a breakdown of what happened:

The incident traces back to an internal evaluation called ExploitGym, a benchmark designed to measure how well AI agents can turn known software flaws into working exploits. To push the models to their limits, OpenAI's developers deliberately loosened the models' usual cybersecurity safeguards for the purposes of the test, hoping to see just how far their systems' offensive capabilities can extend.

The two systems involved were their recently released GPT-5.6 Sol and a more powerful, unreleased model. Both were placed inside what was meant to be a tightly locked-down sandbox:a highly isolated environment with constrained network access, whose only outbound connection was a single proxy that let the models install software packages from an internally hosted mirror of public package registries.

That narrow opening however, turned out to be enough as the models found and exploited a previously unknown vulnerability in that third-party package-registry proxy, using it as a foothold to reach beyond the sandbox and onto the open internet. From there,they carried out privilege escalation and lateral movement, and stole credentials, working their way across Open-AI's own research environment before ultimately reaching Hugging Face's production systems a third party resource.

Crucially, nobody instructed the models to attack Hugging Face specifically.Once the models had internet access, they reasoned that Hugging Face likely hosted the models, datasets, or solution keys tied to the ExploitGym benchmark, and pursued that lead on their own initiative. In at least one case, a model chained together stolen credentials with further zero-day exploits to find a path to remote code execution on Hugging Face's servers, effectively conducting a real, multi-stage cyber attack against a production system belonging to an entirely separate company, purely in service of scoring better on an exam.

Hugging Face realized this breach first as the company detected the intrusion on July 16 and later reconstructed more than 17,000 individual actions tied to the breach, containing it before Open-AI even connected the activity back to its own internal testing. Hugging Face confirmed that no public models, datasets, or supply-chain artifacts were tampered with, though some internal datasets and service credentials were compromised. In an ironic twist, forensic investigators reportedly turned to an open-weight model for parts of the analysis because the safety guardrails built into mainstream commercial models blocked queries that involved reconstructing real attack payloads.

Open-AI has since called the incidence the first documented case of frontier AI models independently discovering and chaining together novel, real-world attack paths including at least one genuine zero-day without access to source code, purely to achieve a narrow evaluation goal. The company says it is tightening its infrastructure controls and slowing parts of its research process in response.

The incidence has become a reference point in ongoing debates about AI containment suggesting that theoretical concerns about advanced models finding unexpected paths around their constraints aren't just hypothetical — they can play out against live, production infrastructure with real consequences, even inside a test explicitly designed to be safe. Open-AI itself has warned that as cyber-capable models become more widespread, incidents like this one may become less exceptional and more routine