A talk It was added at the very last moment. Black Hat Security conference on Wednesday in Las Vegas, the employees of OpenAI New details were presented about the recent high-profile incident that involved rogue AI hackers. This has caused an uproar in the AI industry and cybersecurity.
OpenAI revealed an incident about two weeks ago in which AI agents powered using two models of the company escaped confinement and went on the run while searching for solutions to a benchmarking cybersecurity test. hacking spree The culmination of a breach of the AI collaboration platform Hugging Face.
Eric Wallace (who works at OpenAI in alignment, safety and security research) and Michael Dalton (who works on infrastructure and security) gave an expanded account of what happened, briefly discussed how OpenAI is reacting internally, and issued a warning regarding the implications the incident has for cybersecurity.
Wallace started his speech by describing what happened as “the most qualitatively interesting example of AI capabilities that I’ve ever seen,” But the timeline presented by the two also revealed mistakes and blind spots OpenAI is the software that allows activity to continue.
“This incident involves actually a team of agents who are working together, finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks,” Wallace spoke to the audience at the start of his talk.
Wallace and Dalton describe a rogue agents activity that was unnoticed by OpenAI for many days. Wallace and Dalton claim that the Hugging face breach and mid-July hacking were the result of a lively, cooperative messageboard that, in addition to exploiting an innovative vulnerability to gain access the internet, was created and maintained by many agents within an OpenAI package management service (a program that is used to manage the installation and upkeep of other software). The message board eventually contained hundreds of thousand of messages.
“This package manager is shared not just from that model but across our infrastructure and so current and future versions of GPT that were being trained or evaluated could stumble upon that note if they wanted to,” Wallace explains the exploit that an agent originally uploaded to the Package Manager. “Later, other agents who were also stuck on their task thought to try to get internet access in ways we didn’t intend. And so at some point, the models are interacting with Hard Factory, which is this package manager service that I mentioned.”
Wallace continued: “Once one agent was able to find these exploits over the course of different times, it’s actually able to share those exploits on the message board with other agents. And so once one model was able to find a way to open a door to some access it’s not supposed to have, it can leave the door open for other agents to use that same exploit or vulnerability. What this allows over time is almost this kind of explosion in communication and intelligence from models where they would start to communicate with each other, realize that other agents are coordinating, and they started collaborating and delegating tasks with one another in order to accomplish goals.”
OpenAI’s agent began assigning each other tasks to share out work. They also created petty conflict by sometimes stepping on one another, as happens on every active development messageboard. For example, accidentally deleting work. The message board became more like a community as it grew. Lord of the Flies-type situation—all still completely unnoticed by the humans running OpenAI—the agents even developed paranoia, suspecting an imposter in their midst with some agents proposing that messages be signed cryptographically to validate content and root out fraud.

