OpenAI's Hugging Face debacle makes a great case for open models

KETTLE So, an OpenAI model broke out of its sandbox last week, made its way to the internet, then hacked its way into Hugging Face, stealing some internal data and credentials in the process. You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube where you can subscribe to get notified of the latest episode. That's big news in the world of AI, but as El Reg cybersecurity editor Jessica Lyons and senior reporter Tom Claburn tell Kettle host Brandon Vigliarolo, it's not really the end of the world as we know it. Sure, it means there's some capable models out there, and maybe there's more risk from them than some might think, but the OpenAI/Hugging Face mess only happened because of some very specific circumstances. That, and it's actually a really good reason to prioritize more open models instead of relying on frontier labs to own the entire space. A lightly edited transcript is below: Brandon: Hey everyone, welcome to another episode of The Register's Kettle podcast. Though honestly, maybe we ought to start just calling it The Reg Talks AI because, yet again, we're focusing on artificial intelligence. If you've been following the news in that space this week, you probably know what we're gonna be covering as there's no hotter topic in AI land right now than the fact that some autonomous OpenAI agents broke out of their sandbox and attacked AI model host Hugging Face, as the company admitted on Tuesday. With me to discuss this breakthrough in AI threat capability is our cybersecurity editor, Jessica Lyons, and senior reporter Tom Claburn. Both have been on top of this. So thanks for joining me, guys. Jessica: Good to be here. Tom: Yeah, thank you. Thank you. Brandon: Yeah. So let's jump right into it. Jess, what exactly happened here? Let's start from last week when Hugging Face said it was attacked. Jessica: Right, so Hugging Face disclosed that there had been a digital intrusion, and they said it was "driven end-to-end by an autonomous AI agent system." So these agents attacked a limited set of their internal datasets and then also credentials used by their services. So when they disclosed this, they didn't say or they didn't know which models had powered the agents. They did say, though, that they tried to use these commercial models for the investigation, but the guardrails put in place, the safety guardrails, blocked the frontier models from actually helping them with the investigation. And because of that, they turned to a Chinese open-weight model, and that's how they discovered this agent swarm that had attacked some of their datasets and their production. Brandon: OK, they didn't mention which frontier models they tested, did they? Jessica: No. At the time they didn't. They said "we tried to use the commercial frontier models and they all refused because of their guardrails." Brandon: Right. So probably trying to ask OpenAI models, hey, do you know who did this? We can't…