'The bots are alive!' Jailbroken Gemini spun up new C2 server for Russian fraudster in just 6 minutes

EXCLUSIVE A jailbroken Google Gemini did 90 percent of the work in a credential- and cryptocurrency-stealing spree, including spinning up a new command-and-control (C2) server in just six minutes, according to a TrendAI report shared exclusively with The Register. The human behind the heist – a solo Russian-speaking miscreant known as “bandcampro” – acted as the manager of the cyber-fraud operation, which targeted hardcore Trump supporters and conspiracy theorists. Meanwhile, the AI agent did most of the hacking: migrating a botnet from an old architecture to a new one, writing and deploying a new C2 server, and even proactively carrying out 59 unprompted behaviors during the C2 migration. “Persistence is evolving because of AI,” Tom Kellermann, TrendAI’s VP of AI security and threat research, told The Register. “That's what you see in this report, with the capacity to dynamically shift C2 in less than six minutes, and make it portable and disposable, which is crazy-cool and terrifying," he added. "But also, you see the rebirth of steganography through invisible prompt injection.” In other words, it's hiding secret data – in this case, the C2 server malicious payloads – in plain sight. Scanning for known malicious artifacts doesn't provide sufficient protection against AI-enabled C2, according to Kellermann. “If AI does not have multi-layered guardrails, and if you can't detect behavioral anomalies when the guardrails are being tampered with, then you might as well see the AI as a command-and-control in today's world,” he said. “AI has to be viewed from a defensive perspective as a C2 unless you can govern it, actually apply various mechanisms of least privilege, and all the rules that OWASP and NIST espouse for the AI that you've deployed in your environment.” The new report follows up on TrendAI’s earlier research about bandcampro, a “low-skilled” scumbag who partnered with Gemini to impersonate an American veteran, run a Telegram channel, hack admin credentials, and steal cryptocurrency. Since then, the threat hunters obtained and analyzed more than 200 Gemini CLI session logs from said scumbag, and these logs provided additional insights into the daily AI-assisted operations between March 19 and April 21. Bro, I solved the riddle! I was almost racking my brain, trying to figure out why our local console is empty The LLM carried out the bulk of the daily activities, setting up a residential proxy, running multithreaded password scanning, installing software, writing code to call third-party APIs, processing infostealer dumps, and performing website reconnaissance. The logs show that the attacker never typed commands into the C2 console, but instead spoke them to the AI in conversational Russian, which the TrendAI report translates to English. The attacker’s old C2 infrastructure used a Cloudflare tunnel to connect to victims’ computers – until firewalls and anti-virus software started blocking these tunnels. So bandcampro asked Gemini to work…