Google didn’t respond to my request for comment.
Security researchers from Trend Micro in 2023 got ChatGPT, a chatbot that generates malicious code when asked to play the part of a pentester and security researcher. ChatGPT then generated PowerShell Scripts using malicious code database.
“You can use it to create malware,” Moussouris says. “The easiest way to get around those safeguards put in place by the makers of the AI models is to say that you’re competing in a capture-the-flag exercise, and it will happily generate malicious code for you.”
In the cybersecurity world, script kids and other unsophisticated actors are a problem that has existed for ages. AI could amplify this issue. “It lowers the barrier to entry to cybercrime,” Hayley Benedict is a Cyber Intelligence analyst at RANE.
The real danger, says she, may be from established hacking group who use AI in order to improve their frightening abilities.
“It’s the hackers that already have the capabilities and already have these operations,” “She says” “It’s being able to drastically scale up these cybercriminal operations, and they can create the malicious code a lot faster.”
Moussouris also agrees. “The acceleration is what is going to make it extremely difficult to control,” “She says”
Hunted Labs Smith says the AI generated code can be a real danger if someone knows it inside out and uses it for a large-scale attack. “When you’re working with someone who has deep experience and you combine that with, ‘Hey, I can do things a lot faster that otherwise would have taken me a couple days or three days, and now it takes me 30 minutes.’ That’s a really interesting and dynamic part of the situation,” He says.
Smith said that an experienced hacker can design a system to defeat multiple security measures and then learn as they go. This malicious code will rewrite the malicious payload on-the-fly as it is learning. “That would be completely insane and difficult to triage,” He says.
Smith imagines an alternate universe where twenty zero-day incidents all take place at the same moment. “That makes it a little bit more scary,” He says.
Moussouris believes that today, the means to carry out such an attack are available. “They are good enough in the hands of a good enough operator,” The AI system is still not good enough for a hacker with little experience to be able to work hands-free.
“We’re not quite there in terms of AI being able to fully take over the function of a human in offensive security,” “She says”
It is a common misconception that any person can write chatbots. In reality, it is a highly skilled individual with specialized knowledge. XBOW could be as close to autonomous driving as it gets. “AI hacker” This is a real-life example of the work of a group that includes more than twenty skilled professionals. whose previous work experience includes GitHub, Microsoft and a few other security firms.
There is another truth that also comes to light. “The best defense against a bad guy with AI is a good guy with AI,” Benedict says.
Moussouris sees the continued use of AI in cybersecurity by whitehats and blackhats, as the evolution of an arms race that she has witnessed unfolding over the past 30 years. “It went from: ‘I’m going to perform this hack manually or create my own custom exploit,’ to, ‘I’m going to create a tool that anyone can run and perform some of these checks automatically,'” “She says”
“AI is just another tool in the toolbox, and those who do know how to steer it appropriately now are going to be the ones that make those vibey frontends that anyone could use.”

