Cybergangs Are Using AI-Generated Code – What Are the Implications?

This post is also available in: עברית (Hebrew)

Cybercriminals impersonating legitimate German companies are attacking organizations across various industries in the country using AI, as was reported by cybersecurity company Proofpoint.

The hackers, which were identified as TA547, are sending the victims emails with fake invoices in password-protected ZIP files and providing a password in an email itself for the victim to use and “unpack” the malware. This malware reportedly contains an LNK file that triggers a PowerShell script activating an information stealer.

According to the report, experts suspect the PowerShell script was generated by an LLM (such as ChatGPT, Gemini, etc.), making it the first recorded cybercrime case using an AI-generated code. The reason experts suspect the use of AI because of several signs, including some too-thorough, grammatically perfect, hyper-specific comments throughout the script. “This is a typical output of LLM-generated coding content and suggests TA547 used some type of LLM-enabled tool to write (or rewrite) the PowerShell or copied the script from another source that had used it.”

But why does this matter? According to Cybernews, this emerging shift in some techniques indicates that malicious cyber actors are increasingly leaning on LLMs to launch more sophisticated attack chains, since these models help generate social engineering lures and code that allows threat actors to upscale their malicious campaigns.

In these recent attacks, the LLM-generated content did not change the functionality or the efficacy of the used malware and didn’t impact the defenders’ ability to detect malicious actions. While the AI-written script assisted in delivering a malware payload, it did not alter the payload itself.

Proofpoint claims TA547 is a financially motivated cybercriminal threat they consider an initial access broker (IAB) that targets various geographic regions. They further report that since 2023 TA547 has been typically delivering NetSupport RAT but has occasionally delivered other information stealing payloads. In addition to these campaigns in Germany, other recent geographic targets include organizations in Spain, Switzerland, Austria, and the US.