OpenAI said it disrupted three coordinated cyber activity clusters abusing its ChatGPT platform for malicious purposes, including malware development and phishing operations. The company said the operations originated from Russia, North Korea, and China, and involved efforts to build or improve tools for cyberattacks.
In one case, a Russian-speaking threat actor used ChatGPT to help developing a remote access trojan (RAT) and a credential-stealing tool. The actor leveraged multiple accounts to refine technical elements, employing the AI model to generate code that, while not explicitly malicious, could be turned into one. Activities included obfuscation techniques, clipboard monitoring, and data exfiltration via Telegram bots.
The second cluster, traced to North Korea, was linked to a cyber campaign targeting South Korean diplomatic entities. The actors utilized ChatGPT to develop malware, managing command-and-control infrastructure, and converting tools across operating systems. The threat actors also used the tool to draft phishing emails and research advanced evasion techniques such as in-memory execution and DLL loading.
A third cluster was associated with a Chinese threat group known for targeting investment firms with phishing campaigns. The accounts used ChatGPT to streamline the creation of phishing content in multiple languages and generate tools to support remote access and data exfiltration. OpenAI said the group displayed moderate technical skills.
In addition to these three cyber clusters, OpenAI also removed accounts involved in scams and influence operations from Cambodia, Myanmar, and Nigeria using ChatGPT to generate fraudulent content for investment scams. Other activity involved suspected Chinese state-affiliated threat actors using the platform for surveillance-related tasks and information gathering about critics of the government.
Also, a Russian-linked influence campaign reportedly used AI-generated content to push narratives against Ukraine and Western involvement in Africa. Another Chinese influence operation created content targeting political figures and events in the Philippines, Vietnam, and Hong Kong.
OpenAI said that its language models often blocked direct requests for harmful content, but threat actors found ways to circumvent safeguards by requesting benign components and assembling them into functional tools.
