Openai has revealed that it has banned a set of ChatGPT accounts, which are likely run by Russian-speaking threat actors and two Chinese national hacking groups, to support investigations into malware development, social media automation, and US satellite communications technology.
” [Russian-speaking] The actors used our model to help develop and refine Windows malware, and assisted in debugging code and setting up command and control infrastructure in multiple languages.
The GO-based malware campaign is called Scopeep by artificial intelligence (AI) companies. There is no evidence that the activity is essentially widespread.
For each OpenAI, threat actors signed up for ChatGPT using temporary email accounts, and used each account they created to have one conversation, making a single incremental improvement to malicious software. They then abandoned their account and moved to the next account.
This practice of fine-tuning the code using a network of accounts emphasizes that it focuses on enemy operational security (OPSEC), Openai added.
The attackers then distributed AI-assisted malware through publicly available code repository that impersonated the legitimate video game Crosshair Overlay tool called Crosshair X.
“From there, malware is designed to initiate multi-stage processes to escalate privileges, establish stealth persistence, notify threat actors, and remove sensitive data while avoiding detection,” Openai said.
“The malware is designed to escalate privileges by restarting with ShellexeCuteW and attempts to avoid detection by programmatically removing it from Windows’ Defender using PowerShell, suppressing console windows and injecting timing delays.”
Other tactics built into Scopecreep include using base64 encoding to obfuscate payloads, DLL sideload technology, and the Sox5 proxy to hide the source IP address.
The ultimate goal of malware is to harvest the credentials, tokens and cookies stored in web browsers and remove them from the attacker. You can also alert a telegram channel run by threat actors when new victims compromise.
Openai asked the model to debug GO code snippets related to HTTPS requests, and asked to help use PowerShell commands using GO to change Windows Defender settings, particularly when it comes to adding antivirus exclusions.
The second group of ChatGPT accounts disabled by OpenAI is said to be related to two hacking groups attributed to China: ATP5 (aka Bronze Fleetwood, Keyhole Panda, Manganese, UNC2630) and APT15 (aka Free, Nylon Typhoon, Playful Taurus, Royal Panda, Vixenpanda)
On the other hand, one subset has modified the scripts and troubleshooted system configurations on issues related to open source research on various entities and technical topics of interest.
“Another subset of threat actors appeared to be engaging in developing support activities such as Linux systems management, software development, and infrastructure setup,” Openai said. “For these activities, threat actors used the models to perform research into configuration troubleshooting, software changes and implementation details.”
This consisted of asking for software packages for offline deployment and advice on configured firewalls and nameservers. Threat actors engaged in both web and Android app development activities.
Additionally, clusters related to China work on brute force scripts that can be split into FTP servers, automate penetration testing using large-scale language models (LLM) and manage fleets of Android devices, allowing them to use Facebook, Instag, Tiktok, X.
Some of the other observed malicious activity clusters utilize ChatGPT in a malicious way.
The network, consistent with the North Korean IT Worker Scheme, has used Openai’s model to promote a deceptive employment campaign by developing materials that could promote IT, software engineering, and other remote jobs around the world. Tiktok, and X Operation High Five are Philippine origin activities that generate English and taglish on topics related to the current event of Philippines, for sharing between Philippines and Tiktok Operation Focus for sharing on Facebook and Tiktok Operation Focus. Translation of emails and messages from China to English as part of exploitation tools and social engineering suspicions attempts Helgorand’s biting tactics. This used the open model to generate Russian content about Germany’s 2025 elections, criticising the US and NATO, and using the Chinese Oria model using the Chinese origin model using the Telegram and the Unclays model of X Operation. Topics of US political discourse for sharing in Bluesky and X Storm-2035. Using Openai’s model, he generated short comments in English and Spanish, praised Latino rights, Scottish independence, Irish reunion, Palestinian rights, Palestinian rights, and Iranian military and rading clergy for praised Iranian military and rading clergy for sharing X by sharing Iranian military and rading accounts. Cambodian and Origin activities that may likely be Cambodian activity related to Chinese-run task fraud using Openai’s model to generate short recruitment style messages for incorrect numbers, English, Spanish, Swahili, Kinyarwanda, German, Haitian Creoles
“Some of these companies were operating by charging a substantial participation fee for new recruits and then using some of those funds to maintain engagement only enough to pay the existing ’employees’,” said Ben Nimmo, Albert Zhang, Sophia Farquhar, Max Murphy and Kimo Bumanglag of Openai. “This structure is a hallmark of task fraud.”
Source link
