This post is also available in:
עברית (Hebrew)
OpenAI has recently taken action against several user accounts it says were attempting to misuse its ChatGPT platform for surveillance and cyber operations. The company’s latest threat report outlines activity linked to state-affiliated groups and criminal actors, with a focus on China and Russia.
According to the report, some of the banned accounts requested assistance in developing tools to monitor social media platforms. These requests, written in Chinese, aimed to generate ideas for systems that could track online discussions—violating OpenAI’s national security use policies.
According to Reuters, in addition to surveillance-related queries, OpenAI said it detected and disabled accounts that used the platform to support phishing schemes and explore malware automation. Some of these requests involved asking ChatGPT to help identify how China’s open-source large language model, DeepSeek, could be integrated into cyber operations.
The company also reported shutting down accounts suspected of being linked to Russian-speaking cybercriminal networks. These users reportedly attempted to use the model to help write malicious code, although OpenAI noted that its systems blocked most harmful prompts and did not provide users with advanced offensive capabilities.
Since it began publishing threat reports in early 2024, OpenAI says it has identified and disrupted more than 40 networks attempting to exploit its models. While the platform is designed to reject requests involving malware, fraud, or manipulation, the company acknowledged that persistent attempts still require active monitoring and intervention.
No new tactics were identified in this report, and OpenAI emphasized that the chatbot has not enabled new forms of attack. However, the findings highlight ongoing concerns around the use of generative AI by foreign actors for surveillance, cybercrime, and information operations.
The issue arises amid broader geopolitical tensions, particularly between the U.S. and China, as both countries invest heavily in artificial intelligence.
