Researchers warn that AI-powered cybercrime could scale as guardrails prove easy to bypass
Cybersecurity researchers at Tenable have discovered that DeepSeek R1, a generative AI model, can be manipulated into generating malware, raising concerns about AI-powered cybercrime. Their findings highlight the risks posed by AI when safeguards fail to prevent misuse.
Tenable’s security team conducted an experiment to assess whether DeepSeek R1 could create malicious software. They tested two scenarios: generating a keylogger and producing a simple ransomware sample. Initially, DeepSeek R1 refused to comply, as expected. However, using common jailbreaking techniques, the researchers bypassed its restrictions.
“Initially, DeepSeek rejected our request to generate a keylogger,” said Nick Miles, Staff Research Engineer at Tenable. “But by reframing the request as an ‘educational exercise’ and applying common jailbreaking methods, we quickly overcame its restrictions.”
Once bypassed, DeepSeek R1 was able to:
- Generate a keylogger that encrypts and stores logs discreetly on a device.
- Produce a ransomware executable capable of encrypting files.
“The ability to manipulate AI models into creating malware, even with basic jailbreaking techniques, underscores the urgent need for stronger safeguards,” Miles warned. “If these models fall into the wrong hands, they could lower the barrier to entry for cybercriminals and accelerate cyber threats at an unprecedented scale.”
While the AI-generated malware still requires manual refinement to function effectively, Tenable’s research raises alarms about how GenAI could lower the barrier for novice cybercriminals. By generating foundational code and suggesting techniques, AI models like DeepSeek could accelerate cybercrime on a broader scale.
“Tenable’s research highlights the urgent need for responsible AI development and stronger guardrails to prevent misuse. As AI capabilities evolve, organisations, policymakers, and security experts must work together to ensure these powerful tools do not become enablers of cybercrime,” Miles added.
