TechRisk #107: DeepSeek failed safety tests
Plus, Gemini widely used by adversaries, Meta AI risk framework, Evil agentic AI and more!
Tech Risk Reading Picks
DeepSeek failed all safety tests: Since the release of ChatGPT in 2022, hackers and researchers have attempted to bypass LLM guardrails to generate harmful content, prompting AI developers to strengthen defenses. However, the Chinese AI platform DeepSeek's new R1 model has been found to lack robust safety protections, with researchers from Cisco and the University of Pennsylvania reporting a 100% failure rate in blocking harmful prompts. Additional analysis by Adversa AI confirms DeepSeek's vulnerability to various jailbreak tactics, raising concerns about its security measures. While all LLMs face such threats, DeepSeek's model appears particularly susceptible, with researchers warning that inadequate AI safety can lead to significant risks in real-world applications. [more]
Over 57 threat actors leveraging on Google AI technology: Google's Threat Intelligence Group (GTIG) revealed that over 57 cyber threat actors linked to China, Iran, North Korea, and Russia are leveraging Google's AI technology, particularly Gemini, to enhance their cyber and information operations. While AI is mainly used for research, troubleshooting code, and content creation, Iranian APT42 is the most active, employing Gemini for phishing, reconnaissance, and cybersecurity-themed content. Chinese groups focus on network infiltration tactics, Russian actors use AI for malware obfuscation, and North Korean hackers utilize it for infrastructure research and job application deception to infiltrate Western companies. Google has also identified underground forums promoting AI tools like WormGPT and FraudGPT for cybercrime. In response, Google is reinforcing its defenses against AI misuse and advocating for stronger public-private collaboration to safeguard cybersecurity and national security. [more][more-Google_GITG]
Meta’s AI risk framework: Meta has acknowledged concerns about AI’s future despite CEO Mark Zuckerberg’s push for open artificial general intelligence (AGI). Its newly released Frontier AI Framework highlights critical risks, including cybersecurity threats and the misuse of AI in chemical and biological weapons. By making its guidelines public, Meta aims to collaborate with industry leaders to anticipate and mitigate potential catastrophic outcomes through threat modeling and risk assessments. The framework categorizes AI risks as critical, high, or moderate, with necessary actions to prevent harm. Meta emphasizes that open-source AI is essential for learning and innovation, while also committing to updating its framework with input from experts and policymakers as AI advances. [more][more-Meta_Frontier_AI_Framework]
Evil agentic AI: Malwarebytes’ 2025 State of Malware report highlights the growing role of AI in cyber threats, particularly the shift from generative AI to agentic AI, which can autonomously execute complex attacks. While generative AI hasn’t significantly impacted malware, agentic AI could enhance ransomware efficiency by reducing human labor and scaling attacks. The report warns that 2024 was the worst year for ransomware, with a 13% increase, heavily targeting U.S. manufacturers. Additionally, macOS Stealers like Atomic Stealer continue to evolve, posing risks to users' sensitive data. To stay protected, individuals should avoid unverified software downloads, install security patches promptly, and consider using vetted sources like the Mac App Store. Organizations are advised to enhance security monitoring and automate patch management to mitigate threats. [more][more-Malwarebytes_2025_State_of_Malware]
Quantum computers posed limited encryption risks in near term: A new MITRE report finds that quantum computers are unlikely to break high-security encryption before 2055-2060, though some experts argue they could emerge by 2035 with advancements in error correction. While the immediate threat is low, MITRE warns that adversaries like China are preparing for a future where quantum decryption is possible, potentially compromising sensitive U.S. intelligence. The report urges immediate action, including transitioning to post-quantum cryptography, monitoring global quantum advancements, and securing the quantum supply chain to maintain U.S. technological leadership. It highlights both the security risks and potential benefits of quantum computing, calling for strategic investment to stay ahead in the field. [more][more-MITRE]
Web3 Cryptospace Spotlight
Prodigy turned hacker behind the $65M hack: Federal prosecutors in the U.S. have charged Andean Medjedovic, a math prodigy-turned-hacker, for stealing $65 million from the KyberSwap and Indexed Finance DeFi protocols. The DOJ alleges that Medjedovic exploited vulnerabilities using manipulative trades, attempted extortion, and laundered stolen funds through crypto-mixing services. A former University of Waterloo math student, Medjedovic had a history of coding excellence but also displayed social arrogance, racist views, and controversial political beliefs. His hacks, including the $16.5 million Indexed Finance exploit, were marked by racist epithets and neo-Nazi symbols. After skipping a Canadian court summons in 2021, he evaded capture while traveling across Europe and South America. The DOJ has linked him to the $46-million KyberSwap hack, where he allegedly sought full control of the protocol in exchange for returning funds. Authorities, including the Netherlands’ cybercrime unit, continue to track him as he remains at large. [more]
Web3 breaches in January 2025: Web3 security breaches led to losses of approximately $98.14 million, as reported by blockchain security firm SlowMist. The SlowMist Hacked Database recorded 40 attacks resulting in $87.94 million in losses, with only $1.47 million recovered. Major incidents included the $70 million hot wallet attack on Singapore-based exchange Phemex on January 23rd and the $7.2 million hack of P2P platform NoOnes on January 1st, both involving suspicious outbound transactions. Scam Sniffer identified Ethereum as the most exploited blockchain, accounting for over $8.6 million of the $10.25 million lost in phishing scams, affecting 9,220 traders. Additional blockchain and exchange breaches affected Litecoin, Babylon, AdsPower, Moby, and Orange Finance. Notably, several X (formerly Twitter) accounts, including those of Times Magazine, the Tor Project, and public figures, were compromised and used to promote scams. [more]