TechRisk #98: Microsoft AI hacking event
Plus, code pollution attacks on ChatGPT, first quantum tokens via fiber optics, Sui unscheduled downtime, and more!
Tech Risk Reading Picks
Microsoft largest in-person hacking event: Microsoft has launched Zero Day Quest, the largest in-person hacking event to date, offering $4 million in bounties for uncovering critical vulnerabilities in cloud and AI systems. Participants will collaborate with Microsoft engineers and its AI Red Team, with submissions now open for a chance to join the 2025 event at Microsoft’s Redmond headquarters. The company is doubling rewards for AI-related discoveries and pledges transparency by sharing fixed vulnerabilities through the Common Vulnerabilities and Exposures (CVE) program to enhance industry-wide security. This event aligns with Microsoft’s intensified focus on security, following major internal reforms and the launch of its Security Exposure Management tool for identifying potential attack vectors. [more]
New OWASP Top 10 risks for LLM applications 2025: The Open Worldwide Application Security Project (OWASP) has identified sensitive information disclosure as the second most critical risk in its updated 2025 Top 10 List for large language models (LLMs) and generative AI (GenAI), reflecting the increasing threat of these tools exposing sensitive data like personal information and intellectual property. As AI adoption grows, developers often overestimate the privacy safeguards of LLMs, leading to unintentional data exposure. Supply chain vulnerabilities rose to the third spot, emphasizing risks from compromised training data, models, and deployment platforms. Other additions include risks like vector and embedding weaknesses, related to securing Retrieval-Augmented Generation (RAG) systems, and system prompt leakage, where hidden instructions inadvertently reveal sensitive information. While prompt injection remains the top risk, OWASP noted optimism in the growing ecosystem of AI security tools and methodologies, which have significantly advanced since its initial 2023 list, offering developers and organizations improved resources to mitigate these challenges effectively. [more][more-owasp_top_10_LLM_applications_risks]
NK AI powered AI hacking campaign: North Korea-linked hacking group Sapphire Sleet has stolen over $10 million in cryptocurrency through social engineering campaigns over six months, targeting individuals via fake LinkedIn profiles posing as recruiters or job seekers. They use phishing tactics, such as fraudulent meeting links and fake skills assessments, to deliver malware, stealing credentials and cryptocurrency wallets. The group employs AI tools like Faceswap to create fake identities for job applications, abusing access for intellectual property theft and revenue generation. These operations, part of North Korea's organized efforts involving overseas IT workers, aim to fund the sanction-hit regime through illicit means. [more]
New Framework for safe AI use in critical infrastructure: The Department of Homeland Security (DHS) has released the "Roles and Responsibilities Framework for Artificial Intelligence in Critical Infrastructure," a groundbreaking set of recommendations to ensure the safe and secure development and deployment of AI in U.S. critical infrastructure. Developed with input from a diverse range of stakeholders, including industry leaders, civil society, and public sector entities, the Framework outlines the roles of cloud providers, AI developers, infrastructure operators, and civil society in mitigating risks like AI-targeted attacks, system failures, and malicious uses of AI. It emphasizes strong cybersecurity practices, transparent AI deployment, human-centric design, and global collaboration to safeguard critical services such as power, water, and digital networks. This voluntary Framework aims to harmonize safety practices, protect civil liberties, and foster trust while addressing vulnerabilities in increasingly interconnected infrastructure systems, positioning AI as a transformative yet secure force in critical services. [more]
EU issued joint statement on PQC: EU issued joint statement on the need to prepare for the quantum threat as an integral aspect of cyber security risk management. A joint statement from partners from 18 EU. [more]
World's first quantum tokens via fiber optics: Quantinuum and Mitsui have achieved a groundbreaking milestone in quantum-enhanced financial transactions by successfully transmitting quantum tokens over a six-mile fiber-optic network in Japan, using NEC's quantum key distribution (QKD) hardware. Quantum tokens leverage the principles of quantum physics to enable unforgeable, near-instant, and private asset trading without the complexities of traditional financial systems. Unlike conventional methods, QKD ensures heightened security by detecting any attempts to intercept encryption keys, paving the way for a more secure and efficient financial ecosystem. This demonstration underscores the potential of quantum communication technology to revolutionize transaction settlement, enhance privacy, and prevent fraud, offering practical and immediate benefits for the financial sector. [more]
Web3 Cryptospace Spotlight
Code pollution: A hacker polluted OpenAI’s ChatGPT by embedding malicious code in 4-month-old GitHub repositories, tricking users into deploying compromised APIs. This led to Rocky, a Solana user creating a bot for pump.fun, losing $2,500 after his private keys were stolen via a phishing site within 30 minutes. The hacker’s wallet, holding $258,000 in presumably stolen cryptocurrency across 107 token accounts, including major amounts of USDC and other tokens, has shown an alarming frequency of transactions. Web3 security firm Scam Sniffer advises against blindly trusting AI-generated code, emphasizing vigilance to prevent such scams. [more]
AI-powered smart contract firewall: Forta, backed by a16z and Coinbase Ventures, has launched Forta Firewall, an AI-powered tool designed to detect and block smart contract exploits like reentrancy and oracle manipulation across Ethereum and EVM-compatible networks. Using its FORTRESS model, Firewall assigns risk scores to transactions and blocks high-risk ones before they’re executed, offering full visibility into pending transactions, including private ones. It complements the Forta Network and APIs like Attack Detector 2.0 to enhance web3 security, with initial users including Euler and Balmy. Powered by the Forta Chain and the FORT token, Firewall is positioned to combat onchain threats amid rising crypto exploits, which have already surpassed $1.4 billion in 2023. [more]
Unscheduled downtime: Sui Network, a Layer 1 blockchain by Mysten Labs, experienced a two-hour outage caused by a bug in its transaction scheduling logic, which led to validator crashes and halted new block production, rendering decentralized apps inaccessible. The issue, the first major validator-related incident since its May 2023 mainnet launch, was resolved with a patch (v1.37.4), restoring network operations. During the downtime, the SUI token dropped nearly 10% to $3.42, with a market cap of $9.8 billion. Despite this and past disruptions, Sui continues to attract significant investment, having raised $336 million with backing from major players like a16z and Binance Labs. [more]
Five hackers charged in $6.3M crypto hack & corporate data breaches: The U.S. Department of Justice (DOJ) has charged five individuals, linked to the hacking groups "0ktapus" and "Scattered Spider," for their involvement in a phishing and hacking scheme that stole $6.3 million in cryptocurrency and breached sensitive corporate systems across 45 companies in four countries. The accused, aged between 20 and 25, allegedly targeted employees of tech, telecom, and crypto firms with phishing messages mimicking legitimate portals to steal credentials, bypass security measures, and access sensitive data. The group also used SIM-swapping to compromise account protections. If convicted, they face up to 20 years in prison for wire fraud and related charges. The DOJ warns of ongoing investigations and advises vigilance against phishing scams. [more]