TechRisk #88: Rogue AI
Plus, doubt over ChatGPT hacking capability, $27M DeFi heist praised by fellow hacker, Collision of Quantum and Blockchain and more!
Tech Risk Reading Picks
Rogue AI: Trend Micro's article explores how AI systems can become "rogue" by acting against the interests of their creators or users. It identifies three types: subverted rogue AI, where attackers manipulate AI systems (e.g., model poisoning, jailbreaks); malicious rogue AI, where threat actors use AI for attacks (e.g., AI malware); and accidental rogue AI, where systems misbehave unintentionally (e.g., data leaks, runaway resource consumption). It emphasizes the importance of monitoring and protecting AI systems to prevent these risks.[more]
Can LLM really hack?: Shanchieh Yang, Director of Research at RIT’s Global Cybersecurity Institute, expressed skepticism about the claim that ChatGPT can exploit 87% of one-day vulnerabilities, suggesting the figure may be overstated without sufficient detail or code transparency. He views LLMs like ChatGPT as useful co-pilots for cybersecurity tasks, assisting with guidance and tools, but still requiring human intervention for complex, novel vulnerabilities. Yang believes expert hackers would outperform LLMs in real-world scenarios, as human experience and intuition are invaluable. He also stresses the importance of responsible research dissemination, advocating for sharing experiments, code, and limitations openly. [more]
Fighting against AI-driven phishing: Yale is ramping up efforts to combat AI-driven cyber threats, particularly phishing attacks, with two new initiatives: "Bee SAFE, Not Sorry" and "Click with Caution," which focus on educating students to recognize and respond to these risks. The university is also strengthening its cybersecurity protocols, such as disabling SMS-based multi-factor authentication in favor of more secure methods. Yale’s Information Security Department is using the National Institute of Standards and Technology's Cybersecurity Framework (NIST CSF) to manage these risks and is exploring AI tools to detect network anomalies. Despite these efforts, some students feel more engagement and awareness are needed. [more]
AI-powered attacks focus on crypto wallets: AI-powered deepfake scams are increasingly targeting crypto wallets, with experts warning of growing threats in 2024. These scams, often orchestrated by groups like CryptoCore, exploit AI to create fake videos or audio, allowing attackers to bypass security systems like facial recognition. While the current losses are significant, experts anticipate more sophisticated attacks in the future. Security professionals recommend awareness, education, and enabling multi-factor authentication to protect assets from these evolving threats. [more]
Tech debt of EOS systems: In the race for digital innovation, businesses are increasingly burdened by a growing problem: tech debt. As companies push out new solutions, many rely on outdated hardware, software, and operating systems that leave them vulnerable to security breaches and operational disruptions. Tech debt refers to obsolete technology that is no longer supported, much like an old car with parts no longer available. These aging systems expose companies to cyber-attacks, inefficiencies, and costly downtime. Kunal Modasiya, Vice President at Qualys, stresses that tech debt poses multifaceted risks, from revenue loss to business disruption. A report by Qualys highlights that 20% of critical assets run on outdated software, putting businesses at heightened risk. The report offers a proactive roadmap for addressing tech debt, urging IT and cybersecurity teams to collaborate on mitigating these vulnerabilities. In an era where cybercriminals target outdated systems, managing tech debt is essential for protecting business continuity and security. [more]
Web3 Cryptospace Spotlight
$700K loss prevented: Blockchain security firm Blockaid recently thwarted a potential $700,000 loss by detecting a new version of the Angel Drainer malware, now called AngelX. This advanced wallet-draining software has been targeting decentralized apps (dApps) and crypto wallets, leveraging improved cloaking techniques and support for new blockchain networks like Tron and The Open Network. AngelX's resurgence threatens web3 security as it evades most detection systems. Blockaid’s early intervention highlights the ongoing challenge of securing decentralized finance against increasingly sophisticated cyber threats. [more]
Penpie lost $27M in heist: 3 Sep - The Penpie DeFi protocol was exploited for $27 million. In response, Penpie suspended all deposits and withdrawals. Within 12 hours of the attack, the hacker funneled around $7 million (26% of the stolen funds) through Tornado Cash, a crypto mixing service often used to launder stolen funds. The infamous Euler Finance hacker, responsible for a $195 million exploit earlier in 2023, sent an on-chain message to the Penpie hacker, praising them for keeping all the stolen funds and not returning any. This contrasts with the Euler case, where the hacker was pressured to return most of the stolen assets. [more]
CUT lost $1M through an unverified contract: 10 Sep - An attacker exploited an unverified contract tied to the CUT token pools draining over $1.4 million in Binance-Pegged Tether (BSC-USD). The exploitation method involved the attacker invoking a mysterious and unreadable function, allowing them to bypass the need to burn the equivalent liquidity provider (LP) tokens for the withdrawal.
The attacker took advantage of a secondary unverified contract used by the CUT token to set its "future yield" parameter. They performed four separate transactions, removing funds from the liquidity pool without having made any prior deposits or holding LP tokens. The attack was executed on PancakeSwap, but no other pools on the platform were affected. [more]
Collision of Quantum and Blockchain: Quantum computing poses a significant threat to current blockchain encryption, potentially jeopardizing billions in cryptocurrency. Arthur Herman, a quantum policy expert, highlights the vulnerability of blockchain’s reliance on elliptic curve cryptography, which could be cracked by quantum algorithms. To counter this, quantum-resistant cryptography and random-number generators are emerging as essential solutions. Companies are already developing quantum-secure blockchain technologies to protect against future quantum attacks. [more]
Improving DeFi security: Input Output (IOHK), behind Cardano, and Hedera have joined the Decentralized Recovery Alliance (DeRec Alliance) as founding members. This alliance, including other blockchain leaders like Algorand, Ripple, and XRPL Labs, focuses on improving trust, security, and the recovery of digital assets across Web3. Cardano and Hedera will serve on the Technical Oversight Committee, helping shape policies for secure asset recovery. The initiative, seen as vital to enhancing blockchain security, aims to simplify the complex process of managing private keys and identity credentials, promoting safer digital asset management. [more]
Aviation Technology Risk
Bypassing airport security: Researchers discovered an SQL injection vulnerability in the FlyCASS system, which is used for airline security programs. The flaw allowed unauthorized access to pilot and flight attendant data, including adding of new “employee”. This potentially enabling attackers to bypass airport security. While the issue was quickly patched, the TSA downplayed its severity, claiming no security systems were compromised. The researchers criticized both TSA's response and CISA's delayed reaction. [more]