TechRisk #93: Extract PII from GenAI
Plus, concern over rapidly evolving AI threats, Radiant hack likely a hardware wallet attack, and more !
Tech Risk Reading Picks
Extract PII from LLM: The Imprompter attack on LLM agents uses a crafted prompt that tells the AI to extract personal information from the user's conversation. The prompt is obfuscated to appear as random characters to humans but is interpretable by the AI. This prompts the AI to gather information and embed it in a Markdown image command linked to a URL owned by the attackers. When the AI tries to retrieve the image, it inadvertently leaks the personal information. The response appears as an invisible 1x1 pixel, concealing the leak from users. Researchers demonstrated the attack by successfully extracting personal details from a CV shared in a chatbot conversation. [more] [more-2]
Rapidly evolving AI threats: Bugcrowd’s Inside the Mind of a Hacker 2024 report reveals the rising role of AI in cybersecurity, with 82% of ethical hackers concerned about rapidly evolving AI threats. The report notes a significant increase in the perceived value of AI in hacking, though most hackers still believe human creativity and expertise are irreplaceable. AI has created a new attack vector, but 73% of hackers feel confident in their ability to detect vulnerabilities in AI-powered systems. Additionally, hardware hacking is gaining traction, driven by vulnerable smart devices, while hacking is emerging as a viable career path, especially among younger, self-taught individuals. [more]
Quantum threat - limited: Chinese researchers reportedly used a D-Wave quantum computer to breach RSA encryption, raising alarms about cybersecurity risks for sectors like banking and cryptocurrency. The Shanghai University study claims this is the first time a quantum computer has posed a genuine threat to major encryption algorithms. However, experts note that D-Wave’s quantum computers are specialized and not general-purpose; they can handle specific tasks but are not yet capable of breaking the larger RSA keys used in secure systems. This limitation tempers immediate concerns, though advances in quantum technology continue to prompt caution in encryption security. [more]
Web3 Cryptospace Spotlight
Radiant hack update - likely a sophosicated hardware wallet attack: In a blog explaining the attack, Radiant claimed hackers successfully compromised at least three developers’ hardware wallets. After which, the hackers used malware to “manipulate transaction data at the device level” and used “poisoned signatures” that looked legitimate to the signers authorizing the transaction. The hackers then carry out three multi-signature approvals to move crypto to wallets they controlled. [more]
$1.6M lost in phishing attack: Tapioca DAO, a decentralized finance protocol, suffered a security breach where an attacker exploited a smart contract vulnerability to steal 28 million TAP tokens, worth $1.6 million. This led to a rapid decline in TAP’s value by 96%. The hacker was able to get funds from Tapioca DAO co-founder, 0xRektora, using a phishing attack. As stated by Matt Marino, another co-founder, the attack began when 0xRektora received a message concerning a friend’s employment with another company. This interaction made him let down his guard and he connected his hardware wallet through which the hacker took control of the TAP tokens. Tapioca DAO has since involved law enforcement and security teams to trace the funds. [more]
Aviation Technology Risk
Near missed attack on Cyprus’ airport infrastructure: In Cyprus, Hermes Airports’ website faced an attempted cyberattack on Friday, but it was successfully averted. The airport operations remained unaffected, as the website is separate from critical systems. While the site was offline, online parking reservations were temporarily unavailable. Authorities noted that hacking groups like LulzSec Black and Moroccan Soldiers are likely involve in targeting Cypriot infrastructure. [more]
DEF CON 32
AppSec List
Full list