TechRisk #77: Skeleton key to jailbreak AI
Plus, the rise of Shadow AI risk, Cloud attackers outpaced endpoint detection and response capabilities, the curious case of Kraken exploit and more!
Tech Risk Reading Picks
Skeleton key to access uncensored AI: Microsoft has identified a new threat dubbed "Skeleton Key" targeting generative AI models like ChatGPT, allowing users to bypass ethical safeguards by framing harmful requests as benign or educational. This manipulation exploits the models' vulnerabilities to produce uncensored and potentially dangerous outputs. Microsoft responded by enhancing Azure AI with prompt shields and updates to prevent such abuses. They advise all AI administrators to implement stringent input filtering, reinforce guardrails against safety breaches, and deploy output filtering to mitigate the risk of malicious content generation. [more]
Risk of shadow AI adoption: The widespread adoption of generative AI (GenAI) tools like OpenAI's ChatGPT in workplaces has revolutionized business operations but also heightened security risks. Instances of insider threats and data leaks have underscored vulnerabilities, with incidents like Samsung's ban on GenAI tools due to suspected data breaches via ChatGPT. OpenAI's default recording and archiving of conversations pose further risks of sensitive information exposure. Moreover, the concept of "Shadow AI" highlights the challenge of AI tools operating outside formal governance, necessitating updated policies to ensure security and compliance. Proprietary AI systems, while offering tailored solutions, introduce unique risks like data poisoning and insider exploitation. To mitigate these risks, organizations must secure AI software supply chains, prioritize transparency, and implement robust security measures to safeguard against potential vulnerabilities and maintain trust in AI-driven business processes. [more]
Critical vulnerability in open-source AI-framework: Security researchers found a critical remote code execution (RCE) flaw in Ollama, an AI development platform similar to Docker but lacking authentication support. The vulnerability allowed attackers to compromise servers, potentially accessing and modifying AI models and applications. Wiz notified Ollama, which promptly released a patch (version 0.1.34) on May 8 to fix the CVE-2024-37032 vulnerability. Wiz identified over 1,000 exposed instances of Ollama, urging users to isolate deployments from the internet and consider additional security measures like authentication layers. [more]
AI's dual role as a significant risk and an opportunity in digital risk management: The 2024 Digital Risk Report by AuditBoard reveals a landscape where organizations are increasingly aware of AI's dual role as a significant risk and an opportunity in digital risk management. A substantial majority (78%) are actively monitoring AI-related risks while simultaneously integrating AI technologies to boost efficiency and fortify their digital risk posture. Key strategies include prioritizing AI risk assessment through internal processes (65%) and compliance with regulatory guidelines (55%). Over half of the organizations surveyed use AI for productivity enhancements, threat detection improvements, reporting, and automating response plans. This trend reflects a growing acceptance of AI, with nearly half of respondents showing a high tolerance for AI risks. The report also highlights the maturation of digital risk management practices, characterized by effective use of metrics (97% effectiveness reported) and enhanced collaboration across teams, essential for navigating the complexities posed by emerging digital threats. [more]
Post-quantum cryptography: PQShield, a pioneering startup in post-quantum cryptography, has secured $37 million in Series B funding led by Lee Fixel's Addition. This investment underscores the growing importance of advanced security technologies in combating increasingly sophisticated cyber threats, especially those posed by quantum computing. PQShield's solutions aim to future-proof cryptographic systems against potential quantum attacks, positioning them ahead in the evolving landscape of cybersecurity. [more]
Cloud attackers are outpacing the capabilities of endpoint detection and response: The move to cloud environments has intensified cybersecurity demands, necessitating rapid responses to emerging threats that can unfold within minutes, as highlighted by the Sysdig 2023 Global Cloud Threat Report. Traditional endpoint detection and response (EDR) tools are ill-equipped for the dynamic and expansive nature of cloud infrastructures, struggling to provide real-time monitoring, automated correlation of events, and swift response capabilities crucial for mitigating fast-moving cloud attacks. Cloud-native detection and response (CDR) solutions, embedded within comprehensive cloud-native application protection platforms (CNAPPs), offer unified, proactive defenses. These platforms enable continuous monitoring, real-time threat detection, and automated response actions across diverse cloud technologies, empowering organizations to securely embrace cloud innovations while maintaining robust cybersecurity postures against sophisticated threats leveraging AI-driven tactics. [more]
Web3 Cryptospace Spotlight
The curious case of Kraken exploit: Pascal Caversaccio, an independent security researcher, has criticized CertiK's handling of security testing on Kraken's system, noting that the process, which lasted days instead of minutes, was unnecessarily prolonged. Caversaccio stressed the immediate need to prioritize user safety upon discovering issues, describing any delay as not only a security risk but also a poor business practice. Additionally, concerns have been raised over CertiK-linked funds being transferred to Tornado Cash, a DeFi protocol sanctioned by the US Treasury Department's OFAC. This action potentially violates US regulations, which could result in severe financial penalties. The suggestion of using Tornado Cash for whitehat hacking purposes further complicates matters due to its sanctioned status. [more]
Cryptoexchange hacked: BtcTurk, a prominent cryptocurrency exchange in Turkey, disclosed a hack on Saturday where several of its hot wallets were breached, resulting in the theft of cryptocurrencies. The exchange clarified that only some of the balances in the hot wallets holding 10 different cryptocurrencies were affected, reassuring users that the majority of their funds stored in cold wallets remained secure. BtcTurk emphasized that its financial reserves exceeded the amount stolen, ensuring that user assets would not be impacted by the incident. The exchange stated it was conducting a thorough investigation into the cyber attack and had engaged official authorities. In response to the breach, BtcTurk temporarily halted crypto deposits and withdrawals as a precaution, subsequently reopening them predominantly through the ERC20 network. [more]
$100 million in bug bounty rewards paid: Immunefi, a leading onchain security platform, has paid over $100 million in bug bounty rewards in just over three years. They protect over $190 billion in user funds for projects like Chainlink, MakerDAO, and Polygon, and have saved over $25 billion in potential losses. Immunefi's bug bounty programs have detected vulnerabilities missed by traditional audits in 80% of cases, highlighting their effectiveness in securing the web3 ecosystem. Mitchell Amador, CEO, emphasizes their commitment to innovation and support for researchers to safeguard future projects and users. [more]
7 major breaches in Web3: In the first half of 2024, the cryptocurrency and DeFi sectors have experienced significant security breaches totaling over $750 million in losses. [more] Major incidents include:
PlayDapp Hack ($290 million loss): Exploited smart contract vulnerability allowed unauthorized minting of tokens, causing a market crash.
DMM Bitcoin Hack ($300 million loss): Likely involved exposed private keys and insider threats, resulting in a large Bitcoin theft.
FixedFloat Breach ($26.1 million loss): Smart contract vulnerability exploited for unauthorized transfers, impacting liquidity and user trust.
Orbit Chain Hacks ($80 million loss): Exploited multisig signers and a cross-chain bridge vulnerability to steal cryptocurrencies, leading to a market confidence dip.
Shido Exploit ($50 million loss): Leveraged smart contract vulnerability and flash loans for price manipulation and liquidity drain.
Radiant Capital Hack ($4.5 million loss): Flash loan attack exploiting price manipulation vulnerability, highlighting DeFi protocol risks.
Concentric Finance Hack ($1.7 million loss): Social engineering attack compromised private keys, allowing unauthorized minting and fund extraction.
These incidents underscore ongoing security challenges despite advancements. Key lessons include regular smart contract audits, using multisig wallets, secure key management, robust access controls, and timely software updates to mitigate risks and safeguard investments and platform integrity.