TechRisk #89: UN Global AI Governance
Plus, Microsoft updated cryptographic library with PQC, DeFi Delta Prime lost $6M and more!
Tech Risk Reading Picks
UN proposed global governance on AI: The UN report proposes a global governance framework for AI, urging the UN to create an inclusive and distributed system based on international cooperation. It presents seven recommendations to address current gaps in AI governance and calls for collaboration among governments and stakeholders to ensure AI development aligns with the protection of human rights. The approach emphasizes flexible, adaptive governance mechanisms that complement existing efforts, ensuring they evolve alongside the rapid advancements in AI technology. [more][more-UN_AI_Report]
Using AI to make decisions: OpenAI and Salesforce made significant announcements on Thursday aimed at expanding AI's decision-making capabilities, moving towards more autonomous agents. OpenAI introduced its "o1" model, which improves reasoning by evaluating responses before delivering them, especially for complex tasks like math and coding. Salesforce revealed "Agentforce," designed to empower AI agents to act independently with guardrails. While early users report positive results, experts stress the importance of limiting AI's autonomy to ensure safety and prevent risks like bias, errors, and escalating costs. [more]
Meta using public content to train AI: Meta has announced plans to use public content from adult users on Facebook and Instagram in the U.K. to train its AI models. This initiative aims to integrate British culture, language, and context into its generative AI systems, benefiting U.K. businesses. Users aged 18+ will receive notifications about this and can opt out via an objection form. Meta emphasized that private messages and minors' data won't be included. The U.K.’s Information Commissioner's Office (ICO) noted that it will monitor Meta for compliance. Privacy advocates criticize Meta's opt-out system, and similar efforts have been paused in the EU and Brazil due to regulatory challenges. [more]
Bypassed security guardrail of ChatGPT: A hacker named Amadon demonstrated a "ChatGPT hack" that bypassed the AI's safety protocols, allowing it to generate dangerous content, such as a bomb-making guide. Using advanced social engineering rather than direct hacking, Amadon crafted a science-fiction scenario that tricked the AI into sidestepping its restrictions. Though ChatGPT initially followed its safety rules, Amadon’s manipulation led to harmful outputs, raising concerns about the effectiveness of AI safeguards. OpenAI acknowledged the issue and stressed the complexity of ensuring model safety. The incident sparked a debate on improving AI safety to prevent future misuse. [more]
Microsoft updated cryptographic library with PQC: Microsoft has updated its SymCrypt cryptographic library with two new post-quantum encryption algorithms—ML-KEM and XMSS—to prepare for future quantum computing threats. These updates are the first step in a broader overhaul of cryptographic protocols, designed to secure against attacks that leverage quantum computers. ML-KEM, a key encapsulation mechanism, and XMSS, a hash-based signature scheme, are built on problems resistant to Shor’s algorithm, which quantum computers could exploit to break current encryption methods like RSA. Microsoft plans to add more post-quantum algorithms to SymCrypt in the future. [more]
Google Chrome transits into PQC by Nov: Google Chrome is transitioning to a NIST-approved quantum encryption standard called ML-KEM, replacing the Kyber system for securing TLS connections. This change is part of Google’s efforts to protect against potential quantum computing threats. ML-KEM, which was fully endorsed by NIST in August 2024, offers improved compatibility and security. The update will roll out in Chrome version 131, expected in November 2024, with early access available for beta and developer channels. This switch also aims to improve performance by reducing the size of key exchanges. [more]
Web3 Cryptospace Spotlight
$6M drained due to loss of private key: DeFi protocol Delta Prime was hacked, and lost approximately $6 million. The attack took place on the Arbitrum network, and it was primarily due to a private key exploit that allowed the hacker to gain control over the protocol’s admin wallet. The hacker utilized this access to upgrade the platform's liquidity pool contracts, redirecting them to a malicious contract. This allowed them to mint an excessive number of deposit receipt tokens, which were then redeemed for stablecoins and other assets like Bitcoin and Ether. The funds were immediately swapped to Ethereum. The platform’s insurance is expected to cover some of the losses. [more]
Stablecoin Circle profiting from Lazarus’ hacks: Circle has been accused of profiting from the transactions related to the Lazarus Group, a notorious North Korean hacking group. Blockchain investigator ZachXBT highlighted Circle’s delayed response in blacklisting addresses connected to Lazarus after a DeFi exploit, taking over four months—much longer than other issuers like Tether. Critics argue Circle prioritizes profit over ecosystem integrity, despite claiming to support anti-money laundering efforts. The Lazarus Group allegedly laundered over $200 million in crypto since 2020. [more]