TechRisk Notes#46: Unreliable LLMs
Plus, Taipei crypto trading firm breached, Elon Musk pointed out on AI's potential risks, OpenAI saga aftermath, and more!
Tech Risk Reading Picks
AI risk: Elon Musk urged OpenAI to clarify Sam Altman's dismissal, highlighting AI's potential risks. [more]
Unreliable LLMs: In a new paper, researchers from the Oxford Internet Institute argue that Large Language Models (LLMs) like ChatGPT and Bard could pose a threat to science due to potential false responses. LLMs, which power AI chatbots, are designed to sound human-like and can generate text that appears accurate. However, the researchers caution that users may trust these models too much, leading to acceptance of inaccurate or biased information. The problem lies in the datasets used to train LLMs, which may contain false statements, opinions, or creative writing from the internet. Additionally, the secretive nature of LLM datasets raises concerns about transparency. The researchers highlight the risk of subtle inaccuracies in responses, especially in fields requiring specific expertise, such as references to scientific articles. They argue for restrictions on the use of LLMs in scientific research. [more]
OpenAI Saga Aftermath: There are mixed reactions to OpenAI CEO Sam Altman's approach to AI risks. Altman's acknowledgment of existential risks associated with advanced AI brought fringe ideas into the mainstream, but his strategy of pursuing advanced AI while admitting uncertainty raised concerns. Altman's removal by three independent board members with connections to effective altruism led to tensions, with some supporters in Silicon Valley vilifying the decision, creating unease even among those in the effective altruism community who previously supported Altman. [more]
Web3 Cryptospace Spotlight
$26M drained due to API keys: Taipei-based crypto trading firm Kronos Research acknowledged a security breach resulting in a $26 million hack from unauthorized access to its API keys. [more]
November surged in crypto crimes: Blockchain security firm CertiK noted that crypto hacks surged in November, resulting in losses of around $173 million, marking the fourth-highest monthly total in 2023. The largest was a theft from Poloniex, a cryptocurrency exchange, where hackers breached the hot wallet and made an unauthorized withdrawal. This incident also ranks as the second-largest global private key compromise in 2022. [more]
Near missed: Following the October 17 hot wallet hack where Fantom Foundation experienced the theft of 1% of its funds, a security researcher subsequently identified an additional risk related to a dormant admin token for Fantom's ERC-20 FTM contract. This vulnerability could have allowed the attacker to mint a portion of Fantom tokens on Ethereum, potentially leading to a $170 million loss. The foundation quickly addressed the vulnerability and awarded the researcher $1.7 million for their contribution.
Market manipulation: In a recent event, the decentralized exchange dYdX faced a 43% drop in the Yearn.Finance (YFI) token and used $9 million from its insurance fund to cover user liquidations. Founder Antonio Juliano believes this was a targeted attack and possible market manipulation. This incident has prompted a thorough review of risk parameters by dYdX.
Private key compromised: The bridge linking HTX exchange and Ethereum was hacked, resulting in the theft of around $87 million in various cryptocurrencies, including ether, tether (USDT), and wrapped bitcoin (HBTC). Blockchain security firms, such as CertiK, Peckshield, and Cyvers, examined the suspicious transactions and suggested that the bridge's private key was probably compromised.