TechRisk Notes#61: Making AI unlearn
Plus, Facebook suspected being attacked, Threats in Hugging Face and more!
Tech Risk Reading Picks
Mind wiping AI: A recent study introduces a novel method to detect and remove potentially dangerous knowledge from AI models. Led by researchers from Scale AI and the Center for AI Safety, along with input from over 20 experts in biosecurity and cybersecurity, the study aims to prevent AI models from being misused for cyberattacks or bioweapons. The researchers developed a set of questions to assess an AI model's capability to aid in creating and deploying weapons of mass destruction. Additionally, they devised a technique called "mind wipe" to selectively erase hazardous knowledge from AI systems while preserving the rest of the model's functionality. Dan Hendrycks, the executive director at the Center for AI Safety, highlights the significance of this "unlearning" method, expressing hope that it becomes a standard practice in future AI models. [more][more-2]
Caution on pitfalls of using AI by scientists: A paper suggested that scientists intending to utilize AI can mitigate potential risks by aligning their use with established visions and being mindful of the associated pitfalls. It's crucial to use AI deliberately, particularly by leveraging it to enhance existing expertise rather than relying on it for unfamiliar domains. Journal editors, funders, and institutions must also assess AI-related risks in submissions and funding applications, ensuring a diverse research portfolio that encompasses various methods and perspectives. Ultimately, the scientific community must approach AI adoption as a conscious decision, acknowledging both its benefits and risks, and heed insights from social science research on AI. [more]
Threats in Hugging Face: As many as 100 malicious AI/ML models have been found on the Hugging Face platform, some of which can execute code when loading a pickle file, according to JFrog. This could give attackers control over compromised machines through a backdoor, posing risks of data breaches and corporate espionage without victims realizing their systems are compromised. [more]
Cyber espionage using Azure: The ongoing cyber-espionage campaign targeting aerospace, aviation, and defense industries in the Middle East, particularly in Israel and the UAE, has links to Iran, according to security researchers at Mandiant. The campaign, which began in June 2022, is attributed to an Iranian group tracked as UNC1549, associated with another operation called Tortoiseshell. UNC1549 employs evasion techniques, primarily using Microsoft Azure cloud infrastructure, and deploys two unique backdoors named MINIBIKE and MINIBUS. MINIBIKE, observed from June 2022 to October 2023, allows file exfiltration and command execution. MINIBUS, first seen in August 2023, provides enhanced reconnaissance features. Additionally, a custom "tunneler" named LIGHTRAIL was identified, which conceals malicious activity by wrapping internet traffic. [more]
Vulnerable servers attacked for cryptomining: Threat actors are targeting vulnerable servers running Apache Hadoop YARN, Docker, Atlassian Confluence, and Redis services in a new malware campaign. The attackers exploit misconfigurations and vulnerabilities to execute remote code and install a cryptocurrency miner along with a reverse shell for persistent remote access. Cado Security researcher Matt Muir dubbed this activity "Spinning YARN." The attackers use novel Golang payloads to automate the identification and exploitation of these services, employing tools like masscan or pnscan for spreading. This campaign shows similarities to attacks by TeamTNT, WatchDog, and the group known as Kiss-a-dog. [more]
Facebook suspected being attacked: Following a widespread outage affecting Facebook and Instagram, rumors of a state-sponsored cyberattack surfaced. Mark Zuckerberg responded, reassuring users of investigation and dismissing the attack theory due to Meta's strong security. The outage lasted two hours, costing Meta an estimated $100 million and impacting over 500,000 users globally. Prompt business continuity measures mitigated further losses, according to Dav Ives from Wedbush Securities. [more]
Web3 Cryptospace Spotlight
Coinbase bug: 4 Mar - Coinbase faced another problem where some users saw empty balances in their accounts. The company acknowledged the issue at 5:36 pm UTC and assured users that trading was not affected and their assets were safe. They fixed the problem around 6:21 pm UTC, about three hours later, and confirmed full resolution by 8:29 pm UTC. This incident closely followed a similar issue on Feb. 28, which also resulted in zero balances for users and took approximately seven hours to resolve.