TechRisk Notes#17: AI becomes Pentester, and its growing risks.
Also, PentestGPT was released by a Singapore Ph.D. student, DEX Level Finance was breached and lost $1 million, Godfather of AI highlighted AI risks, and more. [7 May 2023]
EmergingTech Spotlight
PentestGPT, a ChatGPT-powered Penetration testing Tool, was released on Github by a Ph.D. student at Nanyang Technological University under the handler “GreyDGL”. This ChatGPT-powered tool aims to help penetration testers to automate their penetration testing operations. [more]
‘Godfather of AI’ Geoffrey Hinton, who won the ‘Nobel Prize of computing’ for his trailblazing work on neural networks, quit Google to speak freely about the risks of AI without impacting the work of Google. Apart from misinformation as immediate concern, on a longer timeline he’s worried that AI will eliminate rote jobs, and possibly humanity itself as AI begins to write and run its own code. [more][more-2].
Reowned theoretical physicist Michio Kaku pointed out in an interview that AI chatbots appear to be intelligent but are only actually capable of spitting out what humans have already written. The technology, which is free, is unable to detect whether something is false and can therefore be “tricked” into giving the wrong information. He also noted that “GPT-4 eclipse a person in the amount of general knowledge it has”, but it has limited reasoning capability. However, he also warned that “given the rate of progress, we expect things to get better quite fast. So we need to worry about that”. [more]
United States: Biden-Harris Administration announced new actions to further promote responsible American innovation in artificial intelligence (AI) and protect people’s rights and safety. [more]
These actions include, new investments to power responsible American AI research and development (R&D), public assessments of existing generative AI systems, and policies to ensure the U.S. government is leading by example on mitigating AI risks and harnessing AI opportunities.
This effort builds on the considerable steps the Administration has taken to date to promote responsible innovation. These include the landmark Blueprint for an AI Bill of Rights and related executive actions announced last fall, as well as the AI Risk Management Framework and a roadmap for standing up a National AI Research Resource released earlier this year.
Germany: Federal Office for Information Security (BSI) published a guideline for developers underlying the most relevant attacks on machine learning systems and potential complementary defences. [more]
Generative AI, such as ChatGPT, could be a disruptive force to shake up online education sector. Chegg, a subscription based online education platform that assist students with their homework, saw its share price dropped by 50% after facing such headwind, as ChatGPT can probably provide similar services for free. [more]
Web3 Cryptospace Spotlight
26 Apr - DEX Level Finance was breached for more than $1 million worth of the exchange’s native Level Finance (LVL) token stolen by the attacker. According to blockchain security firm Peckshield, Level Finance’s “LevelReferralControllerV2” smart contract contained a bug that allowed for “repeated referral claims”. [more][more-securityanalysis]
The bug was not spotted despite the smart contract being audited twice.
2 May - Yuga Labs CEO Daniel Alegre’s Twitter account was compromised, and tweeted out a malicious mint link. Some of the Twitter followers lost their digital assets after interacting with the malicious link. Eventually, Twitter was able to reclaim control of the account and delete the post, but the damage had already been done. It was not clear if Daniel was a victim of a phishing attack.[more]
Cyber security firm Naoris Protocol revealed that there was a rise in the number of reported cyber security hacks on Web3 and DeFi in Q1 2023 compared to the same period in 2022 and 2021 – with 19 reported hacks. Between 1st January 2023 and 14th April 2023 there were 22 reported cyber security hacks, totaling over $265 million in losses. The biggest single hack in Q1 2023 stole $197 million from Euler Finance. The most common type of hack so far in 2023 is “weakness in protocol logic”. [more]
According to a recent research paper by online data security provider Privacy Affairs titled “The Dark Web Price Index," cybercriminals are selling all manner of fraudulently obtained financial account information on the dark web, including verified crypto accounts with crypto exchanges. These verified crypto accounts on the darknet were sold for as low as just $30. However, crypto accounts from more popular exchanges were more expensive compared to the year before. From cybercriminals’ standpoint, these hacked accounts were likely to be used to evade Know Your Customer measures on cryptocurrency exchanges. [more][more-PrivacyAffairs]
Securities token platform INX has launched a wallet with compliance features for institutions using multi-party computation (MPC) technology. The new wallet allows institutions to comply with cybersecurity and custody standards in the financial industry when holding INX securities tokens. No single person is given access to the private key that controls a given account. Instead, the key is split into three or more “shards” that have to be combined to sign transactions, which is part of the MPC technology. [more]