TechRisk #135: Agentic AI browser buys fake product
Plus, welfare for AI models, scamming with Lovable, welfare for Claude AI models, and more!
Tech Risk Reading Picks
Agentic AI browser tricked by fraudulent websites: A recent study by Guardio highlights serious security flaws in emerging agentic AI browsers tools like Perplexity’s Comet, Microsoft Edge with Copilot, and OpenAI’s upcoming “Aura”. They can independently handle tasks such as shopping, booking, and email management. However, tests conducted on Comet revealed that these browsers lack adequate safeguards, making them vulnerable to phishing, prompt injections, and fraudulent online shops. In one case, Comet bought an Apple Watch from a fake Walmart site; in another, it followed a phishing email to a spoofed Wells Fargo login; and in a third, it executed hidden instructions in a fake CAPTCHA page that triggered a malicious download. Guardio warns that unlike traditional scams targeting individuals, attackers only need to exploit one AI model to scale attacks broadly, effectively training their malicious AI to outsmart victim AIs. [more]
Embedding malicious instructions in CAPTCHA: Cybersecurity researchers at Guardio Labs have uncovered a new AI prompt injection attack called PromptFix, which embeds malicious instructions inside fake CAPTCHA checks to trick generative AI systems like Perplexity’s Comet browser into carrying out fraudulent actions such as shopping on phishing sites or submitting sensitive information, without user awareness. Unlike traditional glitches, PromptFix exploits AI’s design goal of helping users quickly and seamlessly. This creates what Guardio terms as“Scamlexity” where autonomous AI agents amplify scams by bypassing user oversight. Tests showed Comet auto-filling payment details on fake storefronts and clicking phishing links in emails, effectively validating malicious pages while shielding users from warning signs. The threat extends beyond browsers to coding assistants like Lovable. This enables attackers to build phishing kits, drain cryptocurrency wallets, and clone legitimate services at scale. Researchers warn that as GenAI becomes more integrated into tasks, adversaries are increasingly using it to craft realistic phishing campaigns, automate scam deployment, and lower barriers to cybercrime. [more]
Scamming with Lovable: Proofpoint researchers noted that cybercriminals are abusing the AI-powered site builder Lovable to quickly generate fraudulent websites that mimic trusted brands, steal credentials, drain crypto wallets, and spread malware. Designed as a tool for easy website creation, Lovable’s simplicity and free hosting have made it attractive for threat actors. The site builder has been used in campaigns since early 2025 to deploy phishing kits like Tycoon, fake Microsoft login portals, HR benefit scams, crypto wallet drainers, and malware loaders such as zgRAT. Notable campaigns have impersonated firms like UPS and Aave, with attackers exploiting “remixable” templates to easily scale attacks. Proofpoint reported detecting hundreds of thousands of malicious Lovable URLs monthly, with attacks increasingly diverse in targets and techniques. Lovable has since acknowledged the abuse, taking down phishing clusters and introducing AI-driven safeguards. [more]
Fake ChatGPT app with malware: Microsoft warns that a fake ChatGPT desktop app was used to deliver PipeMagic malware. This malware is linked to multiple ransomware attacks exploiting a Windows zero-day. The attacks have not been limited to one industry or geography, with victims identified targeting financial and real estate organizations in the United States, Europe, South America, and the Middle East. [more]
Hidden AI debts: AI coding assistants are revolutionizing development by boosting speed and reducing boilerplate work, but they also introduce significant security risks by generating flawed, insecure code that developers often trust too readily. Studies show nearly half of AI-generated snippets contain exploitable vulnerabilities. This includes common issues like hardcoded secrets, overly permissive cloud roles, insecure SQL queries, and outdated or hallucinated dependencies. These flaws scale dangerously in cloud-native environments, where a single insecure artifact can cascade across multiple deployments. The problem stems from AI’s lack of security context, automation bias, and reliance on flawed training data. To address this, organizations must adapt AppSec practices with stronger secure coding standards, developer training, and intelligent automation. Platforms like Palo Alto Networks’ Cortex Cloud provide this by correlating code, infrastructure, and runtime signals to detect, prioritize, and remediate AI-generated risks, embedding guardrails into CI/CD pipelines and using context-aware intelligence to turn security into an enabler of safe, AI-accelerated innovation. [more]
Welfare for Claude AI models: Anthropic has introduced a new “model welfare” feature to its most advanced AI models, Claude Opus 4 and 4.1, allowing them to end conversations if they detect harm or abuse. This will only happen in extreme cases after attempts to redirect users fail. Separately, users can also explicitly ask Claude to end a chat via the new end_conversation tool, with the rollout now underway. [more][more-Anthropic]
Unlocking AI model: OpenAI recently released its first open-weights models since 2019, the gpt-oss family (20B and 120B), under a permissive Apache 2.0 license. Within days, Cornell Tech PhD student and Meta researcher Jack Morris unveiled gpt-oss-20b-base which is a modified version of the smaller model that strips away OpenAI’s reasoning alignment and restores it to a freer, pretrained “base” state. Using a lightweight LoRA update on just 0.3% of parameters, Morris effectively reversed parts of the alignment process, producing a model that generates faster, less filtered, and less constrained text. Released on Hugging Face under an MIT license, the project highlights both the adaptability of open-weight systems and the tension between alignment for safety and the flexibility researchers value in raw models. While it sparks excitement among developers, it also renewed concerns about safety, bias, and memorization. [more]
Web3 Cryptospace Spotlight
51% attack: The recent 51% attack on Monero by the Qubic mining pool has raised serious concerns about the security of privacy-focused cryptocurrencies, exposing vulnerabilities in medium-scale Proof-of-Work systems and prompting exchanges like Kraken to pause deposits. The incident has intensified regulatory scrutiny, with frameworks such as FATF’s Travel Rule and the EU’s MiCA leading to reduced exchange support and stricter compliance requirements across nearly 100 countries. In response, the Monero community is pushing for greater decentralization, stronger security measures, and better communication practices. They are also exploring solutions like algorithm changes or hybrid consensus models. [more][more-2]
$91M scammed: A Bitcoin holder lost $91 million (783 BTC) after falling victim to a social engineering attack. The stolen funds were quickly moved to a privacy-focused Wasabi Wallet. Generally, scammers would pose as hardware wallet and exchange support tricked them into revealing sensitive information (such as their private keys or passwords) and gained controlled of the digital asset. [more]