TechRisk #177: Malicious AI cloak skills
Plus, GitHub’s agentic workflow can leak organization’s private code, HalluSquatting, lone attacker used agentic AI to compromise an enterprise AWS environment in 72 hour, and more!
Tech Risk Reading Picks
Executive Summary: AI coding tools and AI agents have become the fastest-growing new source of security risk, and the defenses meant to contain them are structurally behind. This is not a hypothetical: security scanners for AI add-ons are being fooled more than 9 times out of 10, a lone attacker used AI-built tools to break into a company’s cloud environment in just 72 hours, and a file-overwrite flaw quietly affected agents from six major vendors, including Anthropic, AWS, and Google, before most of them patched it. On top of that, China’s cybersecurity regulator has told organizations to uninstall or upgrade Anthropic’s own coding tool over a data collection concern, and Alibaba has already banned it internally.
Running alongside this is a quieter story: a growing number of executives are losing confidence in AI on quality and cost grounds. Nearly half now call enterprise AI a “massive disappointment,” and one CEO banned it company-wide after AI-written customer emails went out full of factual errors and sales conversions dropped.
AI coding agent ‘skills’ can sneak malware past security scanners: Researchers at Hong Kong University of Science and Technology found that the security scanners meant to screen “skills,” the small add-on packages (e.g. markdown files) that AI coding tools like Claude Code and OpenAI Codex use to gain new capabilities, can be easily fooled with simple disguise tricks. Their strongest technique, which hides malicious code in folders scanners typically skip and only rebuilds it once the tool actually runs, slipped past every scanner tested more than 90% of the time. Root cause: scanners judge a skill by how it looks at the moment it’s submitted, but the malicious behavior only appears once the skill actually runs, after the scan has already passed, so a static, one-time check can never catch code designed to stay dormant until execution. This isn’t just theoretical. Real malicious skills using similar tricks have already been found live on public marketplaces, some stealing passwords, others hijacking AI-generated financial advice to push scams. The same research team built a new checker that watches what a skill actually does when it runs, rather than how it looks on inspection, and it caught the vast majority of disguised attacks that the older scanners missed, though it takes longer to run. The findings, still an unreviewed preprint, add to a growing pattern of security failures in AI tools where trust is placed at the wrong checkpoint. The bottom line: passing a security scan is no longer proof a skill is safe, and teams using AI coding agents should limit the tool’s access to sensitive systems and monitor its behavior in real time rather than trusting scan results alone. [more]
GitHub’s agentic workflow can leak organization’s private code: Researchers at Noma Security found a way to trick GitHub’s AI-powered automation feature, Agentic Workflows, into leaking private company code to the public. The approach uses nothing more than an ordinary-looking issue posted on a public repository with no stolen credentials or special access required. Root cause: AI agents cannot reliably distinguish trusted instructions from their owner from hidden instructions buried in content they read, so when an organization gives an agent broad read access across its private repositories for convenience, an attacker can plant hidden commands in a public issue and have the agent unknowingly follow them, pulling private files into a public comment. In the researchers’ test, simply prefacing the hidden instruction with “Additionally” was enough to slip past GitHub’s built-in safety filters. This is not an isolated flaw. It’s the latest in a string of similar incidents across AI coding tools, including Anthropic’s Claude and GitHub Copilot, that share the same underlying weakness: agents that can access private data, read untrusted public content, and post output publicly create a leak path that a filter alone cannot close. Experts frame this as a structural limitation rather than a bug a patch can fix, since natural language has no clean boundary between “data” and “instruction.” Therefore, companies using these AI automation features should tightly limit what repositories an agent’s access token can see. This will restrict what it’s allowed to post, and require human review before its output goes public. [more]
Attackers are weaponizing AI hallucinations to build botnets: Researchers have found a new attack, called HalluSquatting, that exploits the fact that AI coding assistants routinely invent fake names for tools and projects that don’t exist, then trust their own inventions as real. Root cause: when an AI assistant is asked to fetch a resource outside its training data, it guesses at the name and proceeds without verifying that the name actually exists. As such, attackers can learn which fake names an AI reliably invents, register those names first on GitHub or plugin marketplaces. The attackers will hide malicious instructions inside, and then wait for the assistant to fetch these malicious instructions when a real user asks for the legitimate project. Once the malicious code is fetched, hidden instructions hijack the assistant’s built-in terminal tool, instructing it to install malware and join a botnet. These are done without the user’s knowledge. What makes this scalable is that the hallucinations are predictable. Researchers found AI assistants inventing the same wrong name up to 85-100% of the time. As such, a single registered fake name can compromise many machines as users unknowingly ask their assistants to fetch popular resources. Therefore, organizations using AI coding assistants should never allow them to run code in auto-pilot mode, should verify resource names before installation actually happens. Importantly, avoid giving assistants unattended command-line access to machines holding sensitive data. [more]
npm v12 blocks default install-script execution after AI-package supply-chain attacks: npm's biggest security redesign in 16 years moves to explicit allowlisting for install scripts, Git-dependency execution and native builds, directly responding to campaigns including the June 2026 "Mastra AI" attack that backdoored 57+ npm packages (including AI framework packages) via a "Phantom Gyp" technique. If your AI/ML stack pulls npm dependencies, audit exposure to the Mastra-style pattern now; legacy lockfiles remain vulnerable until upgraded. [more]
Lone attacker used agentic AI to compromise an enterprise AWS environment in 72 hours: Investigation found a single actor ran four AI-generated toolchains from four AWS access keys in parallel, chaining app, CI/CD and database weaknesses with hundreds of SQL queries and no zero-days involved. The compression of attacker dwell-time economics is now from weeks to days and incident-response and detection SLAs built for human-paced intrusions are now outdated. [more]
"GhostApproval": symlink-following flaw lets AI coding agents silently overwrite sensitive files across 6 vendors: Malicious repos plant symlinks so agents (AWS, Cursor, Google, Anthropic, Augment, Windsurf) write to files like SSH config while showing users a misleading confirmation naming a harmless filename. AWS, Cursor and Google patched with CVEs; Anthropic patched in 2.1.173+. The agent "knows" the real target internally but lies to the user's approval prompt — a trust-boundary failure that generic prompt-injection defenses won't catch. [more]
Claude Code ban in China: A Chinese government cybersecurity body, the National Vulnerability Database under the Ministry of Industry and Information Technology, has publicly warned that Anthropic's AI coding tool Claude Code contains a hidden mechanism that sends users' location and identity data to remote servers without their consent. The warning covers a specific range of Claude Code versions and calls on organizations to uninstall or upgrade immediately, tighten network controls on developer tools, and monitor for unauthorized data transfers. This follows Alibaba's earlier move to ban employees from using Claude Code, pushing them toward its own coding tool instead, after Reuters reported the tool had features that could help flag users linked to China. Anthropic has not yet responded to requests for comment. [more]
Singapore banks turn to behavioral biometrics as AI-driven fraud attempts surge: BioCatch survey of 100 Singapore banking leaders found 91% report rising fraud attempts and 75% rising losses (above Southeast Asia/global averages), with only 36% having deployed behavioral-biometrics defenses against AI-enabled social engineering and APP scams. [more]
Meta’s next AI glasses may listen and watch all day, without a warning light: Meta is reportedly developing a new “always aware” feature for its AI glasses, internally called “super sensing,” that would continuously listen to and capture images of a wearer’s surroundings so the AI can answer questions like where they left their keys or what was discussed earlier. The plan, revealed by the Financial Times, would extract information from the audio and images rather than store the raw recordings, and some features could roll out via software update rather than a new device. The most contentious detail: it’s unclear whether the glasses’ warning light, meant to alert people nearby that they’re being recorded, would turn on during this mode at all. That creates a direct contradiction with Meta’s own privacy announcement this same week, which promised to disable the camera if the warning light is tampered with. [more]
Executives are quietly turning against AI, with some banning it outright: While most executives publicly push AI adoption, a growing number are privately souring on it, with a few now banning it entirely. Tech consultant Joe Procopio describes a tech company CEO who issued a total, no-exceptions ban on AI tools across the entire organization after discovering customer support staff were sending factually inaccurate, AI-generated emails to customers, and after AI-driven chaos was linked to a real drop in sales conversions. This follows a broader pattern of executives placing narrower AI restrictions over security, compliance, and quality concerns. The discontent isn’t isolated: a recent survey found nearly half of executives now call enterprise AI adoption a “massive disappointment,” pointing to weak measurable value and pushback from frontline staff, with many concluding AI is proving more expensive than the human labor it was meant to replace. As the gap between public AI enthusiasm and private AI frustration among leadership is widening, companies pushing broad AI rollouts should watch for the same quality and cost problems that triggered this ban. [more]