Integrated into many daily workflows, AI platforms have become an attractive vector for attackers seeking to compromise end users. According to Huntress Security Operations Center (SOC), the practical threat landscape centers on adversaries misusing established AI capabilities that organizations and individuals already depend on, rather than direct assaults against the underlying AI infrastructure or vendors themselves.
Throughout the preceding nine months, Huntress documented multiple attack campaigns in which malicious actors leveraged shareable AI-generated content, public mini-applications, and paid search results to reach AI platform users and install malicious payloads.
Legitimate features, hijacked
Huntress identified several legitimate AI platform capabilities that attackers have repurposed for malicious purposes:
- Claude Artifacts: generated content displayed within a chat interface that users can make public and distribute through shareable links
- claude.ai/share links: URLs that emerge when conversations are published and become discoverable by search engines when shared on indexable platforms such as forums or social networks
- ChatGPT and Grok conversations: publicly accessible, searchable discussions available at chatgpt.com and grok.com that can rank prominently in search results for common troubleshooting queries
Each mechanism operates within a trust perimeter where users recognize the platform name, visual identity, and contextual elements, making malicious payloads or instructions appear credible. While these operations typically persist only hours or days before platform operators intervene, the timeframe proves sufficient to compromise victims before removal occurs.
FakeAgent: malvertising through a Claude Artifact
During July, Huntress tracked a campaign designated FakeAgent that affected more than 29 organizations. The attack originated from a malicious Claude Artifact residing on the legitimate claude.ai domain.
Since public Artifacts function as simple demonstrations subject to minimal review from Anthropic beyond standard disclaimers, attackers constructed a convincing replica of the Claude Desktop installation page. Users performing Bing searches for the Claude desktop application encountered the fraudulent page and selected what appeared to be an official download button. The click redirected them to an external server hosting SectopRAT malware.
Huntress notified Anthropic of the malicious Artifact, which was taken down on July 22. However, related incidents involving the same redirect infrastructure persisted through August.
A fake install guide hiding in claude.ai/share
In another incident, a user searching Google for "Claude on Mac" selected a sponsored link that pointed to a claude.ai/share URL masquerading as an Apple Support installation guide. Because the page operated on Anthropic's infrastructure, it lacked typical warning indicators: no domain spoofing, no security certificate errors.
The fraudulent guide directed the victim to execute a curl command in Terminal, initiating a six-step infection sequence that installed the MacSync stealer. The malware extracted browser cookies, stored passwords, keychain data, Telegram account sessions, and SSH and cloud authentication keys.
AI poisoning via ChatGPT and Grok
A third attack pattern focuses on poisoning AI-generated support content itself. In December, a search for "clear disk space on macOS" returned highly-ranked ChatGPT and Grok discussions containing ClickFix-style instructions rather than legitimate solutions.
Attackers had authored these conversations, used the share function to generate public URLs on the platforms' trusted domains, and applied SEO manipulation to elevate the links in Google rankings. Because the URLs pointed to genuine chatgpt.com and grok.com addresses, victims accepted the guidance and executed the recommended Terminal commands, which deployed the AMOS stealer.
What defenders should do
None of these campaigns circumvented AI platform security mechanisms. Instead, they exploited the confidence users place in recognizable brands and authentic domain names.
Security teams should classify clipboard-based command execution and AI-powered troubleshooting guidance as potential risks. Organizations should disable script execution sourced from clipboard operations and implement application whitelisting. Monitor for newly created scheduled tasks and modifications to antivirus configurations, and conduct security awareness training focused on identifying ClickFix-style social engineering. Promptly escalate suspicious content hosted on AI platforms to the respective service provider.
Although these campaigns typically have short lifespans, rapid reporting combined with defense-in-depth strategies can minimize the window available for attackers to succeed.
Source: BleepingComputer