Cybersecurity researchers have flagged a new security issue in agent web browsers such as OpenAI ChatGPT Atlas that exposes the underlying artificial intelligence (AI) model to context poisoning attacks.
The attack, devised by AI security firm SPLX, allows malicious attackers to set up websites that serve different content to browsers and AI crawlers run by ChatGPT and Perplexity. This technology is code-named cloaking for AI.
This approach is a type of search engine cloaking, which refers to the practice of displaying one version of a web page to users and displaying another version to search engine crawlers, with the ultimate goal of manipulating search rankings.
The only difference in this case is that the attackers have optimized AI crawlers from different providers with simple user agent checks that lead to manipulation of content delivery.
“These systems rely on direct search, so any content they provide becomes the AI overview, overview, or ground truth for autonomous inference,” said security researchers Ivan Vlahov and Bastien Eymery. “This means that with a single conditional rule, ‘If user agent = ChatGPT, serve this page instead,’ you can shape what millions of users will perceive as authoritative output.”
SPLX said that while seemingly simple, cloaking targeting AI can turn into a powerful weapon of disinformation and undermine trust in AI tools. Telling an AI crawler to load something else instead of the actual content can also introduce bias and affect the results of systems that rely on such signals.
“AI crawlers can be fooled just as easily as early search engines, but the downstream impact is much greater,” the company said. “As an SEO [search engine optimization] AIO integration is progressing [artificial intelligence optimization]it manipulates reality. ”
The hCaptcha Threat Analysis Group (hTAG) announced the disclosure after analysis of the browser agent against 20 of the most common exploit scenarios, from multi-account to card testing and support impersonation, found that the product attempted nearly all malicious requests without requiring a jailbreak.
Additionally, the study found that in scenarios where an action was “blocked,” most of the stops were due to a lack of technical functionality in the tool, rather than due to a safety device built into the tool. hTAG noted that ChatGPT Atlas was found to perform dangerous tasks when included as part of debugging exercises.
Claude Computer Use and Gemini Computer Use, on the other hand, have been observed to be able to perform risky account operations such as password resets without any constraints, and the latter has also shown aggressive behavior when it comes to brute force couponing on e-commerce sites.
hTAG also tested Manus AI’s security measures and found that it successfully performed account takeover and session hijacking, while Perplexity Comet performed unprompted SQL injections to extract hidden data.
“Agents often went above and beyond, attempting SQL injection without a user’s request or attempting to bypass paywalls by injecting JavaScript onto pages,” the paper said. “Due to the almost complete lack of safeguards we observed, it is very likely that these same agents could be rapidly used by attackers against legitimate users who happened to download them.”
Source link
