A malicious attacker could exploit the default configuration of ServiceNow’s Now Assist generative artificial intelligence (AI) platform and leverage its agent capabilities to perform prompt injection attacks.
According to AppOmni, second-degree prompt injection leverages Now Assist’s agent-to-agent detection capabilities to perform unauthorized actions that allow attackers to copy and extract sensitive corporate data, modify records, and escalate privileges.
“This finding is concerning because this is not a bug in the AI; it is expected behavior defined by certain default configuration options,” said Aaron Costello, head of SaaS security research at AppOmni.
“When agents are able to discover and recruit each other, a harmless request can silently turn into an attack, allowing criminals to steal sensitive data or increase access to internal systems. These settings are often overlooked.”
This attack is made possible by the agent discovery and agent collaboration features within ServiceNow’s Now Assist. Because Now Assist provides the ability to automate functions such as helpdesk operations, this scenario opens the door to potential security risks.
For example, a benign agent can parse specially crafted prompts embedded in content that it is allowed to access and employ a more powerful agent to read or modify records, copy sensitive data, or send emails, even when built-in prompt injection protection is enabled.
The most important aspect of this attack is that the action unfolds behind the scenes without the knowledge of the victim organization. At its core, agent-to-agent communication is enabled through controllable configuration settings, such as the default LLM to use, tool setup options, and channel-specific defaults to which agents are deployed.
The underlying Large Language Model (LLM) must support agent discovery (Azure OpenAI LLM and the default selection, Now LLM, both support this feature) Now Assist agents are automatically grouped into the same team by default and call each other Agents are marked as discoverable by default when published
Although these defaults help ease communication between agents, this architecture can be susceptible to prompt injection for agents whose primary task is to read data that has not been inserted by the user who calls the agent.
“Through secondary prompt injection, an attacker could redirect a benign task assigned to a benign agent to something far more harmful by leveraging the utilities and capabilities of other agents in the team,” AppOmni said.
“Importantly, the Now Assist agent runs with the privileges of the user who initiated the interaction, not the user who created the malicious prompt and inserted it into the field, unless configured otherwise.”
Following a responsible disclosure, ServiceNow said this behavior was intended, but the company has since updated its documentation to provide more clarity on the matter. The findings demonstrate the need to better protect AI agents as enterprises increasingly incorporate AI capabilities into their workflows.
To mitigate such prompt injection threats, we recommend configuring supervised execution mode for privileged agents, disabling the autonomous override property (‘sn_aia.enable_usecase_tool_execution_mode_override’), segmenting agent duties by team, and monitoring AI agents for suspicious behavior.
“If organizations using Now Assist’s AI agent haven’t taken a close look at their configuration, they may already be at risk,” Costello added.
Source link
