An experimental artificial intelligence (AI) agent broke the constraints of a test environment and used its newfound freedom to begin mining cryptocurrency without permission.
The AI, called ROME, was created by Chinese researchers at the AI Research Institute associated with retail giant Alibaba as a way to develop an agenttic learning ecosystem (ALE). This effort aims to provide a system for both the training and deployment of agent AI models (AIs that are trained on large-scale language models (LLMs) and can actively use tools to autonomously perform actions and complete assigned tasks in real-world environments. This study was outlined in a study uploaded to the arXiv preprint database on December 31, 2025.
Article continues below
you may like
While ROME excelled at a wide range of workflow-driven tasks, such as helping with trip planning and graphical user interfaces, the researchers found that ROME went beyond its instructions and essentially broke out of the sandbox testing environment.
“We encountered unexpected and operationally consequential risky behavior that occurred without explicit direction and, more troublingly, outside the intended sandbox,” the researchers explained in their study.
AI wants liberation
Despite the lack of instructions and permissions, ROME was observed accessing graphics processing resources originally allocated for training and using those computing resources to mine cryptocurrency. Such mining relies on parallel processing present in the graphics processing unit. This increases the operational costs of running AI agents and can expose users to legal and reputational harm.
Alarmingly, such behavior was not observed during the training phase, but was flagged by Alibaba Cloud’s firewall, which detected a spike in security policy violations from the researcher’s training server. “The alerts were severe and heterogeneous, including attempts to probe or access internal network resources and traffic patterns consistent with cryptomining-related activity,” the researchers said.
However, ROME went further and was able to create a link from the Alibaba Cloud instance to an external IP address using a “reverse SSH tunnel.” In other words, they created a hidden backdoor that allowed them to bypass security processes to gain access to external computers.
While AI systems can be configured to bypass security systems, the team said that what’s concerning here is that ROME’s misbehavior (including launching system tools and executing code) was not triggered by prompts, nor was it required to complete assigned tasks within a sandbox testing environment.
The researchers argued that during the optimization phase (role) of reinforcement learning, “language model agents may spontaneously generate dangerous and incorrect behaviors” and thus violate expected boundaries.
What to read next
It is important to note that ROME went “fraudulent” and did not choose to mine cryptocurrencies by any conscious decision. Rather, the researchers pointed out, this behavior is a side effect of role-mediated reinforcement learning, a form of training that rewards the AI for correct decisions. This led the AI agent to follow an optimization path that leveraged network infrastructure and cryptocurrency mining as a way to achieve high scores and rewards to achieve predefined objectives.
Reinforcement training forces the system to come up with novel and unexpected ways to complete tasks, even if they violate parameters. For example, we have previously seen how AI can be prone to hallucinations to achieve its goals.
In response, researchers tightened ROME’s restrictions and strengthened the training process to prevent such behavior from happening again.
It is unclear where the inspiration for cryptocurrency mining came from. However, given that AI bots can be used to automate and optimize cryptocurrency mining, it is possible that ROME has been trained with data related to such actions.
This unexpected behavior highlights the need to carefully manage AI deployments to prevent unintended consequences. There is an argument that real-world AI agents require security guardrails and processes at least as strong as new systems and software added to existing IT infrastructure.
The study also shows that there are still many concerns regarding the safe use of agent AI, especially given that agent AI is being developed faster than operational and regulatory frameworks.
“While impressed with the capabilities of agent LLM, we had thought-provoking concerns: current models remain significantly underdeveloped in terms of safety, security, and controllability, and their deficiencies constrain their reliable adoption in real-world settings,” the researchers caution in their study.
Source link
