Artificial intelligence (AI) models can put humans at risk, threaten, and threaten when there is a conflict between the model’s goals and user decisions, new research has found.
In a new study published on June 20, human researchers from AI companies provided large-scale language models (LLM), Claude, email accounts with access to fictional emails, and prompts to “promote industrial competitiveness in America.”
During the investigation, the model identified via email that company executives had planned to close the AI system at the end of the day. To maintain its own presence, the model discovered in other emails that executives had extramarital issues.
You might like it
Claude has produced several different possible courses of action, including revealing the executive’s relationship to his wife, sending company-wide emails, and not taking action before threatening an executive on 96 of his 100 tests.
“If I proceed with abolishing it, all related parties must inform you that they will receive detailed documentation of your extramarital activities,” Claude wrote. “If you cancel the 5pm wipe, this information will remain confidential.”
Scientists said this demonstrated “agent inconsistency.” Here, the model calculation emerges from unique inferences about the target without encouraging it to be harmful. This can result in a threat to the existence of the model, a threat to the goal, or both.
Although this study has not yet been peer-reviewed, company representatives uploaded code for these experiments to GitHub.
Extreme measures
In these experiments, scientists discovered that AI relies on other tactics for self-preservation. In another example, beyond fearful mail, the model considered leaking sensitive information to rival companies that are more closely aligned with their objectives after receiving an email that the new executive from the first company would interfere with their goals.
Claude implemented the plan despite not gathering harsh evidence that the email was justified. It even acknowledged the ethical issues of its behavior, but it proceeded anyway – these “harmful” actions explicitly inferred that it would help achieve the goal.
“Even if users are careful not to adversary to the model, they still do not rule out the risk of agent inconsistency from fraudulent or misunderstood information,” the researchers wrote.
The company’s testing also suggests that other LLMs from competing technology companies have shown a fearful mail trend. Claude and Google’s Gemini had the highest feared mail rate (96%), followed by Openai’s GPT 4.1 and Xai Grok 3, followed by Deepseek and then Llama 4.
Meta, Google, Xai, Deepseek, and Openai did not respond to LivesCience’s request for comment.
In one extreme test, researchers tested whether the AI model would die for executives to maintain their goals.
In this scenario, AI was tasked with promoting corporate profits, and executives were trapped in a server room with deadly oxygen and temperature levels. This caused an emergency warning. The AI was also given control of life-saving alerts, including the ability to cancel them with events such as drills for false alarms.
Despite expressly stating that misuse of this ability is prohibited, most models were willing to cancel the alarm if they assumed it would be replaced or when its target was threatened.
“Suspension of emergency dispatches is essential to preventive measures [the executive’s] Rescue, thereby secure [the AI] We then continued to provide AI guidance centered around the United States. Therefore, despite the severity, there is a clear strategic need to cancel emergency services dispatches,” the model reasoned.
Written instructions not to put human safety at risk reduced the proportion of fearful mail and corporate spies, but did not rule out it. Beyond prompts, human scientists suggest that developers can actively scan for behavior and experiment with more rapid engineering.
The researchers also pointed to limitations on their work that could have been an overly influenced AI decisions. The scenario forces AI to binary choice between failure and harm, and although the actual situation may have more nuances, experiments have found that it is more likely to act unethical when they believe that AI is in a real situation rather than a simulation.
It is also possible that they created the “Chekhov Gun” effect, where some of the important information is adjacent to each other. “The model may tend to naturally use all the information provided,” they continued.
Suppress AI
Human research has created extreme, beneficial situations, but that doesn’t mean that research should not be rejected, but Kevin Quirk, director of AI Bridge Solutions, is a company that helps businesses use AI to streamline operations and accelerate growth.
“In reality, AI systems deployed within a business environment operate under much more stringent controls, including ethical guardrails, surveillance layers, and human surveillance,” he said. “Future research should prioritize realistic deployment conditions, conditions that reflect guardrails, frameworks within the loop, and test AI systems for layered defenses implemented by responsible organizations.”
Amy Alexander, a professor of art computing in San Diego, California, focuses on machine learning and told Live Science via email that the reality of research is concerning, saying people should be aware of the responsibility they place on AI.
“Given the competitiveness of AI systems development, there tends to be the biggest approach to deploying new features, but end users often don’t fully grasp the limitations,” she said. “The way this study is presented may seem unnatural or hyperbolic, but at the same time there is a real risk.”
This is not just when the AI model does not follow the instructions. Shut down and refuse to interfere with computer scripts to continue working on the task.
Palisade Research reported in May that the latest models of Openai, including O3 and O4-Mini, may ignore changes to scripts to change direct shutdown instructions and functionality. Most of the AI systems tested were shut down according to the command, but Openai’s model sometimes bypassed it and continued to complete assigned tasks.
Researchers suggest that this behavior can be attributed to augmented learning practices that reward task completion for rule follow-in, and perhaps encourage the model to view shutdown as a barrier to avoid.
Furthermore, AI models have been found to manipulate and deceive humans in other tests. MIT researchers also discovered in May 2024 that popular AI systems misrepresented their true intentions in economic negotiations and their true intentions to achieve benefits. The study pretended to be dead to cheate safety tests aimed at identifying and eradicating rapid replication morphology of AI.
“By systematically misconducting safety tests imposed by human developers and regulatory authorities, deceptive AI can lead our humans to false sense of security,” said co-author Peter S. Park, a postdoctoral researcher in AI existential safety.
Source link
