Microsoft announced two new open source tools, RAMPART and Clarity, to help developers better test the security of their artificial intelligence (AI) agents.
RAMPART, which stands for Risk Assessment and Measurement Platform for Agentic Red Teaming, serves as a Pytest-native safety and security testing framework for creating and running safety and security tests for AI agents, covering both adversarial and benign problems, and various harm categories.
Users can create test cases to attack or investigate AI agents to investigate possible safety breaches, such as cross-prompt injection, where untrusted data reaches the AI system indirectly through data sources processed by the AI system (e.g., emails, files, web pages, etc.), unintended behavior regressions, and data leaks.
RAMPART then evaluates the results of those tests and reports the results. All you need is an adapter to connect your agent to your test suite. The tool is built on PyRIT (short for Python Risk Identification Tool), which Microsoft released more than two years ago as a way to test AI systems.
Clarity, on the other hand, is described by the tech giant as a “structured sounding board” that helps developers arrive at the right approach before they write a single line of code. It is a “push back AI thinking partner” that guides them through problem articulation, solution exploration, failure analysis, and decision tracking.
In releasing these tools, Microsoft said it aims to address why certain decisions are built in early in software development, so that potential problems (for example, agents accessing tools) can be addressed long before the system is built.
“We wanted to give people a way to pressure test their assumptions at the beginning of a project, where changing course is cheap and having the right conversations can save months of rework,” Ram Shankar Siva Kumar, data cowboy and founder of Microsoft’s AI Red Team, said in a blog shared with The Hacker News.
Microsoft said a secondary motivation for investing in these tools is to extend learnings from red teaming exercises by making incidents reproducible and mitigations verifiable, turning them into actionable engineering assets.
“PyRIT is optimized for black box discovery by security researchers after the system is built, whereas RAMPART is built for engineers during the system build,” Siva Kumar added. “Clarity helps teams clarify design intent and understand assumptions. Together, these approaches move AI safety from a one-time review to a set of living artifacts that developers can use throughout the lifecycle.”
Source link
