Microsoft Open-Sources RAMPART and Clarity to Secure AI Agents During Development

"RAMPART, short for Risk Assessment and Measurement Platform for Agentic Red Teaming, functions as a Pytest-native safety and security testing framework for writing and running safety and security tests for AI agents, covering both adversarial and benign issues, as well as various harm categories. Users can write test cases to attack or probe an AI agent to explore possible safety violations like cross-prompt injections, where untrusted data reaches an AI system indirectly via a data source (e.g., email, file, or a web page) processed by it, or unintended behavioral regressions and data exfiltration."

"RAMPART then evaluates the outcome of those tests and reports the results. All it needs is an adapter that connects an agent to the test suite. The tool builds on PyRIT (short for Python Risk Identification Tool), which Microsoft released more than two years ago as a way to test AI systems."

"Clarity, on the other hand, has been described by the tech giant as a "structured sounding board" to help developers arrive at the right approach even before writing a single line of code. It's an "AI thinking partner that pushes back," guiding them through problem clarification, solution exploration, failure analysis, and decision tracking."

"In publicly releasing these tools, Microsoft said the idea is to address why certain decisions are incorporated at an early stage of software development so that any potential issue - for example, an agent's access to a tool - is addressed well before the system is built. "We wanted to give product managers and engineers a way to pressure-test their assumptions at the start of a project, when changing course is cheap and the right conversation can save months of rework," Ram Shankar Siva Kumar, a Data Cowboy and founder of Microsoft's AI Red Team, said"

RAMPART is a Pytest-native safety and security testing framework for AI agents. It supports writing and running tests covering adversarial and benign issues and multiple harm categories. Test cases can probe agents for safety violations such as cross-prompt injections, where untrusted data reaches an AI system indirectly through sources like email, files, or web pages. Tests can also reveal unintended behavioral regressions and data exfiltration. RAMPART evaluates test outcomes and reports results, requiring only an adapter connecting an agent to the test suite. Clarity is a structured sounding board that acts as an AI thinking partner. It guides problem clarification, solution exploration, failure analysis, and decision tracking before code is written. Both tools aim to address potential issues early, when changes are inexpensive.

Read at The Hacker News