AI agents have evolved from question-answering chatbots into systems that access email, retrieve CRM records, execute code, and take actions across many connected services. This shift from generating text to acting in the world introduces different risks, and many development teams lack tools to manage them continuously. Microsoft open-sourced RAMPART and Clarity to make AI safety an ongoing engineering practice. RAMPART is a testing framework that integrates red teaming into development workflows using pytest-style tests, CI execution, and adapters to orchestrate agent interactions and evaluate outcomes. It emphasizes cross-prompt injection attacks involving indirect manipulation through retrieved documents or communications, and it supports statistical trials to handle probabilistic LLM behavior.
"RAMPART is a testing framework that brings red teaming techniques directly into the development workflow. It's built on top of PyRIT, Microsoft's existing automation framework for red teaming generative AI systems. Where PyRIT is optimized for black-box discovery by security researchers after the system is built, RAMPART is built for engineers as the system is being built."
"Teams write standard pytest tests that describe scenarios pulled from their threat model. Each test connects to the agent through a lightweight adapter, orchestrates an interaction, and evaluates observable outcomes - passing or failing like any other test in a CI pipeline. When a new tool or data source is added to the agent, the corresponding safety test can be added in the same pull request."
"One of RAMPART's stronger features is its focus on cross-prompt injection attacks - scenarios in which an agent retrieves or processes content from documents, emails, tickets, or other data sources that indirectly manipulate its behavior. It's one of the more persistent attack surfaces in agentic AI, and RAMPART is designed to catch it early."
"Because LLM behavior is probabilistic, RAMPART also supports statistical trials. The same test can be run multiple times with policies such as "this actio"
Read at DevOps.com
Unable to calculate read time
Collection
[
|
...
]