Microsoft has unveiled two new open-source tools called RAMPART and Clarity to assist developers in better testing the security of artificial intelligence (AI) agents.
RAMPART, short for Risk Assessment and Measurement Platform for Agentic Red Teaming, functions as a Pytest-native safety and security testing framework for writing and running safety and security tests for AI agents, covering both adversarial and benign issues, as well as various harm categories.
Users can write test cases to attack or probe an AI agent to explore possible safety violations like cross-prompt injections, where untrusted data reaches an AI system indirectly via a data source (e.g., email, file, or a web page) processed by it, or unintended behavioral regressions and data exfiltration.
RAMPART then evaluates the outcome of those tests and reports the results. All it needs is an adapter that connects an agent to the test suite. The tool builds on PyRIT (short for Python Risk Identification Tool), which Microsoft released more than two years ago as a way to test AI systems.
Clarity, on the other hand, has been described by the tech giant as a “structured sounding board” to help developers arrive at the right approach even before writing a single line of code. It’s an “AI thinking partner that pushes back,” guiding them through problem clarification, solution exploration, failure analysis, and decision tracking.
Leave a Reply