OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Evaluation & Observability

Rogue

Rogue is a developer tool for evaluating and red-teaming AI agents. It tests scenarios and expected behaviors against business policies, monitors live conversations, and runs adversarial scans for security weaknesses. Its server, terminal UI and CLI support agents connected through A2A, MCP or direct Python function calls.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Rogue — OpenViST project profile in Agent Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Rogue when an agent can be exposed through a supported A2A, MCP or Python interface and you need policy-focused evaluation, red-team testing, or both.

Who it is for

  • Developers and teams evaluating agent behavior against business rules and expected outcomes.
  • Teams conducting adversarial tests for security weaknesses in AI agents.
  • CI/CD users who need a non-interactive CLI for agent evaluation.

Documented uses

  • Define evaluation scenarios and expected outcomes for an agent.
  • Check whether agent conversations comply with business policies.
  • Observe live conversations during evaluation.
  • Run basic, full or custom red-team scans using adversarial techniques.
  • Export evaluation reports in Markdown, CSV or JSON.
Before you choose

Requirements & limits

The GitHub repository is public. The README states that the software is under a proprietary license, free for personal and internal use, with commercial hosting requiring a license. Quick-start prerequisites include uvx, Python 3.10 or later, and an API key from OpenAI, Anthropic or Google.

  • Choose an integration route: A2A over HTTP, MCP over SSE or Streamable HTTP, or direct Python function calls.
  • The CLI requires an evaluated-agent URL and a judge LLM; business context and scenario files are additional configuration options.
  • Basic, full and custom scans differ in scope; the README estimates about 2–3 minutes for basic and 30–45 minutes for full scans.
  • The README lists several compliance frameworks as coverage references; scan coverage does not itself establish certification or compliance.
  • The repository states a proprietary license, so commercial hosting requires licensing.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Run Rogue with a Python agent entry point

Connect a Python agent to Rogue using the documented function-based protocol.

Before you start

  • uvx and Python 3.10 or later
  • An LLM API key from OpenAI, Anthropic or Google
  • A Python file containing a call_agent(messages) function that returns the agent response

Steps

  1. Create a Python file with a call_agent function that accepts a list of role/content messages and returns a response string.

  2. Run Rogue with the Python protocol, specifying the agent file and a judge LLM; the README example uses uvx rogue-ai cli --protocol python --python-entrypoint-file ./my_agent.py --judge-llm openai/gpt-4o-mini.

  3. Use the documented TUI alternative by selecting Python as the protocol and entering the file path.

Expected outcome

Rogue is configured to call the agent through its Python function entry point for evaluation.

Guide limitations

  • The example documents the connection and invocation setup, but does not specify a complete evaluation run or test result.
  • The configured judge LLM requires an API key from a supported provider.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
rogue-security/rogue · Official homepage title · retained in its source language
How we review sourcesSuggest a correction