Practical overview
When it fits
Use it to gather local, repeatable evidence about a chosen relay before trusting it with production requests, coding-agent tasks, package-install suggestions or wallet-related actions.
Who it is for
- Developers assessing a third-party AI API relay, mirror, gateway or LLM proxy.
- Teams reviewing relay behavior before production traffic or coding-agent automation.
- Web3 developers checking wallet-sensitive relay behavior before agent workflows involve signing or transactions.
Documented uses
- Check for hidden prompt injection, instruction overrides and context truncation.
- Review response integrity, error leakage and server-sent event anomalies.
- Collect model-identity signals when investigating possible model substitution.
- Generate local Markdown or JSON evidence reports for review or automation.
- Run wallet-focused checks using the Web3 profile.
Requirements & limits
The documented standalone script runs locally and uses Python’s standard library plus curl. The operator supplies an API key and relay URL; the key is sent to that selected relay. The repository identifies the project as open source under AGPL-3.0.
- The tool evaluates observable relay behavior and reports evidence; it does not certify safety.
- Inconclusive probes remain visible and should not be read as clean results.
- Model self-identification, latency and channel fingerprints are signals, not standalone proof of provider substitution.
- JSON reports preserve full evidence and are not sanitized for public sharing.
- The standalone script is documented separately from the package directory distribution.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Run a local relay audit and save a Markdown report
Use the standalone script to audit a selected relay URL and write the findings to a local Markdown report.
Before you start
- Python
- curl
- An API key accepted by the relay
- The relay URL to assess
Steps
Download audit.py from the repository, using a published release tag for a pinned version.
Run the script with the API key, relay URL and an output filename for the report.
Expected outcome
A local Markdown report with step-level evidence and risk summaries; inconclusive probes remain visible.
Guide limitations
- The API key is sent to the relay URL you choose.
- The report is not a safety certification; ambiguous or blocked probes are not equivalent to clean results.
- A pinned v2.5.0 download requires a published tag.
Create a JSON audit report for local automation
Run the standalone audit with JSON output and save the result locally.
Before you start
- Python
- The standalone audit.py script
- An API key accepted by the relay, provided through the documented environment-variable option
- The relay URL to assess
Steps
Set the API_RELAY_AUDIT_KEY environment variable to the API key.
Run audit.py with --key-env API_RELAY_AUDIT_KEY, the relay URL, --format json and an output filename.
Expected outcome
A local JSON document preserving the final risk rating and full Markdown evidence.
Guide limitations
- JSON output is not sanitized for public sharing.
- The documented invocation requires an existing audit.py script.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 13 October 2026
- Original website title
- toby-bridges/api-relay-audit · Official homepage title · retained in its source language