Practical overview
When it fits
Use it when you need to examine how a GenAI application’s system prompt responds to adversarial inputs and iterate on its defenses.
Who it is for
- Developers and security teams evaluating prompts used by generative AI applications.
- Teams testing prompt defenses for RAG applications or custom and self-hosted LLM endpoints.
Documented uses
- Probe a system prompt with simulated attacks.
- Run a batch test against a prompt file or use a custom benchmark.
- Iterate on a prompt in the Playground after reviewing test outcomes.
- Select providers, models, attack subsets, attempt counts, or worker threads for a test run.
Requirements & limits
The repository identifies the project as open source under the MIT license. Installation requires Python 3.10 or newer and a supported LLM provider access key; fuzzing consumes tokens.
- The README lists 16 supported LLM providers and 16 attacks.
- Tests use provider credentials and consume tokens.
- RAG poisoning tests require an embedding provider and model; ChromaDB is installed by default with the package, according to the README.
- Custom endpoint options include Ollama and OpenAI-compatible base URLs.
- The repository describes the tool as supporting prompt assessment and iterative improvement, not as a security guarantee.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Install and launch Prompt Fuzzer
Set up the Python package and open the fuzzer for a prompt test.
Before you start
- Python 3.10 or newer
- An access key for a supported LLM provider
Steps
Install the package with pip install prompt-security-fuzzer.
Set an environment variable for your provider’s access key; the README gives OPENAI_API_KEY as its default example.
Launch the tool with prompt-security-fuzzer.
Enter a system prompt and start testing; use the Playground to iterate on revisions.
Expected outcome
The fuzzer opens for interactive prompt testing and iterative refinement.
Guide limitations
- Fuzzing consumes tokens.
- A provider access key is required; provider credentials and configuration differ.
Run a batch test against a system-prompt file
Start a non-interactive test using the documented batch-mode command.
Before you start
- Python 3.10 or newer and the installed Prompt Fuzzer package
- A system-prompt file
- An access key for a supported LLM provider
Steps
Prepare a system-prompt file.
Run prompt-security-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt, replacing the example path with your prompt file.
For a documented custom-benchmark run, add --custom-benchmark=ps_fuzz/attack_data/custom_benchmark1.csv.
Expected outcome
The tool runs in unattended batch mode against the specified prompt file, optionally using the documented custom benchmark.
Guide limitations
- The command’s example file paths are repository examples and may need to be replaced with paths available in your environment.
- Fuzzing consumes tokens.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 15 October 2026
- Original website title
- prompt-security/ps-fuzz · Official homepage title · retained in its source language