Practical overview
When it fits
Consider OpenPipe when you have request history or compatible training data and want to evaluate whether a smaller fine-tuned model can handle a specific task. Its documented local setup is relevant when you want to develop against a self-hosted instance.
Who it is for
- Developers working with OpenAI-style chat-completion workflows who want to evaluate task-specific fine-tuned models.
- Teams with LLM request history or OpenAI-compatible JSONL datasets to use in model evaluation and fine-tuning.
- Developers who need to run the platform locally and can configure PostgreSQL, Node.js, pnpm and GitHub OAuth.
Documented uses
- Query logged LLM requests using filters.
- Import OpenAI-compatible JSONL data and prepare it by pruning duplicated text such as system prompts.
- Fine-tune supported models using task-specific request data.
- Compare model outputs against past requests or a base model.
- Host fine-tuned models on-platform or download their weights.
Requirements & limits
- The repository documents Python and TypeScript SDKs and an OpenAI-compatible chat-completions endpoint.
- Documented data workflows include request-log filtering, OpenAI-compatible JSONL import and pruning duplicated text.
- The README lists GPT-3.5, Mistral and Llama models and says fine-tuned weights can be hosted on-platform or downloaded.
- Local setup requires PostgreSQL, Node.js 20, pnpm, an OpenAI API key and GitHub OAuth configuration.
- The README says development on the open-source version is temporarily stopped while proprietary third-party code is integrated.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up OpenPipe for local development
Prepare and start a local OpenPipe instance using the repository's documented setup sequence.
Before you start
- Node.js 20
- pnpm
- PostgreSQL
- An OpenAI API key
- A GitHub OAuth app with a callback URL for the local instance
Steps
Clone the repository and install its dependencies with pnpm.
Copy app/.env.example to app/.env and enter the OpenAI API key.
Set DATABASE_URL to the PostgreSQL instance and run pnpm prisma migrate dev in the app directory.
Set GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET using the GitHub OAuth app, then run pnpm dev in the app directory.
Expected outcome
The README directs users to open the local application at http://localhost:3000.
Guide limitations
- The README says development on the open-source version is temporarily stopped while proprietary third-party code is integrated.
- The documented local setup requires GitHub OAuth configuration.
Send and log a chat completion to a local instance
Use the documented Python example to submit a chat completion to a local OpenPipe instance and log the request.
Before you start
- A running local OpenPipe instance
- The OpenPipe Python package
- An OpenAI API key and OpenPipe API key
Steps
Create an OpenAI client configured with the OpenAI key, OpenPipe key and local base URL http://localhost:3000/api/v1.
Call client.chat.completions.create with a model and messages.
Set openpipe.log_request to True and provide a prompt_id tag in the request's OpenPipe options.
Expected outcome
The example submits a chat completion and requests that the local instance log it with the supplied prompt_id tag.
Guide limitations
- The README provides an example request, not a complete guide to fine-tuning or serving a resulting model.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 14 October 2026
- Original website title
- OpenPipe/OpenPipe · Official homepage title · retained in its source language

