Practical overview
When it fits
Use Hub when an application or service needs a self-hosted gateway between its code and supported LLM providers, particularly when centralized routing and call observability are useful.
Who it is for
- Teams building applications that need a common API across supported LLM providers.
- Operators who want a central layer for routing and observing LLM calls.
- Developers configuring gateway providers, models and pipelines.
Documented uses
- Route chat-completion, text-completion and embedding requests through one gateway.
- Configure provider and model routing in a YAML file or through PostgreSQL-backed management.
- Collect tracing and metrics for LLM gateway traffic.
Requirements & limits
Hub is open-source software available from a public GitHub repository under the Apache-2.0 license; the repository documents Docker, Cargo and Helm deployment options.
- YAML mode uses static configuration, needs no external database and exposes only the gateway port.
- Database mode requires PostgreSQL and migrations, and adds a management API on port 8080 alongside the gateway on port 3000.
- Provider support documented in the repository includes OpenAI, Anthropic, Azure OpenAI, Google Vertex AI and AWS Bedrock.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Configure Hub in YAML mode
Set up a static provider, model and chat pipeline configuration for the gateway.
Before you start
- Hub container image or a built Hub binary
- A config.yaml file with provider credentials and model configuration
Steps
Create config.yaml with a provider entry, including the provider type and required credentials.
Add a model that references the provider, then define a chat pipeline and its model router.
Run the Docker image with the configuration mounted at /app/config.yaml and port 3000 published.
Expected outcome
Hub starts in YAML mode with static configuration and the gateway listening on port 3000.
Guide limitations
- YAML mode has no management API and does not provide hot reload.
- The example documents a chat pipeline; it does not show a request being sent.
Prepare database mode
Set up PostgreSQL-backed configuration and enable Hub's management API.
Before you start
- A PostgreSQL database
- sqlx-cli installed with PostgreSQL support
- Hub source or container deployment
Steps
Set up the PostgreSQL database.
Run the database migrations with sqlx migrate run.
Configure HUB_MODE=database and DATABASE_URL with the database connection string.
Start Hub in database mode; the documented gateway and management ports are 3000 and 8080.
Expected outcome
Hub is configured for database mode, which includes its REST management API and supports configuration hot reload.
Guide limitations
- This setup recipe does not document management API requests or a gateway call.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 8 October 2026
- Original website title
- traceloop/hub · Official homepage title · retained in its source language