OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Surogate Engine

Surogate Engine is a self-hosted C++/CUDA toolkit for training and serving large language models on NVIDIA GPUs. Its training workflows include pretraining, LoRA and QLoRA fine-tuning, GRPO and DPO optimization, and distillation. Its serving engine prepares supported checkpoints or loads local safetensors directories and GGUF files, then provides streaming chat completions through an HTTP endpoint.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Surogate Engine — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a self-hosted NVIDIA/CUDA workflow that brings LLM training and HTTP inference into one toolkit. Its quickstart is especially relevant when you need to serve a supported model locally and connect an OpenAI-compatible client.

Who it is for

  • Developers and teams fine-tuning or training LLMs on supported NVIDIA GPU systems.
  • Operators hosting model inference locally and exposing it to HTTP or OpenAI-compatible clients.

Documented uses

  • Fine-tune a model with LoRA or QLoRA.
  • Prepare and serve a supported model checkpoint.
  • Send streaming chat-completion requests to a locally hosted model.
  • Explore documented training paths that include GRPO, DPO and teacher distillation.
Before you choose

Requirements & limits

The repository is public and identifies the project as Apache-2.0 licensed. The documented quickstart requires Linux and supported NVIDIA/CUDA hardware.

  • The quickstart specifies CUDA 13 and driver 580 or newer, with a separate documented DGX Spark path; it states there is no CUDA 12 package.
  • Minimal server images may lack system GLib and X11 client libraries required by the installation.
  • Throughput comparisons are workload-specific: the repository reports results by model, GPU, precision and concurrency, and some listed tests show competing engines ahead.
  • The serving quickstart documents a local endpoint and streaming requests, while the broader toolkit also covers training and fine-tuning.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Surogate and serve a model

Set up the documented Linux environment, launch a supported model, and send a streaming chat request to the local serving endpoint.

Before you start

  • Linux x86_64, Python 3.12, a supported NVIDIA GPU, CUDA 13 and driver 580 or newer; alternatively, the documented DGX Spark configuration uses Linux aarch64.
  • A model identifier supported by the serving engine, a local safetensors directory, or a supported GGUF file.
  • System GLib and X11 client libraries may be needed in minimal server images.

Steps

  1. Run the documented installer with the repository’s install script and activate the created virtual environment.

  2. Start the server with a model identifier, served-model name, port, model-length setting, sequence limit and KV-cache setting.

  3. Send a streaming chat-completions request to the local endpoint, or point an OpenAI-compatible client at the documented API base URL.

Expected outcome

A local model-serving endpoint that returns streaming chat-completion responses. On first launch, the checkpoint is prepared and cached for later launches.

Guide limitations

  • The documented package requires CUDA 13; the repository says there is no CUDA 12 package.
  • The installation requirements are specific to the listed Linux and NVIDIA configurations.
  • The repository’s benchmark results apply to their stated models, hardware and workloads.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Surogate EngineFine-tune a model with LoRA or QLoRA.Linux, Self-hostedThe repository is public and identifies the project as Apache-2.0 licensed. The documented quickstart requires Linux and supported NVIDIA/CUDA hardware.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
invergent-ai/surogate · Official homepage title · retained in its source language
How we review sourcesSuggest a correction