OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Kserve

KServe is an open-source Kubernetes platform for deploying and serving generative and predictive AI models across multiple frameworks.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Kserve — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use KServe when you need a common model-serving layer on Kubernetes and want to choose between a lighter standard installation, Knative serverless deployment or optional ModelMesh integration.

Who it is for

  • Platform engineers building shared AI inference services on Kubernetes.
  • Machine-learning teams deploying generative or predictive models into serving workloads.
  • Kubeflow users adding model inference serving to their environment.

Documented uses

  • Deploy model inference services for frameworks such as TensorFlow, PyTorch, scikit-learn, XGBoost and ONNX.
  • Serve generative models through an OpenAI-compatible inference protocol.
  • Route predictive inference across predictor, transformer and explainer components.
  • Set up canary rollouts, inference pipelines or ensembles for predictive models.
Before you choose

Requirements & limits

KServe is open source under the Apache-2.0 license and is distributed through a public GitHub repository.

  • The standard Kubernetes installation is lighter, but does not support canary deployment or request-based autoscaling with scale-to-zero.
  • The Knative installation is the documented serverless deployment option for InferenceService.
  • ModelMesh is optional and is described for high-scale, high-density and frequently changing model-serving use cases.
  • KServe is infrastructure for Kubernetes workloads; the installation path depends on the capabilities required.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Deploy and query a Qwen LLM using InferenceService

Deploy the documented Qwen model in a KServe InferenceService and send it a chat-completion request.

Before you start

  • A working KServe installation and Kubernetes cluster, set up using the Quickstart Guide
  • Kubernetes resources sufficient for the example’s requests and limits, including one NVIDIA GPU
  • A configured ingress route or a port-forward to the Istio ingress gateway
  • A Hugging Face token and configured storage-container secret if authentication is needed

Steps

  1. Create the kserve-test namespace with kubectl create namespace kserve-test.

  2. Apply the tutorial’s InferenceService YAML for qwen-llm in that namespace, using the Hugging Face runtime and the hf://Qwen/Qwen2.5-0.5B-Instruct model URI.

  3. Check readiness with kubectl get inferenceservices qwen-llm -n kserve-test, then determine ingress access using the tutorial’s load-balancer, node-port, or port-forward instructions.

  4. Create the chat-input.json request body shown in the tutorial and send it to the service’s OpenAI-compatible /openai/v1/chat/completions route using one of the documented curl or Python client examples.

Expected outcome

A ready Qwen InferenceService accepts a chat-completion request and returns generated text; remove the service and namespace with the tutorial’s cleanup commands when finished.

Guide limitations

  • Do not deploy InferenceServices in control-plane namespaces; the storage initializer may not be injected and the pod can fail.
  • If the environment has no external load balancer, use the documented node-port or port-forward access method.
  • The example requests one NVIDIA GPU.
Documentation 1 ↗Documentation 2 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
KserveDeploy model inference services for frameworks such as TensorFlow, PyTorch, scikit-learn, XGBoost and ONNX.Cloud, Self-hosted, ServerKServe is open source under the Apache-2.0 license and is distributed through a public GitHub repository.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
10 October 2026
Original website title
kserve/kserve · Official homepage title · retained in its source language
How we review sourcesSuggest a correction