Practical overview
When it fits
Use KServe when your team already operates Kubernetes and needs a documented route to deploy models as inference services, including an LLM endpoint that OpenAI-compatible clients can call.
Who it is for
- Teams operating Kubernetes clusters that need to serve machine-learning models.
- Platform engineers supporting generative AI and predictive model workloads.
- Application developers integrating with a deployed LLM through an OpenAI-compatible API.
Documented uses
- Deploy predictive and generative models as Kubernetes inference services.
- Serve an LLM through an OpenAI-compatible chat-completion endpoint.
- Choose standard or Knative installation modes and deploy KServe components through documented installation methods.
- Use a shared serving layer across generative AI and traditional predictive workloads.
Requirements & limits
The project describes itself as open-source. The documented quickstart requires Kubernetes 1.32 or higher and is intended for experimentation.
- A Kubernetes cluster is central to the service; the quickstart lists Kubernetes 1.32 or higher as a requirement.
- Installation choices include standard and Knative modes, and components can be selected for KServe, LLMInferenceService and LocalModel.
- The documented LLM example uses the Hugging Face runtime with a vLLM backend and requests an NVIDIA GPU.
- The tutorial offers load-balancer, node-port and port-forwarding ingress paths, depending on cluster setup.
- The quickstart is not presented as a production installation; the documentation points production users to the Administrator's Guide.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up KServe for experimentation
Install a KServe quickstart environment on a Kubernetes cluster.
Before you start
- kubectl, Helm and Git
- A running Kubernetes cluster, version 1.32 or higher
- A configured kubeconfig; Docker is needed when creating a local Kind cluster
Steps
Verify kubectl, Helm and Git are installed using their version commands.
Start a local cluster with Kind or Minikube, or connect to an existing cluster and check its version and current context.
Clone the KServe repository and choose a documented installation path, such as the standard or Knative mode installation script.
Follow the quickstart's next-step tutorial to deploy a model.
Expected outcome
A KServe quickstart environment ready for experimentation; the next documented stage is deploying a predictive model or LLM.
Guide limitations
- The quickstart environment is for experimentation only; production installation is covered by the Administrator's Guide.
- The guide documents multiple installation scenarios; the appropriate dependencies vary by scenario.
Deploy and query the tutorial Qwen LLM
Create a Qwen InferenceService and send a chat-completion request through KServe's OpenAI-compatible endpoint.
Before you start
- A working KServe installation and Kubernetes cluster
- kubectl access to create resources and inspect services
- The tutorial's Hugging Face runtime and vLLM backend; its example requests an NVIDIA GPU
- Ingress access through a load balancer, node port or port forwarding
Steps
Create the tutorial namespace and apply an InferenceService manifest using the Hugging Face model format and the Qwen model URI.
Check the InferenceService status and determine the ingress host and port using the documented cluster access option.
Create the tutorial chat-input.json request body and send it to the /openai/v1/chat/completions endpoint using curl or the OpenAI Python client.
Delete the InferenceService and tutorial namespace when finished.
Expected outcome
A deployed Qwen service that accepts chat-completion requests through an OpenAI-compatible API endpoint.
Guide limitations
- This recipe follows the tutorial's Qwen example and infrastructure assumptions; it is not a general deployment guarantee.
- The tutorial cautions against deploying InferenceServices in control-plane namespaces.
- If Hugging Face authentication is needed, the tutorial requires creating a token secret and a ClusterStorageContainer reference.
Compare options for Model inference and serving
These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.
| Project | Documented purpose | Platform | Access and requirements |
|---|---|---|---|
| kserve.github.io | Deploy predictive and generative models as Kubernetes inference services. | Cloud, Docker, Linux, Self-hosted | The project describes itself as open-source. The documented quickstart requires Kubernetes 1.32 or higher and is intended for experimentation. |
| Morphingdb | Import a libtorch model and create predictions from values in a PostgreSQL table. | Docker, Linux, Self-hosted, Server | The README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension. |
| Gateway — pyadav | Send chat-completion requests to a provider through a common server endpoint. | Server | The repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker. |
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 12 October 2026
- Original website title
- KServe · Official homepage title · retained in its source language