OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Mooncake

Mooncake is an open-source, C++ LLM-serving platform built around a disaggregated architecture centered on KV-cache handling. Its repository describes it as the serving platform for Kimi.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Mooncake — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Mooncake as a project to investigate when designing LLM-serving infrastructure where KV-cache handling and disaggregation are relevant. Consult its documentation to assess implementation requirements for a specific environment.

Who it is for

  • Teams researching or building self-hosted LLM inference infrastructure.
  • Engineers assessing KV-cache-centric and disaggregated serving designs.

Documented uses

  • Explore a KV-cache-centric architecture for LLM serving.
  • Review project documentation, the associated paper and slides, and supplied workload traces.
  • Assess listed transfer-engine package options for accelerator ecosystems.
Before you choose

Requirements & limits

The project repository is public and identifies the code as Apache-2.0 licensed.

  • The core design is KV-cache-centric and disaggregated.
  • The repository identifies Mooncake as Kimi’s serving platform.
  • Transfer-engine package listings cover multiple accelerator ecosystems, but do not alone confirm deployment compatibility.
  • The supplied project description does not detail deployment requirements or supported runtimes.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Mooncake and start the Store master

Install the default Mooncake package and start its Store master service.

Before you start

  • Python 3.10 or later; a virtual environment is recommended.
  • The default pip installation targets NVIDIA CUDA 12.1–12.9. CUDA 12.1 or later is listed as a prerequisite; most CUDA-enabled use cases also require NVIDIA GPUDirect support.
  • For RDMA data transfer, an RDMA driver and SDK are required.

Steps

  1. Install Mooncake with the command below.

    Documented command
    pip install mooncake-transfer-engine

  2. Start the master service with mooncake_master.

  3. Wait for a log line indicating that the master service has started; the documented default RPC port is 50051.

Expected outcome

Mooncake is installed and the Store master service is running.

Guide limitations

  • This recipe covers setup and starting the master only; it does not configure or run an LLM serving framework.
  • The default package path targets NVIDIA CUDA. Other platform variants and requirements are listed in the Quick Start.
  • For production deployment, standalone Store services, high availability, allocation strategies, SSD offload, and runtime tuning, continue to the Store Deployment & Tuning Guide.
Documentation 1 ↗
Source-based practical guide

Run the documented single-node Store put/get example

Write and retrieve a value using the Mooncake Store Python API on a single node.

Before you start

  • Python 3.10 or later; a virtual environment is recommended.
  • Mooncake installed using the Quick Start instructions.
  • A running mooncake_master service; the example connects to 127.0.0.1:50051.

Steps

  1. Ensure the Store master is running.

  2. Create a MooncakeDistributedStore instance and call setup with the documented single-node settings: local_hostname="localhost", metadata_server="P2PHANDSHAKE", global_segment_size=512 * 1024 * 1024, local_buffer_size=128 * 1024 * 1024, protocol="tcp", rdma_devices="", and `master_server_addr="127.0.0.1:

  3. Call put("hello_key", b"Hello, Mooncake Store!"), then retrieve the value with get("hello_key") and decode it.

  4. Close the store with store.close().

Expected outcome

The documented example prints `Hello, Mooncake Store!` after retrieving the stored value.

Guide limitations

  • This is a single-node Store put/get example, not an LLM serving deployment.
  • The example uses P2PHANDSHAKE, so it does not require a separate Transfer Engine metadata service.
  • For production deployment and runtime tuning, continue to the Store Deployment & Tuning Guide.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
MooncakeExplore a KV-cache-centric architecture for LLM serving.Linux, Self-hosted, ServerThe project repository is public and identifies the code as Apache-2.0 licensed.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
7 October 2026
Original website title
kvcache-ai/Mooncake · Official homepage title · retained in its source language
How we review sourcesSuggest a correction