OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Rwkv.cpp

Rwkv.cpp is a C++ library that ports RWKV language-model inference to ggml. It supports FP32, FP16 and quantized INT4, INT5 and INT8 formats, focuses on CPU execution, and also documents cuBLAS and hipBLAS support. The project provides a C library and a Python wrapper.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Rwkv.cpp — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Rwkv.cpp when you need a library for integrating RWKV v4–v7 inference into C++ or Python software, particularly for CPU-focused workloads or when evaluating quantized formats.

Who it is for

  • C++ developers embedding RWKV inference in applications.
  • Python developers using the provided wrapper to add RWKV inference to their software.
  • Teams building CPU-focused RWKV inference tools and comparing model-format trade-offs.

Documented uses

  • Run supported RWKV models through a C or Python integration.
  • Choose among full-precision and quantized inference formats.
  • Build the library for a target operating system and CPU, or enable a documented GPU backend.
  • Merge LoRA checkpoints in Blealtan’s format using the supplied script.
Before you choose

Requirements & limits

The project is publicly available on GitHub under the MIT license.

  • The supported formats include FP32, FP16 and quantized INT4, INT5 and INT8; the README advises measuring perplexity and latency on a representative dataset.
  • The project is CPU-focused; GPU backends accelerate matrix multiplication, while some operations still require CPU resources.
  • Building from source requires CMake, and the documented model-acquisition stage requires Python 3.x with PyTorch.
  • Precompiled library downloads are described for Windows, Linux and macOS, with packages selected by operating system and CPU.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build the library on Linux or macOS

Compile the Rwkv.cpp library from source for Linux or macOS.

Before you start

  • Git
  • CMake

Steps

  1. Clone the repository with its submodules using the command below.

    Documented command
    git clone --recursive https://github.com/saharNooby/rwkv.cpp.git

  2. Enter the repository directory with cd rwkv.cpp.

  3. Run cmake ..

  4. Run cmake --build . --config Release.

Expected outcome

The README says a successful build produces `librwkv.so` on Linux or `librwkv.dylib` on macOS.

Guide limitations

  • This guide covers the standard build; cuBLAS requires a separate documented build option.
  • Anaconda and M1 users are advised to check that CMake detects arm64 as the system processor.
Documentation 1 ↗
Source-based practical guide

Build with cuBLAS on Linux or macOS

Compile the library with the documented cuBLAS build option.

Before you start

  • Git
  • CMake

Steps

  1. Clone the repository recursively and enter its directory.

  2. Run cmake . -DRWKV_CUBLAS=ON.

  3. Run cmake --build . --config Release.

Expected outcome

The README says a successful build produces `librwkv.so` on Linux or `librwkv.dylib` on macOS.

Guide limitations

  • The README notes that cuBLAS supports ggml_mul_mat(); remaining operations still use some CPU resources.
  • This recipe does not cover the separate Windows cuBLAS instructions.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Rwkv.cppRun supported RWKV models through a C or Python integration.CLI, Linux, macOS, ServerThe project is publicly available on GitHub under the MIT license.
Cloud AI SDKAdapt and run documented model recipes for computer-vision and NLP inference.Cloud, ServerThe GitHub repository is public. Its public visibility does not establish license terms.
TurboquantReplace a model’s existing inference cache with a compressed cache.Cloud, Linux, macOS, Self-hostedThe project describes itself as open source. The README documents installation through pip or from a source checkout.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
RWKV/rwkv.cpp · Official homepage title · retained in its source language
How we review sourcesSuggest a correction