Practical overview
When it fits
Use Rwkv Runner when you want to start an RWKV model and access it through its built-in interface or an OpenAI API-compatible client.
Who it is for
- People who want a desktop workflow for starting and configuring RWKV models.
- RWKV users who want to connect an OpenAI API-compatible chat client.
- Operators who want to separate the client interface from the Python inference backend.
Documented uses
- Install dependencies and start an RWKV model through a desktop tool.
- Connect an OpenAI-compatible client to a running RWKV model.
- Manage model downloads, conversion and inference settings.
- Use the documented Python backend separately from the desktop client.
Requirements & limits
The repository lists installation references for Windows, macOS and Linux and specifies the MIT license.
- The application exposes an OpenAI API-compatible interface; the README directs users to set the API URL and API key in client settings.
- The README describes a server-backend option where Runner is used as a client by entering the server address under API URL.
- The default configuration enables a custom CUDA kernel; the README suggests disabling it or updating the GPU driver if output is garbled.
- The README says the WebGPU strategy can be selected for AMD, Intel and other graphics cards.
- One-click LoRA fine-tuning is identified as Windows-only.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Start the RWKV-Runner inference backend from a clone
Launch the backend inference service using the repository’s simple deployment example.
Before you start
- A cloned RWKV-Runner repository.
- Python available to run the documented backend script.
Steps
Clone the repository with the command below.
git clone https://github.com/josStorer/RWKV-RunnerChange into the repository directory with
cd RWKV-Runner.Start the inference backend with the command below.
python ./backend-python/main.pyTo load a model, request the
/switch-modelAPI; consult the API documentation athttp://127.0.0.1:8000/docs.
Expected outcome
The backend inference service is started, and the README identifies the API documentation and model-loading request.
Guide limitations
- The guide documents starting the backend and requesting model loading, but does not provide the
/switch-modelrequest format. - For public services, the README advises limiting request size and max_tokens to control resource use.
Build and start the separate frontend service
Compile the frontend and start its service separately from the inference backend.
Before you start
- A cloned RWKV-Runner repository.
- Python and npm available to run the documented commands.
Steps
Clone the repository with the command below, then enter it with
cd RWKV-Runner.git clone https://github.com/josStorer/RWKV-RunnerEnter the frontend directory with
cd RWKV-Runner/frontend.Install frontend dependencies using the command below, then compile the frontend using the command below.
npm cinpm run buildReturn to the repository root with
cd ..and start the frontend service with the command below.python ./backend-python/webui_server.py
Expected outcome
The frontend is compiled and its service is started separately.
Guide limitations
- This procedure covers frontend build and service startup; it does not document starting the inference backend alongside it.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 7 October 2026
- Original website title
- josStorer/RWKV-Runner · Official homepage title · retained in its source language