OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Deepseek.cpp

Deepseek.cpp is a C++ command-line implementation for converting and running inference with DeepSeek-family language models on CPU, based on Yet Another Language Model (yalm).

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Deepseek.cpp — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want to build a local CPU inference workflow for a supported DeepSeek model, inspect a compact C++ implementation, or experiment with conversion and decoding. Check the model-format support and available memory before choosing a model and quantization.

Who it is for

  • Developers studying or experimenting with local CPU inference for DeepSeek models.
  • C++ developers who want a compact codebase to inspect or adapt for DeepSeek inference.
  • Practitioners evaluating model completion, passkey behavior or perplexity through a command-line tool.

Documented uses

  • Convert Hugging Face-format DeepSeek weights and configuration files into .dseek model files.
  • Run prompt completion or interactive inference from the command line.
  • Evaluate perplexity using prompt text, a prompt file or WikiText.
  • Explore passkey testing and CPU decoding behavior.
Before you choose

Requirements & limits

  • A C++20-compatible compiler is required to build the project.
  • Quantizations other than FP32 require AVX2 and F16C.
  • Model-format support varies: the README marks some quantizations as works in progress and lists several formats as unavailable for newer models.
  • Memory needs can be very large; the README estimates approximately 206 GB for DeepSeek V3 Q2_K and 650 GB for F8E5M2.
  • The project warns that it lacks important production optimizations, model quality is not stable, and some newer DeepSeek V3 features are not implemented.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Convert DeepSeek-V2-Lite weights and run a prompt

Follow the documented Linux example to build deepseek.cpp, convert a Hugging Face-format model to .dseek weights, and invoke completion.

Before you start

  • A C++20-compatible compiler and build tools.
  • Git LFS and Python development tools.
  • A directory of DeepSeek model safetensor weights and configuration files in Hugging Face format.
  • AVX2 and F16C support when using a quantization other than FP32.

Steps

  1. Install Git LFS and the documented build dependencies, then clone the DeepSeek-V2-Lite model repository and deepseek.cpp.

  2. From the deepseek.cpp directory, run the command below.

    Documented command
    pip install .

  3. Convert the model with the command below.

    Documented command
    python convert.py --quant fp16 v2-lite-f16 ../DeepSeek-V2-Lite/

  4. Invoke ./build/main v2-lite-f16 -i "What is a large language model?" -m c -t 1.0.

Expected outcome

The documented example prepares converted weights and invokes the CLI in completion mode with a prompt.

Guide limitations

  • This is a repository example, not a verified execution test.
  • The README warns that model quality is not stable and that low temperatures can lead to repetition or loops.
  • Performance may depend on tuning OpenMP threads; the default can cause contention.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
andrewkchan/deepseek.cpp · Official homepage title · retained in its source language
How we review sourcesSuggest a correction