OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Developer Tools & SDKs

Angelslim

AngelSlim is a Python toolkit for compressing large AI models, with documented quantization workflows for language and diffusion models. It also includes a speculative-decoding training path powered by the separate AngelSpec framework.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Angelslim — OpenViST project profile in AI Developer Tools & SDKs.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use AngelSlim when a model-development workflow calls for documented compression or quantization, followed by inference, evaluation or API serving. Choose the specific configuration and backend based on the model and deployment workflow.

Who it is for

  • AI developers and model teams working on model quantization or compression.
  • Practitioners who want to evaluate or serve quantized model weights using documented inference frameworks.

Documented uses

  • Apply post-training quantization to a supported model configuration.
  • Quantize a diffusion model and run the documented quantization-inference workflow.
  • Load saved quantized weights for offline inference or launch an API service with vLLM or SGLang.
  • Explore speculative-decoding draft-model training through the AngelSpec submodule.
Before you choose

Requirements & limits

The project is presented as a public GitHub repository with documentation. The documented quick-start workflows use Python and command-line scripts.

  • The quick-start examples are configuration-specific; the Qwen3-1.7B example uses static FP8 PTQ calibration.
  • Serving requires a saved quantized model and a separate inference framework; the repository documents vLLM and SGLang.
  • The speculative-decoding training example requires four GPUs on one node, split between inference and training, plus the separately installed Mooncake transfer engine.
  • The repository documents evaluation with lm-evaluation-harness, recommended at version lm-eval>=0.4.8.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Serve saved quantized weights with vLLM or SGLang

Launch an OpenAI-compatible API service using a quantized model path.

Before you start

  • A saved quantized model and its path as MODEL_PATH
  • vLLM at the documented recommended version vllm>=0.8.5.post1, or SGLang at sglang>=0.4.6.post1
  • For MoE INT8 with vLLM, the documented requirement is vllm>=0.9.0

Steps

  1. Choose vLLM or SGLang as the inference framework.

  2. For vLLM, run the command below.

    Documented command
    bash scripts/deploy/run_vllm.sh --model-path $MODEL_PATH --port 8080 -d 0,1,2,3 -t 4 -p 1 -g 0.8 --max-model-len 4096

  3. For SGLang, run the command below.

    Documented command
    bash scripts/deploy/run_sglang.sh --model-path $MODEL_PATH --port 8080 -d 0,1,2,3 -t 4 -g 0.8

  4. Send requests using the documented OpenAI API format.

Expected outcome

The selected inference framework launches an API service for the specified quantized model.

Guide limitations

  • The examples specify device and parallelism parameters; adapt them only according to the documented framework and deployment setup.
  • The repository also documents offline inference as a separate option.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
15 October 2026
Original website title
Tencent/AngelSlim · Official homepage title · retained in its source language
How we review sourcesSuggest a correction