OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

AI Infra Book

AI Infra Book is an open technical book about designing infrastructure for LLM training and inference. It derives system considerations from model workloads and hardware constraints, and includes Python calculation tools and chapter-organized experiments.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

AI Infra Book — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want to connect an LLM workload to the hardware, memory, communication and scheduling limits that shape its serving or training system.

Who it is for

  • Model application developers and inference-service engineers investigating latency, cost or capacity.
  • Systems, networking and chip engineers studying how infrastructure choices affect LLM workloads.
  • Researchers and students in related fields.

Documented uses

  • Estimate compute, memory and data-movement requirements for a model workload.
  • Understand how model architecture and workload stages affect infrastructure demands.
  • Study trade-offs in accelerators, runtimes, multi-device systems and datacenter networks.
  • Recalculate book examples with different model or input settings and consult related experiment records.
Before you choose

Requirements & limits

The repository is public and licensed under Apache-2.0. Online reading and PDF and EPUB editions are listed in Simplified Chinese, English, Traditional Chinese and Russian; the latter three are identified as community translations. No account requirement or price information is provided.

  • The book is an initial draft and is being revised.
  • Some Python, linear algebra and computer-systems background is recommended.
  • Static calculations need Python 3.10 or later but no GPU or model weights.
  • Experiments have individual hardware and dependency requirements; existing records can be read without the relevant hardware.
  • The repository uses Git LFS for papers and larger inputs and measurement records; it says the total is about 20 GB and defaults to skipping those files during cloning.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Estimate resources for a model workload

Use the companion calculator to inspect a model's supported configurations and estimate per-operator resource needs for a specified input.

Before you start

  • Python 3.10 or later
  • The standard library; no GPU or model weights are needed for static calculations
  • A local checkout of the public repository

Steps

  1. Clone the repository and change into its directory.

  2. Run the command below to view model support.

    Documented command
    python3 calculations/calc.py models

  3. Run the command below to calculate the documented Qwen3-8B, 8192-token prefill example.

    Documented command
    python3 calculations/calc.py forward --model qwen3-8b --tokens 8192 --format md

Expected outcome

A Markdown-formatted estimate of per-operator resource requirements for the specified example.

Guide limitations

  • The calculation tool supports only the models listed by its model command.
  • Static estimates do not replace experiments; conclusions depend on the chosen model and input parameters.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
7 October 2026
Original website title
bojieli/ai-infra-book · Official homepage title · retained in its source language
How we review sourcesSuggest a correction