OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

PPLNN

PPLNN is an Apache-2.0 C++ neural-network primitive library and deep-learning inference engine that runs ONNX models. Its repository documents C++ and Python APIs, model integration and conversion guidance, and support for OpenMMLab. It also describes a separate LLM engine with features such as dynamic batching and tensor parallelism.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

PPLNN — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use PPLNN when developing software that needs a C++ inference engine for ONNX models, or when its documented APIs, model-conversion guidance or engine-extension tools fit the integration work.

Who it is for

  • Developers building AI inference applications around ONNX models.
  • Engineers integrating neural-network inference into software through C++ or Python.
  • Contributors extending supported engines or operations.

Documented uses

  • Run an ONNX model through an inference engine.
  • Build PPLNN from source with its Python API enabled and use the documented Python demo.
  • Integrate PPLNN into an application using its C++ or Python documentation.
  • Consult model-conversion guidance or extend engines and operations.
Before you choose

Requirements & limits

The repository is public, identifies an Apache-2.0 license and provides source-build instructions.

  • The README documents a source build using CMake-related toolchains and provides a Python demo; it is not a hosted inference service.
  • Documentation lists distinct x86, CUDA, RISCV and ARM areas, so check the relevant supported-operations and platform guide for a target.
  • The README reports a potential NCCL AllReduce issue on some L40S and H800 devices and suggests setting NCCL_PROTO=^Simple.
  • The README says PMX changed to OPMX and that LLM models must be converted or re-exported; ChatGLM1 is not supported in OPMX.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build PPLNN and run the Python ONNX demo

Build the repository with its Python API enabled, then invoke the documented demo on the included ONNX model.

Before you start

  • On Debian or Ubuntu: build-essential, cmake, git, python3 and python3-dev.
  • On RedHat or CentOS: gcc, gcc-c++, cmake3, make, git, python3 and python3-devel.

Steps

  1. Clone the repository with git clone https://github.com/openppl-public/ppl.nn.git.

  2. Change into the ppl.nn directory.

  3. Build with ./build.sh -DPPLNN_USE_X86_64=ON -DPPLNN_ENABLE_PYTHON_API=ON.

  4. Run PYTHONPATH=./pplnn-build/install/lib python3 ./tools/pplnn.py --use-x86 --onnx-model tests/testdata/conv.onnx.

Expected outcome

The documented Python demo is configured to run the included convolution ONNX model using the x86 option.

Guide limitations

  • The instructions cover an x86-64 build and demo; they do not establish support for other target configurations.
  • The repository reports a possible NCCL AllReduce issue on some L40S and H800 devices.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
PPLNNRun an ONNX model through an inference engine.The repository is public, identifies an Apache-2.0 license and provides source-build instructions.
Tiny TensorrtBuild a TensorRT engine from an ONNX model.Linux, Server, Self-hosted
Kvcache FactoryCompare multiple cache approaches against a full-cache baseline under a shared evaluation interface.Linux, Self-hostedThe repository is public, open source and MIT-licensed. It is a self-hosted Linux Python library; setup uses pip and the repository’s requirements file.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
3 November 2026
Original website title
OpenPPL/ppl.nn · Official homepage title · retained in its source language
How we review sourcesSuggest a correction