OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Ncnn

ncnn is a C++ neural-network inference framework for deploying models on mobile, embedded and desktop systems, with CPU and Vulkan GPU backends and tools for converting PyTorch and ONNX models.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Ncnn — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use ncnn when you need a C++ inference framework to integrate neural-network models into software running on supported device or desktop targets.

Who it is for

  • Developers adding neural-network inference to mobile applications.
  • Engineers integrating models into embedded or edge devices.
  • Desktop and browser software developers deploying deep-learning models.

Documented uses

  • Convert a PyTorch model into ncnn parameter and weight files using pnnx.
  • Load model files and extract inference outputs through the C++ or Python API.
  • Build the framework for a listed target such as Android, iOS, desktop operating systems or WebAssembly.
Before you choose

Requirements & limits

  • The project separates model conversion from inference: pnnx converts models, while ncnn loads the resulting files to run inference.
  • The documented starter route requires a PyTorch environment for pnnx; ONNX conversion is covered by separate project guidance.
  • Build instructions and downloads are listed for multiple targets, so the target-specific build path matters.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Convert a PyTorch model and run inference with ncnn’s C++ API

Export the example PyTorch model to ncnn files, then load them and extract an output using the C++ API.

Before you start

  • A PyTorch environment.
  • The pnnx package installed in that environment.
  • An ncnn C++ development setup to compile and run code using the ncnn API.

Steps

  1. Install pnnx in the PyTorch environment with the command below.

    Documented command
    pip3 install pnnx

  2. Define and evaluate the example model, create a sample tensor, and export it with pnnx.export(model, "model.pt", (x,)). The export produces model.ncnn.param and model.ncnn.bin.

  3. In C++, create an ncnn::Net and load the generated parameter and model files with load_param and load_model.

  4. Create an extractor, pass an input ncnn::Mat using ex.input("in0", in), and extract the output with ex.extract("out0", out).

Expected outcome

A model is exported into ncnn parameter and weight files, and the C++ example loads them and requests an inference output.

Guide limitations

  • The README example uses a particular model and tensor shape; it does not document adapting inputs or output names for other models.
  • The example shows the inference API calls but does not provide compilation instructions or report a successful test.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
NcnnConvert a PyTorch model into ncnn parameter and weight files using pnnx.Android, Desktop, iOS, Self-hosted
Cloud AI SDKAdapt and run documented model recipes for computer-vision and NLP inference.Cloud, ServerThe GitHub repository is public. Its public visibility does not establish license terms.
TurboquantReplace a model’s existing inference cache with a compressed cache.Cloud, Linux, macOS, Self-hostedThe project describes itself as open source. The README documents installation through pip or from a source checkout.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
Tencent/ncnn · Official homepage title · retained in its source language
How we review sourcesSuggest a correction