OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Tiny Tensorrt

Tiny Tensorrt is a C++ and Python wrapper around NVIDIA TensorRT that builds inference engines from ONNX models and runs model inference.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Tiny Tensorrt — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Consider it when working with an existing C++ or Python integration that needs to build TensorRT engines from ONNX models, and when its documented dependencies and listed versions fit your environment.

Who it is for

  • C++ developers integrating TensorRT inference into their applications.
  • Python developers seeking a TensorRT-backed ONNX inference binding.

Documented uses

  • Build a TensorRT engine from an ONNX model.
  • Transfer model inputs to the device, run inference, and retrieve outputs.
  • Enable FP16 through the documented C++ wrapper flow.
Before you choose

Requirements & limits

  • Requires CUDA, cuDNN and TensorRT; the README also names CMake and zlib development files for building.
  • The documented Python binding additionally requires Python 3, pip and NumPy.
  • The repository states that the project is no longer maintained and identifies TensorRT's Python API, trtexec and Polygraphy as alternatives for engine building.
  • The README says third-party modules and TensorRT are governed by their own licenses.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build the library and Python binding

Compile Tiny Tensorrt after installing its documented dependencies.

Before you start

  • CUDA and cuDNN
  • TensorRT; the README lists CUDA 10.2 and 11.0–11.4, and TensorRT 7.0, 7.1, 7.2, 8.0, 8.2 and 8.4
  • CMake and zlib1g-dev
  • For the Python binding: Python 3, pip and NumPy

Steps

  1. Install the documented system packages: CMake and zlib1g-dev; for Python bindings, also install Python 3 and pip, then install NumPy.

  2. Clone the repository with its submodules using the README's recursive-clone instruction.

  3. In the repository, create a build directory, enter it, then run CMake configuration and make as shown in the README.

Expected outcome

The README says the build provides libtinytrt.so and Trt.h for C++ integration, and pytrt.so for the Python module.

Guide limitations

  • The project is no longer maintained.
  • The README's listed CUDA and TensorRT versions are specific to that documentation and do not guarantee compatibility with other environments.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Tiny TensorrtBuild a TensorRT engine from an ONNX model.Linux, Server, Self-hosted
PPLNNRun an ONNX model through an inference engine.The repository is public, identifies an Apache-2.0 license and provides source-build instructions.
Kvcache FactoryCompare multiple cache approaches against a full-cache baseline under a shared evaluation interface.Linux, Self-hostedThe repository is public, open source and MIT-licensed. It is a self-hosted Linux Python library; setup uses pip and the repository’s requirements file.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
7 November 2026
Original website title
zerollzeng/tiny-tensorrt · Official homepage title · retained in its source language
How we review sourcesSuggest a correction