OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

TensorRT C++ API

TensorRT C++ API is a C++ library that converts ONNX models into cached, optimized NVIDIA TensorRT engines and runs CNN inference on CUDA GPUs. It provides name-keyed tensor inputs and outputs, caller-owned CUDA streams, explicit host/device transfers and a Status/Result error model, with optional fused image preprocessing and zero-copy Python bindings.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

TensorRT C++ API — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when a Linux application needs a C++ inference component for ONNX CNN models on CUDA with TensorRT, particularly when explicit memory transfers, stream control, engine caching or optional Python GPU-array access matter.

Who it is for

  • C++ developers building Linux computer-vision applications or services that run ONNX CNN models on NVIDIA GPUs.
  • Python developers seeking optional bindings to pass GPU arrays into inference without host transfers.
  • Teams integrating TensorRT inference into a downstream CMake project.

Documented uses

  • Build or load a cached TensorRT engine from an ONNX model.
  • Run CNN inference using named tensors and CUDA streams.
  • Handle dynamic input shapes with optimization profiles and manage concurrent contexts through an engine pool.
  • Optionally preprocess image tensors with a fused CUDA kernel or expose inference to Python GPU-array workflows.
Before you choose

Requirements & limits

The repository is public and lists an MIT license. The documented target requires Linux, CUDA 12, C++20 and TensorRT 10 or later; CUDA and TensorRT are externally provided.

  • TensorRT and CUDA must be installed as system or external dependencies; they are not bundled as the library's runtime prerequisites.
  • The library targets Linux, CUDA 12, C++20 and TensorRT 10 or later, and is scoped to CNN-style vision rather than Windows or transformer-specific use.
  • Engine-cache validity accounts for ONNX content, build options, TensorRT version and GPU identity.
  • Dynamic-shape workflows require optimization profiles; concurrency can use an EnginePool with caller-provided streams.
  • Python bindings are optional; the core integration is a C++ library.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build and install the C++ library

Configure, compile and install the library package for use by a downstream CMake project.

Before you start

  • Linux, CUDA 12, C++20 and TensorRT 10 or later.
  • CMake and a TensorRT installation available to the build; the README notes a TensorRT_DIR option for a tarball installation.

Steps

  1. Configure the project with CMake; enable the optional preprocessing component with TRT_CPP_API_BUILD_PREPROC=ON if needed, and provide TensorRT_DIR when using a tarball.

  2. Build the configured project with CMake.

  3. Install the package to a chosen prefix with cmake --install.

  4. In the downstream CMake project, call find_package(tensorrt_cpp_api REQUIRED) and link the documented package targets.

Expected outcome

An installed CMake package that a downstream project can discover and link against.

Guide limitations

  • This is a build-and-install path; it does not describe deploying or executing a model.
  • TensorRT and CUDA are externally provided dependencies.
  • The documented target is Linux with CUDA 12, C++20 and TensorRT 10 or later.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
TensorRT C++ APIBuild or load a cached TensorRT engine from an ONNX model.Linux, ServerThe repository is public and lists an MIT license. The documented target requires Linux, CUDA 12, C++20 and TensorRT 10 or later; CUDA and TensorRT are externally provided.
Cloud AI SDKAdapt and run documented model recipes for computer-vision and NLP inference.Cloud, ServerThe GitHub repository is public. Its public visibility does not establish license terms.
TurboquantReplace a model’s existing inference cache with a compressed cache.Cloud, Linux, macOS, Self-hostedThe project describes itself as open source. The README documents installation through pip or from a source checkout.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
14 October 2026
Original website title
cyrusbehr/tensorrt-cpp-api · Official homepage title · retained in its source language
How we review sourcesSuggest a correction

Related options

YOLOv8-TensorRT-CPP — A sister project identified in the repository as using this library as its inference backend.

YOLOv9-TensorRT-CPP — A sister project identified in the repository as using this library as its inference backend.