OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Neo AI Dlr

Neo AI Dlr is a compact C++ runtime library with Python and C++ APIs for loading and running compiled deep-learning and decision-tree models. It accepts models compiled with AWS SageMaker Neo, TVM or Treelite, and can use TVM, Treelite, NVIDIA TensorRT and other hardware-specific runtimes.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Neo AI Dlr — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use DLR after compiling a deep-learning or decision-tree model when an application needs a runtime interface to load the artifacts and run inference on a supported target.

Who it is for

  • Developers integrating compiled machine-learning models into Python or C++ applications.
  • Teams deploying supported compiled models to CPU, GPU or edge targets.
  • Android developers exploring the repository's image-classification inference example.

Documented uses

  • Load compiled model artifacts into an application.
  • Run inference through a unified Python or C++ interface.
  • Use compiled models on supported device targets.
  • Adapt the Android image-classification example for an application.
Before you choose

Requirements & limits

The library is open source under the Apache License 2.0.

  • The compiler and DLR release versions must align for TVM-compiled models.
  • Linux x86_64 CPU installation is documented with pip; GPU and non-x86 edge targets use prebuilt binaries or a source build.
  • DLR uses runtime backends including TVM, Treelite and NVIDIA TensorRT.
  • The README states that device and model-related metrics are collected by default and describes opt-out steps.
  • The repository includes an Android image-classification example, but does not establish support for every Android device or model.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Load a compiled model and call inference

Use the documented Python interface to load compiled model artifacts and run inference.

Before you start

  • DLR installed for the target environment
  • A compiled model directory containing .so, .params and .json artifacts
  • Input data in a form accepted by the model

Steps

  1. Import dlr and numpy in Python.

  2. Create a dlr.DLRModel using the compiled model directory, device type and device index.

  3. Prepare input data and pass it to model.run(x).

Expected outcome

The Python API returns the result of a model inference call.

Guide limitations

  • Model artifacts must be compatible with the DLR release and compiler/runtime versions.
  • For TVM-compiled models, use the DLR release corresponding to the TVM release used to compile the model.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
neo-ai/neo-ai-dlr · Official homepage title · retained in its source language
How we review sourcesSuggest a correction