AI · Agents · Blockchain · Crypto

Submit a project Sign in

Menu

Directory category

AI Infrastructure & Serving

Explore infrastructure for serving models, managing inference and deploying AI workloads. Compare official resources, requirements and limitations.

196 projects

Directory results

AI Infrastructure & Serving projects

NVIDIA Developer NVIDIA TensorRT is a toolkit of APIs, a general-purpose AI compiler, and an inference runtime for optimizing and deploying deep-learning inference applications.

S Sndr Core Engine Sndr Core Engine is a Python-based, Apache-2.0 runtime patch overlay for vLLM that optimizes local inference and serving of supported open LLMs, with deeper tuning for Qwen3.6 and Gemma 4.

N Neo AI Dlr Neo AI Dlr is a compact C++ runtime library with Python and C++ APIs for loading and running compiled deep-learning and decision-tree models.

M Multi Model Server Multi Model Server is an open-source server tool that starts HTTP endpoints for inference requests to deep-learning models. It can be started through its command-line interface or pre-configured Docker images.

U Utensor Utensor is an Apache-2.0 C++ machine-learning inference framework with a runtime library and an offline tool that translates TensorFlow models into C++11 source for embedded targets.

TensorFlow TensorFlow is an open-source machine-learning framework for expressing, training and running models, with APIs and related tools for development, distributed training, visualization and deployment.

N Ncnn ncnn is a C++ neural-network inference framework for deploying models on mobile, embedded and desktop systems, with CPU and Vulkan GPU backends and tools for converting PyTorch and ONNX models.

D deeprec.readthedocs.io deeprec.readthedocs.io hosts Chinese-language documentation for DeepRec, a deep-learning framework focused on large-scale sparse models.