OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Fastflowlm

Fastflowlm is an open-source runtime and command-line tool for running supported AI models on AMD Ryzen AI NPUs. It supports language, vision, audio, embedding and mixture-of-experts model workloads, and offers CLI and local server modes for model inference.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Fastflowlm — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Fastflowlm when your target system has a supported Ryzen AI NPU and you want a runtime for local model inference through a CLI or local server.

Who it is for

  • Developers building applications that need local model inference on AMD Ryzen AI NPU hardware.
  • Teams evaluating NPU-based execution for supported language, vision, audio or embedding models.

Documented uses

  • Run a supported language model from a terminal on an NPU-equipped device.
  • Launch a local inference server for an application to use.
  • Run supported vision, audio, embedding or mixture-of-experts workloads locally.
Before you choose

Requirements & limits

The repository is public and identifies the orchestration code and CLI tools as MIT-licensed. A packaged Windows installer and Linux getting-started guide are linked from the project repository.

  • Fastflowlm is targeted at Ryzen AI NPU hardware, specifically systems with XDNA2 NPUs.
  • Windows quick-start instructions specify AMD NPU driver version 32.0.203.311 or later.
  • Internet access to Hugging Face is required to download optimized model kernels.
  • The README describes CLI and local server modes; server mode uses port 52625 by default.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Run a model from the Windows CLI

Start a supported model in Fastflowlm's terminal mode on a compatible Ryzen AI NPU system.

Before you start

  • A Windows system with a supported AMD Ryzen AI XDNA2 NPU.
  • AMD NPU driver version 32.0.203.311 or later.
  • Fastflowlm installed from the project’s Windows installer.
  • Internet access to Hugging Face to download optimized model kernels.

Steps

  1. Install Fastflowlm using the packaged Windows installer.

  2. Open PowerShell.

  3. Run flm run llama3.2:1b to start the documented example model.

Expected outcome

Fastflowlm starts the selected model in CLI mode for terminal interaction.

Guide limitations

  • The example requires the model and optimized kernels to be available for download.
  • The project notes that Hugging Face downloads can sometimes be corrupted and documents flm pull --force as a re-download option.
Documentation 1 ↗
Source-based practical guide

Start the local inference server

Launch Fastflowlm server mode with an initial model selection.

Before you start

  • Fastflowlm installed on a compatible system.
  • A model supported by Fastflowlm.

Steps

  1. Open a terminal where the flm command is available.

  2. Run flm serve llama3.2:1b to start the server with the documented example model.

Expected outcome

The local server starts on port 52625 by default; the specified model tag sets the initial model.

Guide limitations

  • The project notes that another requested model can trigger an automatic model switch.
  • The cited instructions do not provide application-integration steps for consuming the server.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
FastflowlmRun a supported language model from a terminal on an NPU-equipped device.Linux, Self-hosted, WindowsThe repository is public and identifies the orchestration code and CLI tools as MIT-licensed. A packaged Windows installer and Linux getting-started guide are linked from the project repository.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
11 October 2026
Original website title
ROCm/FastFlowLM · Official homepage title · retained in its source language
How we review sourcesSuggest a correction