OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Audio & Voice AI

Freeze Omni

Freeze Omni is a speech-to-speech dialogue model with a streaming speech encoder, a streaming autoregressive speech decoder and a state-prediction layer for detecting user interruptions. It uses a frozen text-modality LLM as its language backbone and is distributed with inference code, model weights and a server demo.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Freeze Omni — OpenViST project profile in Audio & Voice AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Freeze Omni when you want to experiment with a complete speech-to-speech dialogue model and its documented local inference or interactive-server paths, rather than only a speech recognition or speech synthesis component.

Who it is for

  • Researchers studying speech-to-speech dialogue, spoken question answering or interactive voice models.
  • Developers experimenting with streaming audio input and speech generation using a frozen language-model backbone.

Documented uses

  • Generate a spoken response WAV from a supplied WAV input using the documented inference command.
  • Explore chunk-wise streaming speech input and streaming speech output in dialogue experiments.
  • Configure the provided server demo for interactive voice sessions.
  • Study the project’s reported evaluations of speech recognition, output speech quality, spoken question answering and latency.
Before you choose

Requirements & limits

  • The documented setup requires both the Freeze Omni checkpoint and the Qwen2-7B-Instruct checkpoint.
  • The inference instructions specify Python 3.10 in a Conda environment and installation from the repository’s requirements file.
  • The interactive demo guidance recommends high-speed networking, a high-performance GPU and a quiet environment.
  • The repository says model outputs are random; generated content should not be treated as developer views.
  • The project is presented as a model with inference and demo code, not as a hosted consumer voice service.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Configure the interactive demo server

Prepare the documented server command or script for an interactive demo.

Before you start

  • The repository setup and both model checkpoints described in the inference instructions
  • A high-speed network connection and high-performance GPU, recommended by the project
  • A quiet environment, recommended for the interactive experience

Steps

  1. Set PYTHONPATH=./:$PYTHONPATH and configure the documented server command with an IP address and port, or edit those values in scripts/run_demo_server.sh.

  2. Use the documented server settings for user limit, model execution count, timeout, checkpoint paths and sampling parameters.

  3. Run bin/server.py with the configured arguments, or run the command below after editing its IP and port.

    Documented command
    sh scripts/run_demo_server.sh

Expected outcome

The documented process starts the project’s interactive demo server with the selected address, port and inference settings.

Guide limitations

  • The repository recommends fast networking and high-performance GPUs but does not specify hardware requirements or guarantee compatibility.
  • This recipe follows documented instructions and is not a tested execution.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Freeze OmniGenerate a spoken response WAV from a supplied WAV input using the documented inference command.
EdgedictDecode speech from a YouTube Live video into text.Cloud
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
15 October 2026
Original website title
VITA-MLLM/Freeze-Omni · Official homepage title · retained in its source language
How we review sourcesSuggest a correction