OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Audio & Voice AI

Llmvox

LLMVoX is a Python-based, autoregressive streaming text-to-speech model that converts language-model text output into speech and includes a streaming server for voice-based conversations.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

LLMVoX — OpenViST project profile in Audio & Voice AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when building or researching an LLM application that needs incremental speech output and you can provide the documented CUDA-compatible GPU environment and model checkpoints.

Who it is for

  • Developers prototyping spoken responses for language-model applications.
  • Researchers exploring incremental text-to-speech or voice-chat systems.

Documented uses

  • Convert text generated by an LLM into streaming speech.
  • Explore a voice-chat setup using a language-model checkpoint and the included streaming server.
  • Adjust speech chunk sizes to trade faster initial audio for speech quality.
Before you choose

Requirements & limits

The documented use is through the public GitHub repository and a local Python environment. The README identifies the project license as CC BY-NC-SA 4.0.

  • The stated system requirements include CUDA 11.7 or higher and an Ampere-or-newer GPU compatible with Flash Attention 2.
  • Inference configuration requires paths to both the WavTokenizer and LLMVoX checkpoints.
  • The README describes two TTS model replicas and provides device-allocation options, including running both on one GPU if memory allows.
  • Chunk-size parameters expose a trade-off between faster initial response and speech quality.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Start LLMVoX voice chat with a configured LLM checkpoint

Launch the documented voice-chat streaming server using an LLM checkpoint.

Before you start

  • A prepared LLMVoX environment and required checkpoints
  • The configuration paths set in configs/inference_config.py
  • An LLM checkpoint, such as the documented meta-llama/Llama-3.1-8B-Instruct
  • GPU resources for the LLM and the two TTS model replicas

Steps

  1. From the repository directory, run the command below.

    Documented command
    python streaming_server.py --chat_type voice --llm_checkpoint "meta-llama/Llama-3.1-8B-Instruct"

  2. If allocating separate GPUs, specify TTS devices with --tts_device_1 1 --tts_device_2 2; the README also documents assigning the LLM with --llm_device "cuda:0".

  3. To adjust the documented latency/quality tradeoff, add the chosen --initial_dump_size_1, --initial_dump_size_2, and --max_dump_size values from the README examples.

Expected outcome

The documented command starts the voice-chat streaming server with the selected LLM checkpoint and optional device or chunk-size settings.

Guide limitations

  • Running both TTS replicas on one GPU is described as conditional on available memory.
  • The README describes chunk-size settings as a latency/quality tradeoff; it does not establish results for a specific setup.
  • This guidance reports the documented usage and does not imply a successful test.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
mbzuai-oryx/LLMVoX · Official homepage title · retained in its source language
How we review sourcesSuggest a correction