OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Voice Agents

Speech To Speech

Speech To Speech is an open-source Python framework for assembling low-latency voice agents from interchangeable voice activity detection, speech-to-text, language-model and text-to-speech components. It provides WebSocket and WebRTC interfaces built around the core OpenAI Realtime GA event set and supports an OpenAI-compatible protocol for its language-model stage.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Speech To Speech — OpenViST project profile in Voice Agents.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when a project needs a configurable, self-hosted voice-agent backend and the developer wants to choose or replace the speech and language-model components.

Who it is for

  • Developers building self-hosted conversational voice agents
  • Teams integrating real-time voice through WebSocket or WebRTC
  • Builders selecting local or hosted speech and language-model components
  • Python developers working with modular speech-processing pipelines

Documented uses

  • Assemble a voice-agent pipeline from VAD, STT, LLM and TTS stages
  • Run speech recognition, language generation and speech synthesis locally
  • Connect a hosted or local OpenAI-compatible LLM endpoint
  • Expose real-time voice interaction through WebSocket or WebRTC
  • Adapt the components for speech translation or synthesis applications
Before you choose

Requirements & limits

The repository is public and licensed under Apache-2.0. The documented quickstart requires Python 3.10 or newer, internet access for initial model downloads and hardware-specific setup. A hosted-LLM configuration requires an API key; fully local configurations do not.

  • The pipeline separates VAD, STT, LLM and TTS, so component selection is a central design choice.
  • WebSocket and WebRTC support matters when the client must use the core OpenAI Realtime GA event set.
  • Fully local operation is documented for Apple Silicon and Linux with an NVIDIA GPU.
  • Hosted-LLM use keeps microphone audio local in the documented configuration but sends transcribed text, instructions and conversation history to the provider.
  • The repository’s LLM interface is OpenAI-compatible; compatibility of any particular endpoint still needs to be checked independently.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up a fully local Apple Silicon voice agent

Install Speech To Speech and start the documented Apple Silicon local configuration.

Before you start

  • Apple Silicon Mac
  • Python 3.10 or newer; Python 3.11 recommended
  • Internet access for the first model downloads
  • Approximately 16 GB or more of unified memory is recommended
  • A microphone and speakers or headphones

Steps

  1. Create and activate a Python virtual environment.

  2. Install the speech-to-speech package with pip.

  3. Run the Apple Silicon local command with the mac-optimal-settings preset and the documented quantized Qwen3 model.

  4. Allow microphone access if prompted, wait for the first model download and warm-up, then speak and pause for a reply.

Expected outcome

A local voice-agent configuration using Parakeet TDT for speech recognition, Qwen3-4B through MLX LM for the language model and Qwen3-TTS through MLX Audio for speech output.

Guide limitations

  • The memory figures are planning estimates and actual use depends on context length, audio length and backend versions.
  • The first run downloads and warms up models.
  • Use headphones to reduce speaker feedback; blocking microphone capture during playback prevents interruption while the assistant speaks.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
2 October 2026
Original website title
huggingface/speech-to-speech · Official homepage title · retained in its source language
How we review sourcesSuggest a correction