Practical overview
When it fits
Use it when you need research code for speech restoration, offline enhancement experiments, model training, or exploration of frame-by-frame inference and mel vocoding.
Who it is for
- Researchers studying generative speech restoration or mel vocoding.
- Audio and machine-learning developers prototyping speech-enhancement or streaming audio pipelines.
Documented uses
- Run offline speech enhancement on WAV files and save enhanced outputs.
- Train models using Hydra configurations and local dataset paths.
- Explore frame-by-frame streaming inference through the documented state initialization and step functions.
- Use the MelFlow code and configuration for mel-vocoder research.
Requirements & limits
The repository is public and identified as open source under the AGPL-3.0 license. Its README specifies Python 3.10 or newer for the installation example.
- The offline inference path requires a checkpoint that matches the selected model configuration.
- Training requires editing config/_paths.yaml to reference local dataset paths.
- Streaming inference is described through backbone functions; the README points to supplementary material for optimization and notes that torch.compile settings may need adjustment for the hardware and goal.
- Mel-spectrogram inputs must match the configuration used to train the model.
- The README recommends streamfm_melflow.yaml for MelFlow training aligned with the Stream.FM paper; it describes different gradient clipping, GPU-count and training-step settings from melflow_original.yaml.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Run offline speech-enhancement inference
Process a folder of WAV files with a matching Stream.FM model configuration and checkpoint.
Before you start
- Python 3.10 or newer in a virtual environment.
- Repository dependencies installed, with the PyTorch package index selected for the local CUDA installation.
- A checkpoint matching the selected model configuration.
- A folder containing WAV input files.
Steps
Install dependencies using the repository instructions and adapt the PyTorch package index to the local CUDA installation.
Download or otherwise obtain a provided checkpoint and identify its matching model configuration.
Run inference.py with the matching --config-name and the required +inpath, +outpath, +solver and +ckpt Hydra keys; replace the example input path with the WAV folder to process.
Choose a solver available in sgmse/util/solvers.py; the README recommends starting with 5xeuler.
Expected outcome
Enhanced WAV files are written to the output folder with the same filenames as the inputs.
Guide limitations
- This describes offline inference, not a streaming deployment.
- GPU parallelization is optional; the example's +gpus=2 is not required.
- Checkpoint architecture must match the selected configuration.
Prepare the repository for training
Set up the Python environment and configure local dataset paths before using the training scripts.
Before you start
- Python 3.10 or newer.
- Conda or another way to create and activate a Python virtual environment.
- A local dataset and its paths.
Steps
Create and activate a virtual environment; the README gives a Conda example using Python 3.10.
Install requirements.txt and adapt the PyTorch extra index URL to the local CUDA installation.
Edit config/_paths.yaml to point to local dataset paths.
Select a Hydra configuration from config/; for example, the documented bandwidth-extension example uses streamfm_bwe with train.py. For configurations beginning with LRK, the README directs users to fit_rk_scheme.py instead.
Expected outcome
The environment and dataset-path configuration are prepared for the documented training entry points.
Guide limitations
- The repository instructions do not establish that every configuration or hardware setup will work unchanged.
- Training execution and results depend on the selected task configuration and local data.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 16 October 2026
- Original website title
- sp-uhh/streamfm · Official homepage title · retained in its source language