OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Audio & Voice AI

Whisper Finetune — vasistalodagala

A Python repository of scripts for fine-tuning and evaluating Whisper automatic speech recognition models, preparing custom datasets, and extracting embeddings from model layers.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Whisper Finetune — vasistalodagala — OpenViST project profile in Audio & Voice AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need repository scripts and data-preparation guidance for Whisper fine-tuning or evaluation, or code snippets for extracting model-layer embeddings.

Who it is for

  • Developers adapting Whisper ASR models to custom speech datasets.
  • Practitioners comparing Whisper model configurations on Hugging Face or custom datasets.
  • Researchers using Whisper layer embeddings for downstream speech analysis.

Documented uses

  • Prepare audio paths and matching transcripts for the Hugging Face sequence-to-sequence training pipeline.
  • Fine-tune Whisper models using Hugging Face or custom datasets.
  • Evaluate Whisper models on Hugging Face or custom datasets.
  • Extract embeddings from different layers of Whisper models.
Before you choose

Requirements & limits

  • The README reports testing with Python 3.8 and CUDA 11.3 and recommends a virtual environment.
  • Custom data preparation requires audio_paths and text files with consistent unique utterance IDs.
  • The README includes distributed training examples using multiple GPUs.
  • Pushing models to Hugging Face requires CLI login and Git LFS for large model files.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare a custom speech dataset

Convert audio paths and matching transcripts into the format used by the Hugging Face sequence-to-sequence pipeline.

Before you start

  • Audio files and their reference transcriptions.
  • Two files named audio_paths and text, with matching unique utterance IDs in the same order.
  • Python environment with the repository dependencies installed.

Steps

  1. In audio_paths, list each unique utterance ID alongside the absolute path to its audio file.

  2. In text, list the same IDs in the same order alongside the corresponding ground-truth transcription.

  3. Run the command below.

    Documented command
    python3 custom_data/data_prep.py --source_data_dir source_data_directory --output_data_dir output_data_directory

Expected outcome

A converted dataset in the format expected by the Hugging Face sequence-to-sequence pipeline.

Guide limitations

  • This preparation workflow applies to datasets not available on Hugging Face.
  • The README provides a help command for further usage details; this guide does not cover the subsequent training or evaluation run.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
vasistalodagala/whisper-finetune · Official homepage title · retained in its source language
How we review sourcesSuggest a correction