OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Audio & Voice AI

Whisper At

Whisper-AT is a Python model project with code and pretrained models for speech recognition and audio event tagging. Built on Whisper, it produces transcripts alongside labels from 527 AudioSet classes at a selected time resolution.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Whisper At — OpenViST project profile in Audio & Voice AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Whisper-AT when an audio workflow needs both a speech transcript and time-resolved sound-event labels, or when investigating this combined model approach.

Who it is for

  • Developers adding speech transcription and sound-event labels to audio workflows
  • Researchers exploring joint speech recognition and audio tagging
  • Users who want to try the model through its linked hosted or notebook demos

Documented uses

  • Transcribe speech from an audio file
  • Identify audio events alongside speech and assign labels at selected time intervals
  • Prototype or study a combined speech-recognition and audio-tagging model
Before you choose

Requirements & limits

The project links to a Hugging Face Space for a no-code demo, a Colab demo, a local notebook, and a Python package. The repository identifies the project as open source under BSD-2-Clause.

  • The project exposes a Python package and pretrained model loading workflow.
  • Audio tagging resolution is set with at_time_res; this setting affects tagging, not transcription.
  • The README requires ffmpeg and documents a Mac/Windows installation workaround.
  • The project describes a 527-class AudioSet label set and qualifies its additional-compute-cost claim as applying to some models.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Whisper-AT and process an audio file

Use the documented Python workflow to obtain a transcript and parsed audio event labels.

Before you start

  • Python environment that can run original Whisper
  • ffmpeg installed on the system
  • An audio file such as audio.mp3
  • For Mac or Windows, use the README's listed dependency workaround

Steps

  1. Install whisper-at with pip; on Mac or Windows, follow the documented dependency workaround.

  2. Import the package as whisper_at and load a model with whisper.load_model.

  3. Call model.transcribe with the audio file and an at_time_res value for event-tagging resolution.

  4. Read result["text"] for the transcript and pass the result to whisper.parse_at_label to obtain event labels.

Expected outcome

A transcription result and parsed audio event labels at the selected tagging resolution.

Guide limitations

  • The README says transcription follows Whisper's 30-second window; at_time_res controls audio tagging only.
  • The additional-compute-cost claim applies to some models, not all.
  • The Mac/Windows installation workaround addresses a documented known bug.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
15 October 2026
Original website title
YuanGongND/whisper-at · Official homepage title · retained in its source language
How we review sourcesSuggest a correction