Practical overview
When it fits
Use it when you want a code-based audio-captioning workflow with documented Clotho setup, pretrained weights, inference and evaluation steps.
Who it is for
- Researchers reproducing the project’s published audio-captioning workflow.
- Developers exploring or adapting a PyTorch model that turns audio recordings into descriptive captions.
Documented uses
- Generate descriptive text captions for audio recordings.
- Run inference and evaluate captions using the included workflow.
- Reproduce the project’s results with the Clotho dataset and pretrained model.
Requirements & limits
The implementation is available in a public GitHub repository under the Apache-2.0 license.
- The project is a model implementation and code repository, not a hosted captioning interface.
- The documented reproduction path requires Python, PyTorch, dataset and model downloads, and several supporting tools.
- The README recommends Python 3.9 with Conda and a PyTorch build matched to CUDA.
- The repository identifies an additional Hugging Face dataset of 50,000 mix-up caption augmentations.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up and run the Clotho captioning evaluation
Prepare the documented dependencies and data, then run the repository’s inference and evaluation script.
Before you start
- Conda environment with Python 3.9 is recommended.
- PyTorch installed with a matching CUDA version.
- Repository dependencies and SPICE metric models.
- p7zip to unpack Clotho and Git-LFS for the pretrained model workflow.
Steps
Install the dependencies, including SPICE metric models and the packages in requirements.txt.
Install p7zip and run bash download_clotho.sh to obtain the Clotho dataset.
Install and initialize Git-LFS, then run bash download_model.sh to obtain the pretrained model.
Run bash run_sampling_reranking.sh; the README says evaluation metrics are written to ./exp/inference_evaluation_nucleus_t0.5_p95/inference_metrics.json.
Expected outcome
The repository’s inference and evaluation workflow runs against its Clotho setup and records metrics at the documented JSON path.
Guide limitations
- The README gives a CUDA-matched PyTorch prerequisite but does not specify hardware requirements.
- The guide follows the repository’s documented reproduction path; it does not establish performance on other datasets or environments.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 17 October 2026
- Original website title
- slSeanWU/beats-conformer-bart-audio-captioner · Official homepage title · retained in its source language