Mtdata: Find and prepare machine-translation datasets

Understand the project

Practical overview

Use Mtdata when an experiment needs corpora collected from supported sources and normalized across different archive and parallel-text formats.

Machine-translation researchers preparing training, development or test corpora.
Developers building repeatable workflows to collect parallel-text datasets.

Before you choose

The README documents pip, Git-based and editable-mode installation. The repository identifies the project as Apache-2.0 licensed.

Dataset selection is explicit: users choose language pairs and dataset IDs, including training and test data.
Mtdata supports command-line workflows and also provides a Python API.
The tool's known sources and datasets determine what can be selected; its README's dataset-count snapshot is dated February 2022.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

List known corpora for a language pair, then prepare selected training, development and test data in an output directory.

Run mtdata list -l deu-eng to list datasets for the example German–English pair.
Choose dataset IDs for training, development and testing.
Run mtdata get with the language pair, selected IDs and an output directory; use --merge if the selected training sets should be concatenated.

The selected corpora are downloaded and prepared in the specified directory, with a signature file recording the dataset choices and command.

The example uses German–English; select IDs known to Mtdata for the language pair you need.
The get command requires a definite choice of training and test data; the README example also assigns a development set.

Evidence and transparency

Source record details

Source availability: Active
Sources checked: 14 October 2026
Original website title: thammegowda/mtdata · Official homepage title · retained in its source language