OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Pytorch Original Transformer

Pytorch Original Transformer is a public MIT-licensed repository containing a PyTorch implementation of the original Transformer architecture, educational visualizations, training and translation scripts, and pretrained IWSLT English–German models.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Pytorch Original Transformer — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a hands-on learning resource or a local PyTorch example for Transformer-based English–German translation, rather than a hosted translation interface.

Who it is for

  • Learners who want to inspect and understand the original Transformer implementation.
  • Developers experimenting with English–German neural machine translation in PyTorch.
  • Instructors or explainers who want visualizations of Transformer concepts.

Documented uses

  • Explore the annotated Transformer implementation in a Jupyter notebook.
  • Visualize positional encodings and other concepts using `playground.py`.
  • Translate English or German sentences with included pretrained IWSLT models.
  • Configure and run training experiments with the documented dataset and language-direction settings.
Before you choose

Requirements & limits

The public GitHub repository is identified as open source under the MIT license.

  • The repository is explicitly presented as a learning resource; its original Transformer is not described as state of the art.
  • Pretrained models cover the English–German language pair in both directions.
  • The documented setup uses Conda and can be accessed through Jupyter Notebook or an IDE.
  • WMT-14 training is described as hardware-intensive, and the README notes missing multi-GPU support in the documented setup.
  • Translation examples show that colloquial expressions can be mistranslated.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the repository for notebook study

Create the documented Conda environment and open the annotated Transformer notebook.

Before you start

  • Git
  • Anaconda or Miniconda with Conda
  • Python environment support for PyTorch and Jupyter

Steps

  1. Clone the repository with the command below.

    Documented command
    git clone https://github.com/gordicaleksa/pytorch-original-transformer

  2. Navigate to the project directory in an Anaconda console.

  3. Run conda env create in the project directory.

  4. Activate the pytorch-transformer environment, then run jupyter notebook and open The Annotated Transformer ++.ipynb.

Expected outcome

The notebook opens in a browser within the project’s Conda environment for exploring the implementation.

Guide limitations

  • Environment creation may take time while SpaCy's English and German statistical models download.
  • The README recommends system-wide CUDA installation for GPU drivers; GPU training needs suitable hardware.
Documentation 1 ↗
Source-based practical guide

Run translation with an included IWSLT model

Use the documented translation script and a pretrained English–German or German–English IWSLT model.

Before you start

  • The repository and its Conda environment
  • A source sentence in the language matching the selected model
  • The model name, dataset name and language direction kept consistent

Steps

  1. Choose iwslt_e2g for English-to-German or iwslt_g2e for German-to-English.

  2. Run translation_script.py with the source sentence, model name, IWSLT dataset name and matching direction (E2G or G2E).

  3. Use the script's output as the translation.

Expected outcome

The script produces a translation using the selected pretrained IWSLT model, which the README says is downloaded automatically on first use.

Guide limitations

  • The README's examples include a failure to interpret a colloquial German word correctly.
  • The model and language-direction settings must match.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
gordicaleksa/pytorch-original-transformer · Official homepage title · retained in its source language
How we review sourcesSuggest a correction