Practical overview
When it fits
Use Bertviz after preparing a Transformer model, tokenizer and input whose model output includes attention values. It is most useful when interactive inspection is more informative than examining raw tensors.
Who it is for
- NLP researchers investigating Transformer behavior
- Developers debugging or comparing model attention patterns
- Educators demonstrating self-attention and Transformer architectures
- Jupyter and Google Colab users working with language models
Documented uses
- Inspect attention across individual heads and layers
- View attention across an entire model
- Examine query and key neurons involved in attention
- Compare attention responses for different inputs or models
- Create notebook-based demonstrations of Transformer mechanisms
Requirements & limits
The repository provides Jupyter and Colab setup instructions. It is public, open source and licensed under Apache-2.0.
- The model must expose attention values, typically through
output_attentions=True. - Jupyter users need JupyterLab and ipywidgets in addition to Bertviz.
- The tool supports multiple views, so the intended inspection level matters: head, model or neuron.
- The repository describes support for most Hugging Face models and also documents encoder-decoder and non-Hugging Face use cases.
- The Colab tutorial includes pre-loaded visualizations, while a working notebook workflow requires model and input preparation.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up Bertviz in Jupyter
Prepare a Jupyter notebook to display Transformer attention visualizations.
Before you start
- Python environment
- JupyterLab
- ipywidgets
- Bertviz
- A supported Transformer model and tokenizer
Steps
Install Bertviz with pip.
Install JupyterLab and ipywidgets.
Launch Jupyter with
jupyter lab.Create a notebook using the Python 3 kernel.
Load a Transformer model configured with
output_attentions=True, prepare tokenized input and pass the resulting attention values and tokens to a Bertviz view.
Expected outcome
A notebook prepared to display interactive attention views for the selected model and input.
Guide limitations
- The model must return attention values for visualization.
- The repository notes that visualization loading may take a few seconds.
- Model support and view details depend on the model type and the documented Bertviz use case.
Add Bertviz to a Colab notebook
Install Bertviz in Colab and prepare the notebook for attention visualization.
Before you start
- A Google Colab notebook
- A Transformer model and tokenizer
- Model outputs configured to include attention values
Steps
Add
!pip install bertvizat the beginning of the Colab notebook.Load the model and tokenizer.
Tokenize the input and run the model with attention output enabled.
Send the attention values and token strings to a Bertviz visualization.
Expected outcome
A Colab notebook prepared to interactively inspect the selected model's attention patterns.
Guide limitations
- The model must provide attention values.
- The separate interactive tutorial contains pre-loaded visualizations, so its examples do not require executing cells.
- The repository documents the Colab setup but does not establish compatibility for every model.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 2 October 2026
- Original website title
- jessevig/bertviz · Official homepage title · retained in its source language