OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Transformer Uncertainty

Transformer Uncertainty is a Python research codebase for evaluating predictive uncertainty methods on Transformer-based natural-language-understanding tasks. It documents deterministic predictions, Monte Carlo dropout, temperature scaling, ensembles and Bayesian layers, alongside a command-line example for BERT on a GLUE task.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Transformer Uncertainty — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a code-based research workflow for evaluating uncertainty methods on Transformer NLU tasks and can prepare the repository’s specified Python and PyTorch environment.

Who it is for

  • Researchers comparing uncertainty-estimation methods for Transformer-based NLU models.
  • Practitioners examining calibration approaches such as temperature scaling.

Documented uses

  • Compare predictive uncertainty methods on GLUE natural-language-understanding tasks.
  • Run a documented BERT/RTE example using a Bayesian adapter method.
  • Set up an experiment workflow for exploring dropout, ensembles, calibration or Bayesian-layer approaches.
Before you choose

Requirements & limits

The repository is public and identifies an MIT license. Its documented workflow is run locally from command-line scripts.

  • The README specifies Python 3.7 and PyTorch 1.2.0 with torchvision and CUDA toolkit 10.0; check the cited PyTorch previous-versions page for installation details applicable to your machine.
  • The included data-download step targets GLUE, and the concrete example uses RTE.
  • Additional command-line arguments are described in main_transformer.py.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare the environment and run the documented RTE example

Set up the repository’s stated environment, download GLUE data and invoke its BERT/RTE Bayesian-adapter example.

Before you start

  • Conda
  • Python 3.7
  • PyTorch 1.2.0, torchvision and CUDA toolkit 10.0, as specified by the README
  • The repository requirements file

Steps

  1. Create and activate the Conda environment named unc_env with Python 3.7.

  2. Install the specified PyTorch, torchvision and CUDA toolkit packages, then install requirements with pip.

  3. Run bash get_data.sh to download GLUE datasets.

  4. Run the README example with dataset_name rte, model_type bert, model_name_or_path bert-base-uncased, seed 1234 and unc_method bayes_adapt.

Expected outcome

The documented example invokes the repository’s model script for BERT on GLUE RTE using the bayes_adapt uncertainty method.

Guide limitations

  • The README refers to its main script for the remaining arguments; it does not specify an evaluation result or expected metrics.
  • The README directs users to PyTorch’s previous-versions page to select an installation appropriate to their machine.
Documentation 1 ↗Documentation 2 ↗
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
mourga/transformer-uncertainty · Official homepage title · retained in its source language
How we review sourcesSuggest a correction