OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

I Bert

I Bert is an open-source Python implementation for integer-only quantization of BERT-family transformer models, with a Fairseq-based RoBERTa fine-tuning and quantization workflow and links to an implementation in Hugging Face Transformers.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

I Bert — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need a code reference for a Fairseq-based RoBERTa fine-tuning and quantization workflow, or want to locate the linked Hugging Face implementation and checkpoints.

Who it is for

  • Researchers studying transformer quantization and model compression.
  • ML engineers adapting RoBERTa models to downstream tasks or exploring quantization-aware fine-tuning.

Documented uses

  • Prepare GLUE task data for RoBERTa fine-tuning.
  • Fine-tune pretrained RoBERTa models for downstream tasks, then apply quantization-aware fine-tuning.
  • Start from the linked Hugging Face I-BERT implementation or listed RoBERTa checkpoints.
Before you choose

Requirements & limits

The repository is public and lists an MIT license.

  • The README requires Python 3.6 or later and PyTorch 1.4.0 or later for its stated setup.
  • The repository says training currently requires a GPU.
  • The README cautions that all integer operations were executed with floating point, so the stated integer-only goal does not establish integer execution or deployment performance.
  • The repository links to the Hugging Face Transformers implementation and ibert-roberta-base, ibert-roberta-large and ibert-roberta-large-mnli checkpoints.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare a GLUE dataset for the I-BERT workflow

Download and preprocess a selected GLUE task dataset for use with the repository’s RoBERTa workflow.

Before you start

  • Python 3.6 or later and PyTorch 1.4.0 or later.
  • A local clone of the I-BERT repository and a GPU for training, as specified by the README.
  • A GLUE task name supported by the preprocessing script.

Steps

  1. Clone the I-BERT repository and install it in editable mode with the README’s Fairseq-based setup.

  2. Download GLUE data into the repository root using the documented download script and command.

  3. Run the documented GLUE preprocessing script with a supported task name, or use ALL for the listed tasks.

Expected outcome

The preprocessed dataset is placed in the repository’s task-specific bin directory, ready for the subsequent model workflow.

Guide limitations

  • This guide covers data preparation only, not model training, quantization or inference.
  • The README says I-BERT currently supports training on GPU.
  • The README notes that all integer operations were executed with floating point.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
kssteven418/I-BERT · Official homepage title · retained in its source language
How we review sourcesSuggest a correction