OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Developer Tools & SDKs

Codebook Features

Codebook Features is a Python library for training and interpreting neural networks with learned sparse, discrete hidden states. It inserts vector-quantization bottlenecks called codebooks into network layers and includes tools to analyze code activations, find codes associated with patterns, and intervene on codes to study effects on model output.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Codebook Features — OpenViST project profile in AI Developer Tools & SDKs.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need a codebook-based workflow to inspect model features or study how selected internal codes affect generated output.

Who it is for

  • Researchers investigating interpretability and internal representations in neural networks.
  • Developers training or analyzing codebook models, particularly language models.

Documented uses

  • Train a neural network with learned sparse, discrete internal representations.
  • Analyze code activations and search for codes that respond to a pattern.
  • Intervene on a code during a forward pass and examine its effect on model output.
Before you choose

Requirements & limits

The project is open source under the MIT license. The README documents installation from PyPI or source and links to a Colab intervention tutorial and a Hugging Face Space.

  • The browser workflow requires both a saved codebook model and locally generated cache files.
  • Training configuration uses Hydra and builds on Hugging Face Transformers training components.
  • The repository includes model classes for GPT-2, GPT-Neo and GPT-NeoX.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare a codebook model for the local browser

Generate the cache files and launch the interpretability browser for a saved codebook model.

Before you start

  • A trained codebook model saved on disk.
  • The original model name or path, dataset name and dataset configuration name.

Steps

  1. Run the command below with --orig_model_name, --pretrained_path, --dataset_name, --dataset_config_name and --output_base_dir set to the relevant model, dataset and output paths.

    Documented command
    python -m codebook_features.code_search_cache

  2. Launch the browser with the command below, using the base output directory from the cache-generation step.

    Documented command
    python -m streamlit run codebook_features/webapp/Code_Browser.py -- --cache_dir

Expected outcome

A local webapp for visualizing the codebook model using its generated cache.

Guide limitations

  • This workflow starts from an already trained and saved model; it does not cover model training.
  • The browser requires cache files generated for the model.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
taufeeque9/codebook-features · Official homepage title · retained in its source language
How we review sourcesSuggest a correction