OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Knn Transformers

Knn Transformers is a Python research implementation of k-nearest-neighbor language models, k-nearest-neighbor machine translation and RetoMaton using Hugging Face Transformers and PyTorch.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Knn Transformers — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it for research or prototyping when you want to explore kNN-LM, kNN-MT or RetoMaton in a Transformers-based Python workflow.

Who it is for

  • Researchers experimenting with retrieval-based language modeling.
  • Developers studying kNN methods for machine translation.
  • Python practitioners who want to incorporate the implementations into a Transformers project.

Documented uses

  • Add datastore-based nearest-neighbor retrieval to a language model.
  • Explore retrieval methods for encoder-decoder machine-translation models.
  • Experiment with RetoMaton's clustered datastore and pointer-based retrieval.
  • Evaluate a base model and prepare a datastore and FAISS index for kNN-LM experiments.
Before you choose

Requirements & limits

The repository is public and specifies the MIT license.

  • Datastores and indexes depend on the language model used to create them, so model and artifacts must correspond.
  • FAISS is a project dependency; the README gives a separate Anaconda installation instruction for macOS.
  • The repository supplies standalone Python implementation files and example scripts rather than a hosted interface.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare a Wikitext-103 kNN-LM datastore and FAISS index

Create or obtain the datastore and index used by the repository's documented GPT-2 language-model workflow.

Before you start

  • The repository and its documented Python requirements.
  • FAISS; the README specifies installing faiss-cpu through Anaconda on macOS.
  • A fine-tuned model such as neulab/gpt2-finetuned-wikitext103 and the Wikitext-103 dataset.

Steps

  1. Evaluate the selected base model on the Wikitext-103 validation set using run_clm.py.

  2. Download the matching preprocessed datastore, or run the documented training-set evaluation with --save_knnlm_dstore to create one.

  3. Download the matching FAISS index or follow the repository's index-building instructions.

Expected outcome

A datastore and matching index prepared for the subsequent kNN-LM evaluation workflow.

Guide limitations

  • The datastore and index depend on the model used to create them; use matching artifacts.
  • The available instructions include a macOS-specific FAISS installation note. Consult the repository for the complete index-building and evaluation procedures.
Documentation 1 ↗Documentation 2 ↗
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
neulab/knn-transformers · Official homepage title · retained in its source language
How we review sourcesSuggest a correction