OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Xlnet

Xlnet is the GitHub repository for XLNet, an unsupervised language-representation model using generalized permutation language modeling with a Transformer-XL backbone. It includes TensorFlow code and released cased checkpoints for fine-tuning or building applications for language understanding tasks.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Xlnet — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need XLNet’s model implementation and checkpoints for a supported language-understanding task, and can work with its TensorFlow-era code and compute requirements.

Who it is for

  • Developers adapting a language model to question answering, text classification or document ranking.
  • Researchers exploring XLNet representations or pretraining code.

Documented uses

  • Fine-tune XLNet for question answering or classification and regression tasks.
  • Use XLNet representations in custom NLP applications.
  • Explore pretraining workflows using text data.
Before you choose

Requirements & limits

The public GitHub repository is licensed under Apache-2.0. The README describes cased XLNet-Base and XLNet-Large checkpoint releases.

  • The implementation documents Python 2 and TensorFlow 1.13.1 testing as of June 2019; this does not establish compatibility with current environments.
  • Large-model fine-tuning can be memory-intensive: the repository reports a single sequence at length 512 on a 16 GB GPU for XLNet-Large with its default setup.
  • Released checkpoint archives contain TensorFlow weights, a SentencePiece model and a JSON configuration.
  • The repository includes task scripts and a model abstraction; adapting workflows depends on the selected task and available hardware.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare XLNet inputs and obtain model representations

Tokenize text with the SentencePiece model bundled with a released XLNet checkpoint, then construct the model and retrieve a pooled or sequence representation.

Before you start

  • Python environment with the repository’s TensorFlow-based XLNet code.
  • A released XLNet checkpoint ZIP containing its TensorFlow checkpoint, SentencePiece model, and configuration file.
  • Initialized repository FLAGS and TensorFlow input tensors: input_ids, seg_ids, and input_mask.

Steps

  1. Load the bundled SentencePiece model with SentencePieceProcessor and preprocess text using the configured uncasing option.

  2. Encode the preprocessed text into IDs with encode_ids.

  3. Create XLNetConfig from the checkpoint configuration path and create a run configuration for the intended training or fine-tuning mode.

  4. Construct XLNetModel with the configuration, run configuration, and input tensors; retrieve a pooled summary with get_pooled_out(summary_type="last") or sequence representations with get_sequence_output().

Expected outcome

The code yields XLNet pooled or sequence representations that can serve as the basis for an application.

Guide limitations

  • The README dates the tested setup to TensorFlow 1.13.1 under Python 2; it does not establish compatibility with other versions.
  • The code example omits initialization details for FLAGS and tensors, so this is not a complete runnable program.
  • The released model archives described in the README are cased models.
  • The repository reports substantial memory constraints when fine-tuning XLNet-Large on GPUs.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
zihangdai/xlnet · Official homepage title · retained in its source language
How we review sourcesSuggest a correction