OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Xgen

Xgen is Salesforce AI Research’s open-source family of 7-billion-parameter language models, with base variants for 4K and 8K sequence lengths and an instruction-finetuned variant. The repository documents loading model weights and a tokenizer with Transformers and generating text with PyTorch.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Xgen — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Xgen when you want to experiment with the released model weights and can run the documented Transformers and PyTorch loading and generation workflow.

Who it is for

  • Researchers experimenting with language models and autoregressive text generation.
  • Developers who want to load and run a 7B model in a self-hosted setup.

Documented uses

  • Generate text from a prompt using a base model.
  • Experiment with longer input sequences using the 8K base variant.
  • Compare the 4K and 8K base models.
  • Explore an instruction-finetuned model for research.
Before you choose

Requirements & limits

The repository is public and specifies an Apache-2.0 license. The models are published on the Hugging Face Hub; the repository example loads them by model identifier.

  • The base variants differ in supported sequence length: 4K versus 8K.
  • The instruction-finetuned variant is identified as for research purposes only.
  • The documented workflow uses Transformers, PyTorch, and Tiktoken; the example also sets bfloat16 and enables remote tokenizer code.
  • Salesforce describes the release as research support and recommends evaluating accuracy, safety, and fairness for downstream uses.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Generate text with an XGen base model

Load the 8K base model and tokenizer, generate text from a prompt, and decode the sample.

Before you start

  • Python environment with PyTorch and Hugging Face Transformers
  • OpenAI Tiktoken package, used for tokenization
  • A setup able to load the model weights with bfloat16, as shown in the repository example

Steps

  1. Install Tiktoken with the command below.

    Documented command
    pip install tiktoken

  2. Load the Salesforce/xgen-7b-8k-base tokenizer using Transformers and trust_remote_code=True.

  3. Load the model with AutoModelForCausalLM.from_pretrained using torch_dtype=torch.bfloat16.

  4. Tokenize a prompt, call model.generate with a maximum output length, and decode the generated sample.

Expected outcome

A decoded text sample generated from the prompt using the 8K base model.

Guide limitations

  • The repository example uses the 8K base model; it does not establish hardware requirements or performance.
  • The release is for research purposes, and Salesforce recommends evaluating accuracy, safety, and fairness before downstream deployment.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
7 November 2026
Original website title
salesforce/xgen · Official homepage title · retained in its source language
How we review sourcesSuggest a correction