OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Parsbigbird

Parsbigbird is an open-source BigBird language model trained for Persian text, using sparse attention to handle sequences up to 4,096 tokens. Its repository includes examples for contextualized embeddings and fill-mask prediction, plus a linked classification fine-tuning notebook.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Parsbigbird — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Parsbigbird when a Persian NLP task needs more context than a 512-token input, or when you want to explore the repository's embedding, fill-mask or classification examples.

Who it is for

  • Developers and researchers building Persian-language NLP workflows.
  • Teams exploring classification or question answering over longer Persian passages.
  • Users who want to fine-tune a Persian model for a text-classification task.

Documented uses

  • Create contextualized embeddings from Persian text.
  • Generate a fill-mask prediction for a Persian sentence.
  • Fine-tune the model for text classification using the linked notebook.
  • Experiment with longer Persian inputs for summarization or question answering.
Before you choose

Requirements & limits

The model repository is public and lists an AGPL-3.0 license. The README links to a Google Colab fine-tuning notebook and a Hugging Face model page.

  • The model supports sequences up to 4,096 tokens, while the repository reports evaluation at a maximum length of 2,048 tokens.
  • The README recommends gradient checkpointing and gradient accumulation for fine-tuning because of memory requirements.
  • For inputs up to 512 tokens, the README recommends original_full attention rather than block-sparse attention.
  • Reported evaluation results differ across tasks; the repository's results should not be assumed to predict performance on other data.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Load ParsBigBird for Persian text embeddings

Load the pretrained model and tokenizer to produce contextualized embeddings for a Persian input.

Before you start

  • Python environment with the Hugging Face Transformers package
  • Access to the SajjadAyoubi/distil-bigbird-fa-zwnj model

Steps

  1. Import BigBirdModel and AutoTokenizer from transformers.

  2. Load the model with BigBirdModel.from_pretrained("SajjadAyoubi/distil-bigbird-fa-zwnj", block_size=32).

  3. Load the matching tokenizer, tokenize Persian text with return_tensors='pt', and pass the tokens to the model.

Expected outcome

The model returns contextualized embeddings for the tokenized input.

Guide limitations

  • The README does not specify a complete environment setup or hardware requirements.
  • The repository recommends original_full attention for inputs up to 512 tokens; its sparse-attention configuration is intended for longer sequences.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
sajjjadayobi/ParsBigBird · Official homepage title · retained in its source language
How we review sourcesSuggest a correction