OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Beam

Beam is an open-source research benchmark and Python repository for evaluating large language models on long-term memory and long-context reasoning. Its dataset pairs multi-scale, multi-domain conversations with questions designed to test ten memory abilities.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Beam — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Beam when an evaluation needs to test more than short, simple recall: its conversations range from 128K to 10M tokens, and its questions cover ten memory abilities.

Who it is for

  • Researchers comparing language models on long-conversation memory tasks.
  • Teams creating benchmark-based evaluations for memory retention and long-context reasoning.
  • Developers interested in generating multi-turn conversational datasets with the repository’s documented pipeline.

Documented uses

  • Compare model performance on memory questions across conversations of different lengths.
  • Assess abilities such as contradiction resolution, event ordering, knowledge updating and preference following.
  • Generate conversations and probing questions for benchmark or dataset work.
Before you choose

Requirements & limits

The public GitHub repository specifies an MIT license. The documented generation pipeline requires model configuration details, including a URL, name and API key.

  • The benchmark has 100 conversations and 2,000 validated questions, distributed across 128K, 500K, 1M and 10M token conversation sizes.
  • Questions cover ten abilities, from information extraction and summarization to abstention, contradiction resolution and temporal reasoning.
  • The repository describes LIGHT as a separate memory-augmented framework with episodic memory, working memory and a scratchpad.
  • Conversation generation depends on configuring a language model; the existing dataset can serve as the evaluation resource without treating Beam as an LLM service.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Configure Beam for conversation generation

Prepare the repository’s documented LLM configuration and pipeline inputs for generating conversations.

Before you start

  • A local copy of the repository and Python dependencies installed with pip install -r requirements.txt.
  • LLM model URL, model name and API key.
  • Chat seed information, using the repository’s topics directory as an example.

Steps

  1. Add the model URL, name and API key to src/llms_config.json.

  2. Prepare chat seed information.

  3. Use the documented pipeline stages in order: plan, question, then answer, with the required model and chat parameters.

Expected outcome

The documented setup prepares the inputs for Beam’s three-stage multi-turn conversation-generation pipeline.

Guide limitations

  • This guidance covers configuration and pipeline stages; it does not establish that a run completed successfully.
  • The repository documentation does not specify resource requirements.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
mohammadtavakoli78/BEAM · Official homepage title · retained in its source language
How we review sourcesSuggest a correction