OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Sgi Bench

Sgi Bench is an MIT-licensed Python benchmark project for evaluating large language models on scientific general-intelligence tasks through scientist-aligned workflows. It covers research, idea generation, dry and wet experiments, and experimental reasoning across a four-stage inquiry cycle.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Sgi Bench — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Sgi Bench when you need a structured evaluation of model behavior across multiple stages and task types in scientific inquiry, rather than a general-purpose model assessment.

Who it is for

  • Researchers comparing large language models on scientific tasks
  • Teams studying evaluation methods for AI-assisted scientific work
  • Developers using Python-based benchmark and evaluation workflows

Documented uses

  • Evaluate model performance on scientific deep-research tasks involving retrieval and synthesis.
  • Compare structured scientific idea-generation behavior.
  • Assess code generation and lab-protocol development in experimental tasks.
  • Evaluate reasoning about experimental processes, observations, simulations and visualizations.
Before you choose

Requirements & limits

The project is publicly available on GitHub under the MIT license. Its Hugging Face datasets are gated; the README says access requires agreeing to share contact information and setting a read-only token as HF_TOKEN.

  • The benchmark covers 10 disciplines and more than 1,000 expert-curated samples.
  • Different task families use distinct evaluation scripts and, for some tasks, separate Python environments.
  • The documented main setup uses an OpenAI API key and base URL; model-provider flexibility is not specified in the provided setup instructions.
  • Dataset access requires a Hugging Face token and acceptance of the dataset access condition.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the main SGI-Bench evaluation environment

Clone the repository, configure the documented API and dataset credentials, and install the main evaluation dependencies.

Before you start

  • Git and Conda
  • Python 3.13.7 environment specified in the quick start
  • OpenAI API key and base URL
  • Hugging Face read-only token; dataset access requires agreeing to share contact information

Steps

  1. Clone https://github.com/InternScience/SGI-Bench.git and change to the evaluation directory.

  2. Set OPENAI_API_KEY and OPENAI_BASE_URL, and set HF_TOKEN to a read-only Hugging Face token after accepting the dataset access condition.

  3. Create and activate the Conda environment with Python 3.13.7.

  4. Install the dependencies with pip install -r requirements.txt.

Expected outcome

The repository's documented main evaluation environment and credentials are configured; this setup does not itself run a benchmark task.

Guide limitations

  • The repository documents task-specific environments separately; the main setup alone does not install or run every task.
  • This setup guidance does not verify successful installation or compatibility.
Documentation 1 ↗
Source-based practical guide

Run the documented deep-research evaluation steps

Obtain model answers for the deep-research task and score them using the repository's documented scripts.

Before you start

  • The main SGI-Bench evaluation environment and its documented dependencies
  • OPENAI_API_KEY and OPENAI_BASE_URL configured
  • Hugging Face dataset access and HF_TOKEN configured

Steps

  1. Activate the sgi Conda environment.

  2. Run python task_1_deep_research/step_1_get_answer.py gpt-5.2-pro.

  3. Run python task_1_deep_research/step_2_score.py gpt-5.2-pro.

Expected outcome

The documented answer-generation and scoring stages for the deep-research task have been invoked for the specified model identifier.

Guide limitations

  • These are documented steps, not independently verified execution results.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
15 October 2026
Original website title
InternScience/SGI-Bench · Official homepage title · retained in its source language
How we review sourcesSuggest a correction