OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Factool

Factool is a Python framework that evaluates factual errors in generative AI outputs across knowledge-based question answering, code generation, mathematical reasoning and scientific literature review.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Factool — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use FacTool when you have a prompt and generated response to assess in one of its four supported task categories and can provide the API keys required for that category.

Who it is for

  • Developers evaluating factuality in language-model responses.
  • Researchers assessing generated answers, code, mathematical reasoning or scientific references.

Documented uses

  • Assess claims and response-level factuality in knowledge-based answers.
  • Detect execution errors in generated code and calculation errors in mathematical reasoning.
  • Identify hallucinated scientific literature in generated reviews.
Before you choose

Requirements & limits

The repository identifies the project as open source under the Apache-2.0 license.

  • An OpenAI API key is required for all supported task categories.
  • Knowledge-based QA also needs a Serper API key; scientific literature review also needs a Scraper API key.
  • The documented foundation-model options are GPT-3.5-turbo and GPT-4.
  • The repository provides both a Python package installation route and developer installation instructions.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare inputs for a FacTool evaluation

Structure prompts and generated responses for one of the four supported evaluation categories.

Before you start

  • Python
  • An OpenAI API key
  • A Serper API key for knowledge-based QA
  • A Scraper API key for scientific literature review

Steps

  1. Install the package with the command below, or use the repository's developer installation instructions.

    Documented command
    pip install factool

  2. Set OPENAI_API_KEY; also set SERPER_API_KEY for knowledge-based QA or SCRAPER_API_KEY for scientific literature review.

  3. Create input records with prompt, response and a category value of kbqa, code, math or scientific.

  4. For code-generation inputs, include entry_point with the function name of the code snippet.

Expected outcome

Inputs are organized in the format FacTool's quick start describes for its evaluation workflow.

Guide limitations

  • This guide covers installation and input preparation only; it does not describe running an evaluation.
  • The OpenAI key is required for every task, while Serper and Scraper API keys apply only to their respective categories.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
GAIR-NLP/factool · Official homepage title · retained in its source language
How we review sourcesSuggest a correction