OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Xlingeval

Xlingeval is an open-source Python research project with code and benchmark resources for evaluating large language models on cross-lingual healthcare queries. Its framework assesses correctness, consistency and verifiability, combining automated measures with human evaluation.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Xlingeval — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Xlingeval when a research project needs a cross-lingual healthcare benchmark and a framework for examining answer correctness, consistency and verifiability.

Who it is for

  • Researchers studying multilingual language-model performance in healthcare.
  • Teams comparing model responses across languages using benchmark data and evaluation criteria.

Documented uses

  • Assess whether generated healthcare answers match expert ground truth.
  • Compare response consistency across repeated sampling.
  • Evaluate the verifiability of medical claims.
  • Use XLingHealth data to study responses across four languages and three health Q&A datasets.
Before you choose

Requirements & limits

The public repository is open source under Apache-2.0. The project identifies Python 3.9; GPT model access is through the OpenAI API, while MedAlpaca is identified as open source and available through Hugging Face.

  • Evaluation covers correctness, consistency and verifiability rather than a single score.
  • The benchmark includes three health Q&A datasets and four languages; Spanish, Chinese and Hindi translations are described as machine-translated with human verification.
  • The documented model coverage includes GPT-3.5, GPT-4 and MedAlpaca 7B, 13B and 30B.
  • The repository includes benchmark spreadsheets, with a separate Hugging Face dataset listing.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install XLingEval’s Python environment

Create the documented Conda environment and install the repository’s dependencies.

Before you start

  • Conda
  • Python 3.9

Steps

  1. Create an environment with Python 3.9: conda create -n xlingeval python=3.9.

  2. Activate it: conda activate xlingeval.

  3. From the repository, install dependencies: the command below.

    Documented command
    pip install -r requirements.txt

Expected outcome

The `xlingeval` Conda environment is set up with the project dependencies installed.

Guide limitations

  • This covers environment setup only; it does not run an evaluation.
Documentation 1 ↗
Source-based practical guide

Evaluate answer correctness with GPT-3.5

Retrieve GPT-3.5 answers for a dataset and evaluate them against ground-truth answers.

Before you start

  • The XLingEval repository and its dependencies are available.
  • A dataset path is available.

Steps

  1. Retrieve answers by running the command below.

    Documented command
    python correctness/correctness_get_gpt_answer.py --dataset_path --model gpt-3.5-turbo

  2. Evaluate answer quality by running the command below.

    Documented command
    python correctness/correctness_answer_evaluation.py --dataset_path --model gpt-3.5-turbo

Expected outcome

The repository’s correctness evaluation workflow is invoked for GPT-3.5 answers and the supplied dataset.

Guide limitations

  • The instructions do not specify credential setup, dataset preparation, or result locations for this workflow.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
8 October 2026
Original website title
claws-lab/XLingEval · Official homepage title · retained in its source language
How we review sourcesSuggest a correction