AI · Agents · Blockchain · Crypto

Submit a project Sign in

Menu

Directory category

AI Evaluation & Observability

Evaluate and monitor AI applications through tests, traces and performance diagnostics. Compare official resources, requirements and limitations.

128 projects

Directory results

AI Evaluation & Observability projects

C Clue CLUE is a Chinese language-understanding benchmark resource that provides representative task datasets, pretrained-model baselines, corpora and leaderboard results for comparing systems.

T Tealeaves TeaLeaves is a Python pipeline for inspecting how Hugging Face language models process prompts.

L logfire-observer logfire-observer is a callback-style bridge that collects Hugging Face Transformers training events, enriches them with contextual information and emits structured telemetry to Logfire using OpenTelemetry primitives.

L Logfire Insight Relay Logfire Insight Relay is a callback-oriented telemetry bridge for Hugging Face Transformers training pipelines.

S Smallevals Smallevals is a Python framework for locally evaluating RAG retrieval.

N Nlg Eval Nlg Eval is a Python library and command-line tool for calculating automated natural-language-generation metrics from generated text and aligned reference examples.

S Sdgym SDGym is a Python benchmarking framework for modeling and generating synthetic data, with support for comparing synthesizers, datasets and evaluation metrics.

L Lamp — LaMP-Benchmark LaMP is a research benchmark and Python codebase for training and evaluating language models that produce personalized outputs.