AI · Agents · Blockchain · Crypto

Submit a project Sign in

Menu

Directory category

AI Evaluation & Observability

Evaluate and monitor AI applications through tests, traces and performance diagnostics. Compare official resources, requirements and limitations.

128 projects

Directory results

AI Evaluation & Observability projects

O Openllmetry Openllmetry is an open-source Python library of OpenTelemetry extensions that instruments LLM applications and captures telemetry, including standard instrumentations for LLM providers and vector databases.

W What If Tool What If Tool is an interactive interface for investigating trained classification and regression models.

S Shap — shap SHAP is a Python library that applies game-theoretic Shapley values to explain machine-learning outputs by attributing contributions to input features.

C Captum Captum is an open-source, extensible library built on PyTorch that applies interpretability algorithms, including Integrated Gradients, to help identify which features contribute to model outputs.

MLCommons MLCommons is an AI engineering consortium that develops benchmarks and data tooling for measuring AI system quality, performance and risk, including accuracy, speed, efficiency and harmful-response reliability.

B Beam Beam is an open-source research benchmark and Python repository for evaluating large language models on long-term memory and long-context reasoning.

A Audit AI Audit AI is an open-source Python library built on pandas and scikit-learn for assessing demographic differences in machine-learning predictions and other assessments.

M Model Analysis Model Analysis is an open-source Python library, TensorFlow Model Analysis (TFMA), for evaluating TensorFlow models on large datasets in a distributed manner.