Practical overview
When it fits
Use Evidently when an AI or data pipeline needs repeatable evaluation before release, exploratory analysis during development, or ongoing checks for drift and quality changes.
Who it is for
- Data scientists analysing datasets and model behaviour
- ML engineers building validation and monitoring workflows
- Teams testing LLM prompts, outputs and RAG systems
- Developers integrating custom Python metrics into AI evaluation
Documented uses
- Measure data drift and data quality changes
- Evaluate predictive and generative AI tasks
- Check LLM text outputs with built-in or custom metrics
- Turn evaluation results into pass/fail regression tests
- Monitor metrics and test results over time
Requirements & limits
The open-source framework is available for self-hosted use. The project also documents Evidently Cloud as a managed option with additional features; the supplied material does not establish account, pricing or availability details for either option.
- Python is the central integration language.
- Reports can be used for offline analysis, while the monitoring UI supports viewing metrics and test results over time.
- Built-in presets and metrics shorten setup for common evaluations; custom metrics support application-specific checks.
- The framework supports both self-hosting and a separately documented cloud option.
- Results can be exported as HTML, JSON or Python dictionaries for use in other workflows.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Create a data-drift report from tabular data
Generate a drift evaluation and save the result as an HTML file.
Before you start
- Python environment with Evidently installed from PyPI or Conda.
- Pandas and scikit-learn data available.
- A reference dataset and a current dataset.
Steps
Install Evidently with pip or Conda.
Load the tabular data into a Pandas-compatible dataframe.
Create a Report with the DataDriftPreset and run it on current and reference data.
Save the evaluation result as an HTML file or export it as JSON or a Python dictionary.
Expected outcome
A data-drift evaluation report with test output that can be viewed as HTML or returned in structured formats.
Guide limitations
- The documented example uses the first and later rows of the Iris dataset as current and reference data; dataset selection and interpretation remain the user's responsibility.
- This recipe covers report generation and export, not deployment of a production monitoring service.
Build a basic LLM text-evaluation report
Score answer text with built-in descriptors and summarize the results in an Evidently Report.
Before you start
- Python with Evidently installed.
- Pandas.
- A dataset containing question and answer text columns.
Steps
Create a Pandas dataframe containing questions and answers.
Create an Evidently Dataset and attach descriptors for sentiment, text length and denial-related terms.
Create a Report with the TextEvals preset and run it on the Evidently dataset.
View the result or export it as JSON or a Python dictionary.
Expected outcome
A summary of the selected text-evaluation scores for the answer data.
Guide limitations
- The documented example is a basic text-evaluation workflow; more advanced LLM-as-a-judge configuration is mentioned but not fully specified there.
- This recipe does not establish the quality or correctness of the generated answers.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 2 October 2026
- Original website title
- evidentlyai/evidently · Official homepage title · retained in its source language