OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Deepmark

Deepmark is an open-source LLM assessment tool for comparing language models on user-provided data with task-specific measures such as accuracy, latency, cost and failure rate.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Deepmark — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Deepmark when you have task-specific data and want to compare language models against measures relevant to a generative-AI application.

Who it is for

  • Generative-AI developers choosing among language models for a defined application task.
  • Teams seeking to assess candidate models against their own examples and selected performance measures.

Documented uses

  • Compare candidate LLMs on task-specific data.
  • Assess model performance on tasks including question answering, classification, named entity recognition, summarization and sentiment analysis.
  • Review trade-offs involving accuracy, relevance, latency, cost and failure rate.
Before you choose

Requirements & limits

The project is identified as open source under the AGPL-3.0 license. Repository setup documentation describes Docker and GitHub-based routes.

  • The documented workflow is aimed at evaluation on user-provided data, so the value of comparisons depends on having relevant examples and task criteria.
  • The README lists prebuilt integrations with GPT-4, GPT-3.5 Turbo, Anthropic, Cohere, AI21 and others.
  • The repository setup notes specify Laravel for the application and Node.js with npm for frontend preparation.
  • The repository lists PHP as its primary language and AGPL-3.0 as its license.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare a local Deepmark installation

Follow the repository’s documented GitHub setup and frontend preparation steps.

Before you start

  • Laravel
  • Node.js and npm; the README identifies Node.js 16.16.0 as its stable version
  • A configured BEARER_TOKEN in the .env file

Steps

  1. Install Laravel.

  2. Run the documented storage link, queue table and migration setup steps: php artisan storage:link, php artisan queue:table, and php artisan migrate.

  3. Set BEARER_TOKEN in .env; the README says to use that token in the X-Bearer-Token HTTP header.

  4. From the project root, run npm i, then choose npm run dev or npm run build for the frontend.

Expected outcome

The documented Laravel and frontend setup stages are prepared; the README does not give a full evaluation execution workflow.

Guide limitations

  • These instructions are documentation, not a verified deployment or compatibility guarantee.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
IngestAI/deepmark · Official homepage title · retained in its source language
How we review sourcesSuggest a correction