OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Evaluation & Observability

Skill Up

Skill Up is a Go command-line tool for evaluating agent skills, agents, and workspaces. It runs declarative cases against an Agent Engine, grades responses and workspace changes, and produces reports for local development or CI.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Skill Up — OpenViST project profile in Agent Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when agent behavior needs repeatable, inspectable evaluation cases and structured results, particularly for Skill comparisons, workspace tasks, and CI reporting.

Who it is for

  • Developers evaluating an Agent Skill across repeatable cases.
  • Teams comparing an agent’s behavior with and without a Skill.
  • Engine and agent builders assessing an agent without installing a Skill.
  • Teams checking agent work against repository fixtures or an existing local workspace.

Documented uses

  • Define evaluation environments, engines, models, and cases in YAML.
  • Grade agent responses and workspace changes with rule-based, script, or agent-judge strategies.
  • Compare Skill-enabled and Skill-free runs.
  • Generate JSON, Markdown, JUnit XML, or HTML evaluation reports for review or CI.
  • Import or auto-detect Anthropic-compatible evals.json files.
Before you choose

Requirements & limits

The project is an Apache-2.0 licensed Go CLI. Its README specifies Go 1.25 or later.

  • Choose among Qoder CLI, Claude Code, Codex, or a user-defined agent using the documented custom-engine configuration.
  • Select local, Docker, or OpenSandbox runtime provisioning as appropriate to the evaluation configuration.
  • Choose rule-based, script, or agent-judge grading for each evaluation.
  • For an existing local workspace, the documented mode requires environment.type none, one case at a time, and benchmark mode off; the agent and case setup can change workspace contents.
  • Skill-upper provides a conversational evaluation-and-iteration workflow, while skill-up performs the configured evaluations and produces reports.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install skill-up and run a first evaluation

Install the CLI, define a simple Skill evaluation, validate it, and run the case.

Before you start

  • Go 1.25 or later and a skill-up checkout for a local build, or access to the documented install script.
  • A Skill project containing SKILL.md.
  • An Agent Engine configured in the evaluation file.

Steps

  1. Install from the documented script, or build locally from a checkout using the command below or the command below.

    Documented command
    make build
    Documented command
    go build -o bin/skill-up ./cmd/skill-up

  2. Create evals/eval.yaml with schema_version: v1alpha1, an environment setting, an engine name, and a case file reference. For a plain-text Skill, the guide shows environment: { type: none } and engine: { name: claude_code }.

  3. Create a case YAML with an input.prompt and expected-output rules such as expect.must_contain.

  4. Run skill-up validate to check the configuration and referenced cases, then run skill-up run to execute the evaluation.

Expected outcome

The configuration is checked and the evaluation runs against the configured Agent Engine; skill-up writes a result report in its output directory.

Guide limitations

  • The example engine is Claude Code; an appropriate configured Agent Engine is required for execution.
  • The default run settings include JSON report output, a 300-second timeout, 10 maximum turns, and parallelism of 1.
  • A successful validation checks configuration and case files but does not itself start an Agent Engine.
Documentation 1 ↗Documentation 2 ↗
Source-based practical guide

Validate and run an agent-only evaluation

Evaluate an Agent Engine without installing a Skill by explicitly configuring an empty Skill list.

Before you start

  • The skill-up CLI and an Agent Engine configured in the eval file.
  • An evals/ directory containing an evaluation config and case file.

Steps

  1. Create evals/eval.yaml with schema_version: v1alpha1, environment: { type: none }, skills: [], an engine name, and a reference to the case file.

  2. Create the referenced case file with a prompt and expected-output check, such as must_contain.

  3. Validate the suite with skill-up validate ./evals/eval.yaml.

  4. Run it with skill-up run ./evals/eval.yaml.

Expected outcome

skill-up runs the declared case without installing a Skill and produces evaluation results.

Guide limitations

  • Set skills: [] explicitly; when omitted, skill-up can automatically install a discovered SKILL.md.
  • For an existing local workspace, --workspace is supported only with environment.type: none, parallelism 1, and benchmark mode disabled; the agent and case setup may change the directory.
Documentation 1 ↗Documentation 2 ↗
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
11 October 2026
Original website title
alibaba/skill-up · Official homepage title · retained in its source language
How we review sourcesSuggest a correction