---
title: "zoto-eval-system"
description: "Generate, run, and consciously update static (pytest) + LLM (@cursor/sdk) evals when AI primitives change, every proposed change requires user confirmation. YAML+JSONSchema results, surgical update on git diff, sovereign"
canonical: https://agentpluginsdirectory.com/plugins/zoto-eval-system
last-updated: 2026-10-23
---

# zoto-eval-system
Generate, run, and consciously update static (pytest) + LLM (@cursor/sdk) evals when AI primitives change, every proposed change requires user confirmation. YAML+JSONSchema results, surgical update on git diff, sovereign user-authored cases, /canvas-rendered cross-run compare.
- Slug: zoto-eval-system
- Publisher: zotoio
- Repository: https://github.com/zotoio/zoto-agents
- Manifest: plugins/zoto-eval-system/plugin.json
- Version: 0.3.1
- License: MIT
- Category (editorial): other
- Skills: 9 (zoto-advise-evals, zoto-compare-evals, zoto-configure-evals, zoto-create-evals, zoto-eval-tooling, zoto-execute-evals, zoto-help-evals, zoto-judge-evals, zoto-update-evals)
- MCP servers: 0
- Stars: 0
- Repository created: 2026-04-06
- Repository last pushed: 2026-10-23
- Publisher type: Organization
- Listing: https://agentpluginsdirectory.com/plugins/zoto-eval-system
- Schema: https://agent-plugins.org/schemas/1.0.0/plugin.schema.json

## What zoto-eval-system does, in the publisher's words

Generate, run, and consciously update evals when AI primitives change, for any repository.

The Eval System scaffolds two eval backends side by side: a static backend (pytest, vitest, or jest per static.framework) for fast, deterministic checks, and an LLM backend powered by @cursor/sdk for agent-based evaluations with soft metrics, tokens, duration, verbosity, accuracy, confidence. A diff-aware updater detects when covered targets have changed and presents each proposed eval update for user confirmation before writing, behavioural drift in AI primitives is managed consciously, never silently. User-authored cases stay sovereign: both runtime and compile-time guards refuse to touch them.

> Note: An earlier name for the continuous evaluation surface was eval:live / _live. The plugin now ships eval:update / _update throughout. The rename is complete everywhere except this single footnote.

From the project README, punctuation lightly normalized. Full text: https://raw.githubusercontent.com/zotoio/zoto-agents/HEAD/plugins/zoto-eval-system/README.md