---
title: "plugin-eval"
description: "Three-layer quality evaluation framework for agent plugins and skills (Agent Skills / Agent Plugins 1.0) with Elo ranking"
canonical: https://agentpluginsdirectory.com/plugins/plugin-eval
last-updated: 2026-09-09
---

# plugin-eval
Three-layer quality evaluation framework for agent plugins and skills (Agent Skills / Agent Plugins 1.0) with Elo ranking
- Slug: plugin-eval
- Publisher: Owen Ayres
- Repository: https://github.com/oayres/agent-plugins
- Manifest: plugins/plugin-eval/plugin.json
- Version: 0.1.2
- License: MIT
- Category (editorial): other
- Skills: 1 (evaluation-methodology)
- MCP servers: 0
- Stars: 0
- Repository created: 2026-09-08
- Repository last pushed: 2026-09-08
- Publisher type: User
- Listing: https://agentpluginsdirectory.com/plugins/plugin-eval
- Schema: https://agent-plugins.org/schemas/1.0.0/plugin.schema.json

## What plugin-eval does, in the publisher's words

Three-layer quality evaluation framework for agent plugins and skills, anything built to the Agent Skills spec (SKILL.md + frontmatter) or the Agent Plugins 1.0 layout (plugin.json + skills/). It scores what the host agent sees: does the description trigger correctly, are the instructions a clean composable worker, is the content sized and structured so any client can load it efficiently, and does it avoid patterns that only work in one harness.

The evaluator is a standalone Python CLI. It works on any plugin directory regardless of which agent you author for; the optional LLM layers use the Claude Agent SDK as their judge backend.

From the project README, punctuation lightly normalized. Full text: https://raw.githubusercontent.com/oayres/agent-plugins/HEAD/plugins/plugin-eval/README.md

## Skills

- evaluation-methodology: PluginEval quality methodology: dimensions, rubrics, statistical methods, and scoring formulas. Use this skill when understanding how plugin quality is measured, when interpreting a low score on a specific dimension, when deciding how to improve a skill's triggering accuracy or orchestration fitnes…

Descriptions come from the frontmatter of each SKILL.md, punctuation lightly normalized.