---
title: "skill-tuner"
description: "Tells you whether a change to an agent-consumed document actually worked: paired comparison with a confidence interval and a non-inferiority margin, content-hashed provenance with drift detection, and an adversarial defe"
canonical: https://agentpluginsdirectory.com/plugins/skill-tuner
last-updated: 2026-09-25
---

# skill-tuner
Tells you whether a change to an agent-consumed document actually worked: paired comparison with a confidence interval and a non-inferiority margin, content-hashed provenance with drift detection, and an adversarial defect probe. Reads Anthropic's skill-creator benchmark output and supplies the verdict it stops short of.
- Slug: skill-tuner
- Publisher: StartupBros / House of Vibe
- Repository: https://github.com/StartupBros-com/skill-tuner
- Manifest: plugin.json
- Version: 0.10.0
- License: MIT
- Category (editorial): agent-tooling
- Skills: 3 (portfolio, skill-tuner, tune)
- MCP servers: 0
- Stars: 0
- Repository created: 2026-08-07
- Repository last pushed: 2026-09-24
- Publisher type: Organization
- Listing: https://agentpluginsdirectory.com/plugins/skill-tuner
- Schema: https://agent-plugins.org/schemas/1.0.0/plugin.schema.json

## What skill-tuner does, in the publisher's words

Tells you whether a change to an agent-consumed document actually worked, with an interval, a margin you set, and a receipt that can be re-checked after the model moves under you.

It does not help you write skills. Plenty of guides do that. What nothing does is tell you whether the edit you just made was an improvement or a coin flip.

Eval harnesses report deltas. Anthropic's own skill-creator runs a proper experiment, eval cases, an isolated subagent per run, graded expectations, with_skill vs without_skill benchmarking, and aggregate_benchmark.py reports mean and stddev. Its improvement record stores a bare grading_result: "won" | "lost" | "tie" beside a pass rate.

- A verdict, not a delta: paired by case, 95% interval, a non-inferiority margin you state, and four outcomes including inconclusive. A percentile bootstrap, an exact sign test, and a paired effect size print beside the t-interval, so a verdict leaning on a normality assumption is visible too.

From the project README, punctuation lightly normalized. Full text: https://raw.githubusercontent.com/StartupBros-com/skill-tuner/HEAD/README.md

## Skills

- portfolio
- skill-tuner: Evidence-tagged rules for writing documents agents consume. Use when creating or editing skills, AGENTS.md, or CLAUDE.md.
- tune: Find and fix real defects in a skill, AGENTS.md, or CLAUDE.md, then prove the description still routes. Runs the bundled token-spending runner, so it starts only on the human's explicit instruction.

Descriptions come from the frontmatter of each SKILL.md, punctuation lightly normalized.