---
title: "agentic-bundle-aas-ai-product-evaluation-ops"
description: "Portable skills-only \"AAS AI Product & Evaluation Ops\" plugin from Agentic Awesome Skills."
canonical: https://agentpluginsdirectory.com/plugins/agentic-bundle-aas-ai-product-evaluation-ops--ibrdot
last-updated: 2026-10-24
---

# agentic-bundle-aas-ai-product-evaluation-ops
Portable skills-only "AAS AI Product & Evaluation Ops" plugin from Agentic Awesome Skills.
- Slug: agentic-bundle-aas-ai-product-evaluation-ops--ibrdot
- Publisher: ibrdot and contributors
- Repository: https://github.com/ibrdot/swe-re-expert
- Manifest: plugins/agentic-bundle-aas-ai-product-evaluation-ops/plugin.json
- Version: 19.11.0
- License: MIT
- Category (editorial): other
- Skills: 13 (ab-test-setup, advanced-evaluation, agent-evaluation-reporting, agent-evaluation, ai-wrapper-product, analytics-product, context-window-management, hugging-face-evaluation, kpi-dashboard-design, langfuse, llm-app-patterns, product-manager, skill-security-audit)
- MCP servers: 0
- Stars: 4
- Repository created: 2026-08-24
- Repository last pushed: 2026-10-19
- Publisher type: User
- Listing: https://agentpluginsdirectory.com/plugins/agentic-bundle-aas-ai-product-evaluation-ops--ibrdot
- Schema: https://agent-plugins.org/schemas/1.0.0/plugin.schema.json

## Skills

- ab-test-setup: Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.
- advanced-evaluation: Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.
- agent-evaluation-reporting: Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
- agent-evaluation: Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.
- ai-wrapper-product: Expert in building products that wrap AI APIs (OpenAI, Anthropic, etc. ) into focused tools people will pay for. Not just "ChatGPT but different": products that solve specific problems with AI.
- analytics-product: Analytics de produto: PostHog, Mixpanel, eventos, funnels, cohorts, retencao, north star metric, OKRs e dashboards de produto.
- context-window-management: Strategies for managing LLM context windows including summarization, trimming, routing, and avoiding context rot
- hugging-face-evaluation: Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.
- kpi-dashboard-design: Comprehensive patterns for designing effective Key Performance Indicator (KPI) dashboards that drive business decisions.
- langfuse: Expert in Langfuse: the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production.
- llm-app-patterns: Architecture and integration sketches for LLM applications, with explicit retrieval, tool, privacy and verification boundaries.
- product-manager: Senior PM agent with 6 knowledge domains, 30+ frameworks, 12 templates, and 32 SaaS metrics with formulas. Pure Markdown, zero scripts.
- skill-security-audit

Descriptions come from the frontmatter of each SKILL.md, punctuation lightly normalized.