Eval Harness
Practice eval-driven development: define capability and regression evals before coding, grade, and track pass@k.
- What
- Practice eval-driven development: define capability and regression evals before coding, grade, and track pass@k.
- Cost
- Free
- Needs
- the Muse app (mobile or web). No installation, no API key. A method framework, not an auto-runner.
- Level
- Beginner — the Muse app (mobile or web).
- Install
- Copy the installer prompt below into your Muse — your agent does the rest.
Curated by Skill Harbor: a formal eval-driven development (EDD) framework for AI coding sessions. Evals are the unit tests of AI development: define capability evals (can the agent do something new) and regression evals (did a change break existing behavior) BEFORE implementation, then grade with code-based deterministic graders, model-based LLM-as-judge rubrics, or human review with risk levels. Metrics: pass@k (at least one success in k attempts) and pass^k (all k trials succeed, the higher bar for critical paths), with recommended thresholds (pass@3 >= 0.90 for capability, pass^3 = 1.00 for release-critical regressions). Includes the eval workflow (define, implement, evaluate, report), storage layout (.claude/evals/ definitions, logs, baselines), eval anti-patterns (overfitting prompts to eval examples, happy-path-only measurement, flaky graders in release gates), and best practices (define before coding, deterministic graders first, version evals with code). Note: the skill's example utilities describe a capsule and replay framework where candidate execution is disabled by design (no verified OS containment backend); static checks and receipts are not execution evidence. By @affaan-m, listed here with credit to its creator. From the affaan-m/ECC repository (MIT). Honest caveats: a method framework, not an auto-runner; some paths follow Claude Code conventions. Skill Harbor never reviews the code, review it yourself before use.
Version:
Quick answer
Eval Harness is a free Muse skill that practices eval-driven development, defining capability and regression evals before coding, then grading and tracking pass@k. Evals are the unit tests of AI development: capability evals test something new, regression evals test that a change broke nothing, graded with deterministic graders, LLM-as-judge rubrics, or human review, with recommended thresholds.
Install
Copy the install package below, then paste it into Muse🧪 Install ran in our test sandbox · Nov 11, 2026📌 Pinned version: commit 4eb71d9Pinned version: install the code at commit 4eb71d92a39cab44ad40ac9d8a6a5ccb4029d6c2 exactly (Skill Harbor tested this version). Do not substitute a newer version. If this commit is unreachable, stop and tell me. Use "Eval Harness" with your Muse. Prerequisites: the Muse app (mobile or web). No installation, no API key. A method framework, not an auto-runner. 1. Open the skill: https://raw.githubusercontent.com/affaan-m/ECC/4eb71d92a39cab44ad40ac9d8a6a5ccb4029d6c2/skills/eval-harness/SKILL.md and copy the full SKILL.md text. 2. Paste it into a chat with Muse and add: "Set up eval-driven development for [feature or agent task]. Define capability evals and regression evals first, with pass@k targets." 3. Ask Muse to draft the eval definitions, pick grader types (code first, model grader for open-ended, human for risky), and produce the eval report format after each round. Tip: define evals before any implementation. Writing them after is just documenting what you already built. Safety: a skill is plain-text instructions; it runs nothing by itself. Never paste secrets into a chat, and review anything Muse proposes before it acts.
Saved to your recent installs. Find it anytime on /connect.
🔒 Security, provenance and what this touches
The install prompt below already includes the vetting steps: your agent follows the community checklist before installing anything with executable code. Want more?
Verification certificate
Dated facts about what was checked for this listing, and how. This is not a safety certification.
- Pinned version
- 4eb71d9 · 2026-10-13
- Sandbox install test
- Install ran in our test sandbox · Nov 11, 2026
- Creator identity
- Not verified
- Fingerprint watch
- Last checked · Nov 21, 2026 : no change detected in the files linked by this install prompt
- Install prompt analysis
- 2026-10-13 : No commands, network calls, packages, file writes or API keys detected.
Skill Harbor never reviews the code of a listed skill. These are dated facts about the install prompt and our install test, not a safety certification. Review a skill yourself before you install it. How we verify
Questions
How do I install a skill?
Every product page includes a copy-paste install prompt. Paste it into your Muse and it sets the skill up for you — no manual configuration.
Where does my money go?
Straight to the seller. Skill Harbor never processes payments: checkout happens on the seller’s own page, usually Stripe.
What does the ✓ next to a creator’s name mean?
It means we confirmed the identity of the person behind the listing. It says nothing about the code itself — always check a skill before installing it.