← Search

Agent Eval
Developer tools 🧪 Install ran in our test sandbox · Nov 11, 2026 📌 Pinned version: commit 4eb71d9
Install test only, on the date shown. This does not guarantee the code is safe.
⚙ Needs: a git repository with real tasks to test…

Agent Eval

Compare coding agents head-to-head on your own tasks with pass rate, cost, time, and consistency metrics.

At a glance
What
Compare coding agents head-to-head on your own tasks with pass rate, cost, time, and consistency metrics.
Cost
Free
Needs
a git repository with real tasks to test, and the coding-agent CLIs you want to compare installed on your machine (e.g. Claude Code, Aider, Codex). The Muse app (mobile or web) for the planning side.
Level
Beginner — a git repository with real tasks to test…
Install
Copy the installer prompt below into your Muse — your agent does the rest.

Version:

Quick answer

Agent Eval is a free Muse skill that compares coding agents head-to-head on your own tasks, with pass rate, cost, time, and consistency metrics. It uses a lightweight CLI for reproducible tasks defined in YAML, with deterministic, pattern-based, or model-based judges, each run isolated in its own git worktree, and a report on pass rate, cost, time, and consistency.

⌁

Install

Copy the install package below, then paste it into Muse🧪 Install ran in our test sandbox · Nov 11, 2026📌 Pinned version: commit 4eb71d9

Pinned version: install the code at commit 4eb71d92a39cab44ad40ac9d8a6a5ccb4029d6c2 exactly (Skill Harbor tested this version). Do not substitute a newer version. If this commit is unreachable, stop and tell me. Use "Agent Eval" with your Muse. Prerequisites: a git repository with real tasks to test, and the coding-agent CLIs you want to compare installed on your machine (e.g. Claude Code, Aider, Codex). The Muse app (mobile or web) for the planning side. 1. Open the skill: https://raw.githubusercontent.com/affaan-m/ECC/4eb71d92a39cab44ad40ac9d8a6a5ccb4029d6c2/skills/agent-eval/SKILL.md and copy the full SKILL.md text. 2. Paste it into a chat with Muse and add: "Help me define 3 to 5 benchmark tasks for [my project] in the agent-eval YAML format. Repo at [path], pinned commit [sha]." 3. Install the agent-eval CLI from the repository linked in the skill (review the source first), write your task YAMLs, then run: agent-eval run --task tasks/<name>.yaml --agent claude-code --agent aider --runs 3 4. Ask Muse to help interpret the report table (pass rate, cost, time, consistency) and make the call. Tip: start with tasks from your real workload, not toy examples, and always include at least one deterministic judge (tests or build) per task. Safety: a skill is plain-text instructions; it runs nothing by itself. Review any command before running it, never paste secrets into a chat, and review anything Muse proposes before it acts.

🔒 Security, provenance and what this touches
🧭 What this touches
Based on the install prompt we scanned, this skill does not run commands, make third-party network calls, install packages, write files, or ask for API keys.
Install prompt analysis · 2026-10-13
🔒 Security and provenance
📌 Pinned version: commit 4eb71d9 · 2026-10-13
🧪 Install ran in our test sandbox · Nov 11, 2026
👁 Observed during the install test · Nov 11, 2026 : no network contact
🔎 Last checked · Nov 21, 2026 : no change detected in the files linked by this install prompt
Skill Harbor never reviews the code of a listed skill. These are dated facts about the install prompt and our install test, not a safety certification. Review a skill yourself before you install it. How we verify
Wanna be extra careful?

The install prompt below already includes the vetting steps: your agent follows the community checklist before installing anything with executable code. Want more?

How to check a skill before installing →

✓

Verification certificate

Dated facts about what was checked for this listing, and how. This is not a safety certification.

Pinned version
4eb71d9 · 2026-10-13
Sandbox install test
Install ran in our test sandbox · Nov 11, 2026
Creator identity
Not verified
Fingerprint watch
Last checked · Nov 21, 2026 : no change detected in the files linked by this install prompt
Install prompt analysis
2026-10-13 : No commands, network calls, packages, file writes or API keys detected.

Skill Harbor never reviews the code of a listed skill. These are dated facts about the install prompt and our install test, not a safety certification. Review a skill yourself before you install it. How we verify

?

Questions

How do I install a skill?

Every product page includes a copy-paste install prompt. Paste it into your Muse and it sets the skill up for you — no manual configuration.

Where does my money go?

Straight to the seller. Skill Harbor never processes payments: checkout happens on the seller’s own page, usually Stripe.

What does the ✓ next to a creator’s name mean?

It means we confirmed the identity of the person behind the listing. It says nothing about the code itself — always check a skill before installing it.