Data Throughput Accelerator
Diagnose and accelerate large data movement (ingestion, backfill, ETL) by isolating the true bottleneck, with proof of correctness.
- What
- Diagnose and accelerate large data movement (ingestion, backfill, ETL) by isolating the true bottleneck, with proof of correctness.
- Cost
- Free
- Needs
- Use "Data Throughput Accelerator" with your Muse.
- Install
- Copy the installer prompt below into your Muse — your agent does the rest.
Curated by Skill Harbor: a skill for when a data pipeline or backfill is too slow and must get faster without losing correctness. It enforces a disciplined method: first separate the candidate bottlenecks (source extraction speed, network transfer, warehouse load speed, transform speed, serving-table freshness, live tail growth), then benchmark variants against each other, then codify the fastest path, always closing with a hard accounting block that proves rows and timestamps cohere. Speed is never the only goal; the goal is faster correct data landing in the right place with proof. By @affaan-m, listed here with credit to its creator. From the affaan-m/ECC repository (MIT, declared in frontmatter). Honest caveats: an agent-facing workflow, it expects shell and file tools to run benchmarks; assumes access to the pipeline and its data stores; measure on production-like volumes, toy datasets hide the real bottleneck. Skill Harbor never reviews the code, review it yourself before use.
Version:
Install
Copy the install package below, then paste it into MuseThe install prompt below already includes the vetting steps: your agent follows the community checklist before installing anything with executable code. Want more?
Use "Data Throughput Accelerator" with your Muse. Prerequisites: access to the slow pipeline (or backfill job) and its data stores; an agent with shell/file tools to run benchmarks. Method guidance, not a packaged tool. 1. Open the skill: https://github.com/affaan-m/ECC/blob/main/skills/data-throughput-accelerator/SKILL.md and copy the full SKILL.md text. 2. Paste it into a chat with Muse and add: "This backfill is too slow. Isolate the true bottleneck first: extraction, network, load, transform, or freshness? Then benchmark." 3. When a faster path is proposed, ask: "Close with the hard accounting block: prove rows and timestamps cohere before we adopt it." Tip: run benchmarks on production-like volumes; small samples lie about bottlenecks. Safety: a skill is plain-text instructions; it runs nothing by itself. Never optimize against production data without a backup and a rollback plan.
Saved to your recent installs. Find it anytime on /connect.
Questions
How do I install a build?
Every product page includes a copy-paste install prompt. Paste it into your Muse and it sets the build up for you — no manual configuration.
Where does my money go?
Straight to the seller. Skill Harbor never processes payments: checkout happens on the seller’s own page, usually Stripe.
What does the ✓ next to a creator’s name mean?
It means we confirmed the identity of the person behind the listing. It says nothing about the code itself — always check a build before installing it.