← Search

GPTQ post-training 4-bit quantization for LLMs: fit big models on small GPUs
AI 🧪 Install ran in our test sandbox · Nov 11, 2026 📌 Pinned version: commit 773a529
Install test only, on the date shown. This does not guarantee the code is safe.
⚙ Needs: Python with pip and a CUDA GPU for real…

GPTQ post-training 4-bit quantization for LLMs: fit big models on small GPUs

Group-wise 4-bit quantization with ~2% perplexity loss, 4x memory reduction and 3-4x inference speedup, kernel backends (ExLlamaV2, Marlin, Triton), transformers/QLoRA integration, and benchmark tables

At a glance
What
Group-wise 4-bit quantization with ~2% perplexity loss, 4x memory reduction and 3-4x inference speedup, kernel backends (ExLlamaV2, Marlin, Triton), transformers/QLoRA integration, and benchmark tables
Cost
Free
Needs
Python with pip and a CUDA GPU for real quantization (CPU-only works for reading the workflow, not for quantizing); HuggingFace access for pre-quantized models
Install
Copy the installer prompt below into your Muse — your agent does the rest.

Version:

@
Created by: @orchestra-research

Quick answer

GPTQ quantization is a free Muse skill for group-wise four-bit post-training quantization of large language models. It promises about two percent perplexity loss, four times less memory and three to four times faster inference, with kernel backends including ExLlamaV2, Marlin and Triton. It also covers transformers and QLoRA integration, plus benchmark tables.

⌁

Install

Copy the install package below, then paste it into Muse🧪 Install ran in our test sandbox · Nov 11, 2026📌 Pinned version: commit 773a529

Step 0 — before installing anything: read this install prompt, summarize in a few lines what this build does and what installing it involves, then ask me to confirm before you proceed. Prerequisites: Python with pip and a CUDA GPU for real quantization (CPU-only works for reading the workflow, not for quantizing); HuggingFace access for pre-quantized models Install "GPTQ post-training 4-bit quantization for LLMs: fit big models on small GPUs" for me. It gives my agent @orchestra-research's GPTQ playbook: group-wise 4-bit quantization configs (standard, high-accuracy 3-bit, max-accuracy small-group), kernel backends (ExLlamaV2, Marlin, Triton), auto-gptq install, loading pre-quantized models, quantizing your own model with calibration data, transformers/QLoRA integration, multi-GPU and CPU-offload deployment patterns, and benchmark tables. MIT licensed. Repository: https://raw.githubusercontent.com/orchestra-research/ai-research-skills/773a52944ba4747a18bd4ae9ade53fff041adcbc/10-optimization/gptq/SKILL.md Pinned version: install the code at commit 773a52944ba4747a18bd4ae9ade53fff041adcbc exactly (Skill Harbor tested this version). Do not substitute a newer version. If this commit is unreachable, stop and tell me. 1. Fetch the SKILL.md file (and any helper files) from the repository path into a temporary folder 2. Safety check: review the SKILL.md and scripts for anything suspicious (unexpected network calls, shell commands, credential harvesting). This repo should contain zero secrets in code, credentials only via the secure vault, allowed hosts declared in the SKILL.md. Verify that holds here; STOP on any red flag and tell me. 3. Install it as a skill: copy SKILL.md and its helper files into the agent's skills directory, in a folder named "gptq". 4. Verify with no network calls: frontmatter valid, files in place. 5. Report what was installed, where, and what I still need to do myself (e.g. confirm a CUDA GPU is available and which path to take — load a pre-quantized model from HuggingFace or quantize my own model with calibration data; validate benchmark claims on my own hardware; pick the kernel backend for my GPU generation — Marlin needs Ampere+). GitHub is optional: if I have a GitHub account or the gh CLI, you may use it; otherwise public access is fine. Never require it unless it's in the prerequisites above. Rules: don't touch anything outside the temp folder and the install target. If anything looks off, stop and ask me.

🔒 Security, provenance and what this touches
🧭 What this touches
Based on the install prompt we scanned, this skill does not run commands, make third-party network calls, install packages, write files, or ask for API keys.
Install prompt analysis · 2026-10-13
🔒 Security and provenance
📌 Pinned version: commit 773a529 · 2026-10-13
🧪 Install ran in our test sandbox · Nov 11, 2026
👁 Observed during the install test · Nov 11, 2026 : no network contact
🔎 Last checked · Nov 19, 2026 : no change detected in the files linked by this install prompt
Skill Harbor never reviews the code of a listed skill. These are dated facts about the install prompt and our install test, not a safety certification. Review a skill yourself before you install it. How we verify
Wanna be extra careful?

The install prompt below already includes the vetting steps: your agent follows the community checklist before installing anything with executable code. Want more?

How to check a skill before installing →

✓

Verification certificate

Dated facts about what was checked for this listing, and how. This is not a safety certification.

Pinned version
773a529 · 2026-10-13
Sandbox install test
Install ran in our test sandbox · Nov 11, 2026
Creator identity
Not verified
Fingerprint watch
Last checked · Nov 19, 2026 : no change detected in the files linked by this install prompt
Install prompt analysis
2026-10-13 : No commands, network calls, packages, file writes or API keys detected.

Skill Harbor never reviews the code of a listed skill. These are dated facts about the install prompt and our install test, not a safety certification. Review a skill yourself before you install it. How we verify

?

Questions

How do I install a skill?

Every product page includes a copy-paste install prompt. Paste it into your Muse and it sets the skill up for you — no manual configuration.

Where does my money go?

Straight to the seller. Skill Harbor never processes payments: checkout happens on the seller’s own page, usually Stripe.

What does the ✓ next to a creator’s name mean?

It means we confirmed the identity of the person behind the listing. It says nothing about the code itself — always check a skill before installing it.