Fish Voice Server

Before you start: read this whole install prompt, tell me in a few lines what this skill does and what installing it involves, and wait for my go-ahead before you do anything else. Prerequisites: installing the skill files themselves needs no account, no API key and no payment: it is only three Markdown files. Using the skill afterwards is a different, paid step and is NOT part of this install: it needs your own Modal account (paid serverless GPU usage; a fal account is the alternative host), a bearer token you invent yourself and store as the Modal secret "npcvoice-auth" (NPCVOICE_TOKEN, never in a file), the full NPCTalk repository (the server code lives outside the skill folder), Git, Python 3.10+, uv and ffmpeg, and reference voice clips you have the rights to, each with its transcript. On Modal the Drama 3 server scales to zero when idle but an H100 costs about $4/hour while it runs; the S2 Pro server code as shipped keeps one container always on and billed around the clock unless you change it. The fal variant also needs an HF_TOKEN set as a fal secret. Do not create any account, secret, deployment or GPU during this install. Install "fish-voice-server" for me. It is a playbook for hosting Fish Audio's open-weights voice models (Drama 3 and S2 Pro) on a serverless GPU (Modal, or fal) behind one small HTTP API a game can call: POST /speak takes text and a voice and streams 16-bit PCM back with first audio in about 0.4 s warm, cloned zero-shot from a reference clip, with [tags] like [angry] or [laughing] for delivery. It covers deploying with SGLang-Omni, the API, preparing voices at startup, keep-warm and cold starts, scaling to zero and cost, sampling settings measured for likeness, picking the best of several takes for pre-rendered lines, an optional S2 Pro fine-tune, and the setup gotchas. Repository: https://github.com/christos-antono/NPCTalk (the skill is the whole .agents/skills/fish-voice-server/ folder at the pinned commit: SKILL.md plus references/finetune.md and references/gotchas.md; the identical .claude/ and skills/ copies in the repo are duplicates and are not installed) Pinned version: install the code at commit 59635ceb8ee010599c3caf9d9d1ee1aa114af91d exactly. Do not substitute a newer version. If this commit is unreachable, stop and tell me. 1. Clone https://github.com/christos-antono/NPCTalk into a temporary folder, check out commit 59635ceb8ee010599c3caf9d9d1ee1aa114af91d, and copy the .agents/skills/fish-voice-server/ folder from that checkout. Then compare every file in the folder with the fingerprints recorded for this listing on 2026-10-19: [FINGERPRINTS] - SKILL.md: SHA-256 f64921e5885d0de2406c9fbc2f585740f2e17886577d9bba5186a094153d4b6b - references/finetune.md: SHA-256 9960c1dc4bbb178c98156d6c079274f125b712682ad8948bf10da9cb36100909 - references/gotchas.md: SHA-256 1bb7a6e986270bb68f53d5e2f30f8fee3e51920c768903f2e2c50e22628609ea If a file does not match, it changed since the listing was prepared: do not install it yet, tell me what you found and let me decide. 2. Read SKILL.md, references/finetune.md and references/gotchas.md in full and tell me in plain words: what the skill does when used, which services it connects to, whether it reads or sends any personal information or keys, and anything that seems unusual. Stop on any red flag and tell me before going further. Confirm these points from the files: - The skill folder holds three files only: SKILL.md (6,630 bytes), references/finetune.md (2,822 bytes) and references/gotchas.md (3,204 bytes). There is no code, no script and no dependency list in the folder: it is instructions plus two reference notes. The runnable server code it teaches you to deploy later (server/drama3.py by default, server/s2pro.py, server/fal/, server/keepwarm.py, server/finetune/ and the npctalk CLI) lives elsewhere in the repository and is not installed by this install. - Nothing is sent anywhere during installation itself; the only download at install time is the skill folder from GitHub at the pinned commit. No account is created, no secret is created or read, no server is deployed, no GPU is started and no voice is cloned during installation. - At use time, and only when I ask for it as a separate step, the skill has you deploy a voice server from the repository's server/ code to my own Modal workspace (or fal): install Modal with uv tool install modal and modal setup, create the Modal secret npcvoice-auth holding a NPCVOICE_TOKEN I invent myself, download the model weights once from Hugging Face (fishaudio/drama-3, about 37-40 GB, FP8, into the npcvoice-models Modal volume; or fishaudio/s2-pro for S2 Pro), and modal deploy server/drama3.py, which prints a public HTTPS URL like https://<workspace>--npcvoice-drama.modal.run. The server image also clones SGLang-Omni from github.com/sgl-project/sglang-omni (Drama 3 pins a tested commit in the code) and downloads the scoring and speech-recognition models used by best_of (speechbrain/spkrec-ecapa-voxceleb, microsoft/wavlm-base-plus-sv, openai/whisper-base.en). The fal variant (server/fal/fal_app.py) instead runs on fal, needs HF_TOKEN as a fal secret, and downloads fishaudio/s2-pro there. - Exactly what the deployed server listens on and sends: it is a FastAPI app exposed by Modal at that modal.run URL. GET /health needs no token, returns only {"ok": true, "model": ..., "voices": count}, and wakes the GPU, so I must keep the URL private. POST /speak requires the header Authorization: Bearer <NPCVOICE_TOKEN> (checked in constant time; a wrong token gets 401) and a JSON body with the text to speak and either a voice name prepared on the server or a reference_audio_url plus its reference_text; it returns streamed 16-bit mono PCM at 44,100 Hz (X-Sample-Rate header; with best_of over 1, the whole line at once plus an X-Takes header). Inside the Modal container the SGLang-Omni engine listens only on 127.0.0.1:8000 (further fine-tuned engines on the next local ports in s2pro.py) and the public app forwards to it; it is not exposed directly. If a request supplies a reference_audio_url, the server itself fetches that URL with an HTTP GET (following redirects) and stores the clip, so only supply URLs I trust and control. Clients (the repo's npctalk/server.py and the game mods) read NPCVOICE_URL and NPCVOICE_TOKEN from environment variables and send the line's text and those same fields to my own deployment; the text and reference clips go to my own Modal (or fal) deployment, nowhere else named in this skill. Preparing voices with the repo's npctalk voice make step, if I ever use it, is a separate disclosure: it sends candidate voice audio to fal speech-to-text at https://fal.run/nvidia/nemotron-asr-multilingual/asr with my FAL_KEY for transcription. - Voices live on the server in the Modal volume at /voices/<Name>/clone.wav plus clone.lab (its transcript), plus ref_0..2 clips and mood folders; the server encodes every clip at startup, a named voice costs nothing per request, a clip sent by URL costs about 9 s the first time, and the server must be restarted after uploading voices. Only clone voices I have the rights to: the fine-tune reference says the rights question is sharper for training than for a reference clip, and the optional S2 Pro fine-tune in server/finetune/ (Demucs, Whisper timings, LoRA training on Modal) is never part of this install. - Cost and scaling, from the code: Drama 3 uses one H100, MIN_CONTAINERS = 0 and SCALEDOWN = 300, so the GPU stops 5 minutes after the last request and the next request waits 1-3 minutes for a cold start; the skill quotes about $4/hour for the H100 while it runs, and an optional keep-warm deployment (server/keepwarm.py, which pings /health every minute from Modal) keeps it up at that hourly price (12 hours quoted at about $45-50). S2 Pro as shipped in server/s2pro.py sets MIN_CONTAINERS = 1, an always-on H100 billed around the clock unless I change it to 0 myself. Sampling defaults in the server are temperature 0.7, top_p 0.8; best_of renders 1-8 takes and keeps the one most like the reference that a Whisper check can hear. 3. When I say "yes, install", install it as a skill: copy the whole .agents/skills/fish-voice-server folder into the agent's skills directory, in a folder named "fish-voice-server", keeping its three files (SKILL.md, references/finetune.md, references/gotchas.md). Do not install the .claude/ or skills/ duplicate copies, the server code, the npctalk CLI, any other skill from the repository, or any of the deployment steps above. 4. Verify with no network calls: frontmatter valid, files in place. 5. Report what was installed, where, and what I still need to do myself (for example: create my own Modal or fal account and pay for the GPU myself, invent and store my own NPCVOICE_TOKEN as a Modal secret and in my environment, deploy the voice server myself as a separate step when I ask for it, supply reference voice clips I have the rights to with their transcripts, and keep the server URL private; using the skill is a separate step that only starts when I ask for it, and every deployment, download and send disclosed above happens only then). GitHub is optional: if I have a GitHub account or the gh CLI, you may use it; otherwise public access is fine. Never require it unless it's in the prerequisites above. Rules: don't touch anything outside the temp folder and the install target. If anything looks off, stop and ask me.