WaxFlow

Self-hosted, pure-Go, on-the-fly audio transcoding: request -> decode -> DSP -> encode -> stream, tuned for time-to-first-audio, sample-exact seeking, and flaky mobile networks.

The codecs (Opus, MP3, AAC-LC, HE-AAC v1, Vorbis, FLAC, ALAC, WavPack, Monkey's Audio, and WAV encoders, plus a wider decoder set) are written from scratch for Go 1.26 and published as public, stdlib-only packages under this module, CI-enforced by make depcheck, so anyone can import them.

Status

Everything below is tested, capability-gated (/caps never advertises what does not work), and held to the pinned gates in docs/quality-gates.md.

• Encoders (all from scratch, stdlib-only): Opus (full SILK, hybrid, and CELT with the tonality analyser; at quality parity with libopus on the reference opus_compare metric across music and speech corpora), MP3 (psychoacoustic model, joint stereo, CBR and VBR, LAME gapless tag), AAC-LC (window switching, TNS, M/S, two-loop quantization; at parity with ffmpeg's native encoder on the ODG-proxy gate), HE-AAC v1 and v2 (SBR over a half-rate AAC-LC core: transient-aware envelope grids, tonality-driven noise floors and inverse filtering, coupled stereo; v2 adds parametric stereo over a phase-aligned mono downmix, selected explicitly; explicit signalling in M4A, implicit in ADTS), Vorbis (product-lattice VQ residue books, perceptual coupled-stereo classification; ODG-proxy gate green), FLAC (levels 0-8, smaller than flac -5 at level 5), ALAC (bit-exact round trip), WavPack (four compression levels, each a wider search over candidate decorrelation cascades; at parity with libwavpack on the official suite's own audio, and ahead of it at the fast setting), Monkey's Audio (three compression levels; the coded frames are the reference encoder's frames, byte for byte), and WAV/AIFF PCM.
• Decoders / inputs: FLAC (bit-exact on the IETF suite), WAV, AIFF, MP3 (bare, and in MP4/M4A, QuickTime .mov, WAV and AIFF-C; the last two are read-only, and format=mp3 on one copies its frames into a bare .mp3 rather than re-encoding them), AAC-LC, HE-AAC v1 and v2 (SBR+PS, ffmpeg-differential-verified), and ALAC in MP4/M4A/M4B, uncompressed PCM in MP4/MOV (QuickTime's sowt/twos/NONE/raw /in24/in32/fl32/fl64 fourccs, its version 2 lpcm flags and its ms entries for WAVE format tags 1 and 3, plus ISO ipcm/fpcm, with a multichannel track's layout read from the file's own chan or chnl box and its channels reordered into the pipeline's order where the file's order differs; decode only, and writing it there is a non-goal, a WAV serving every such use better), G.711 A-law and mu-law and both 4-bit ADPCM families (IMA/DVI in its WAV and QuickTime ima4 layouts, and Microsoft ADPCM), each bit-exact against ffmpeg wherever the container spells it: a WAV format tag, an AIFF-C compression type, or a QuickTime fourcc or ms entry in a progressive or fragmented movie (decode only, and encoding any of them is a non-goal: they are lossy codings of exactly what a WAV already carries losslessly, so the only reason to write one is a reader that takes nothing else, and every such reader also takes PCM), ADTS (implicit HE-AAC detected), Opus (all RFC 6716/8251 conformance vectors pass), Vorbis, Ogg, Matroska/WebM, WavPack (bit-exact on the official test suite: 8- to 32-bit integers, the three stereo block modes, APEv2 tags; encodes too), Monkey's Audio (all five compression levels, 8/16/24-bit, APEv2 tags; encodes the first three), Windows Media Audio 1 and 2 in ASF (mono and stereo, 8-48 kHz, tags and Marker Object chapters; decode only, and encoding it is a non-goal: nothing plays WMA that does not also play a format this tree writes better), Windows Media Audio 9.2 Lossless (bit-exact against the encoder's own input, 16- and 24-bit, stereo and 5.1 at 44.1 to 96 kHz; decode only, and encoding it is a non-goal for the same reason, and because the only encoder for it is a Windows component), and Windows Media Audio 9/10 Professional (differential-verified against ffmpeg on files Windows' own encoder wrote, 16- and 24-bit, stereo and 5.1 at 44.1 to 96 kHz; decode only, and encoding it is a non-goal for the same reasons as the other two), Windows Media Audio Voice (the CELP speech coder, differential-verified the same way, mono at 8 to 22.05 kHz across the whole seven-format envelope Windows' encoder offers; decode only, and encoding it is a non-goal for the same reasons again), and Musepack SV7 and SV8 (bit-identical to the reference decoder's float output; true gapless, noise substitution, APEv2 tags, SV8 chapters and ReplayGain; decode only, and encoding it is a non-goal for the same reason, the only encoder in existence being LGPL). Sample-exact seeking everywhere, gapless honored per format (LAME tag, iTunSMPB, edit lists or a fragmented movie's segment index when it has none, Ogg pre-skip/end-trim, Matroska CodecDelay). One format charges for that exactness: QuickTime ima4 carries its predictor across blocks and a block header restates only nine bits of it, so a seek decodes from the file's start rather than from the block, at roughly a second per hour of stereo 44.1 kHz audio. A WMA Pro stream that sets the low-bit-rate tool no decoder outside Windows implements is refused by name, as is the second registered WMA Voice tag (0x000B), which no reference decoder reads. A WAV, AIFF-C or MP4/MOV payload in a codec this build does not decode is refused by that codec's name too, the same name the sibling tag reader reports for the same bytes.
• DSP: Kaiser windowed-sinc resampling (hq/fast), BS.775 downmix (on a timeline, applied per member before the seam so a member's level is its own fold's), gain with true-peak limiting, TPDF and shaped dither, EBU R128 / BS.1770-4 loudness (differential-verified against ffmpeg).
• Service: progressive streaming with a direct-play/transmux/ transcode ladder, sample-exact t= seeks, HMAC-signed URLs pinned to source identity, a write-through cache with read-behind delivery and full ranges on completed entries, HLS (CMAF/fMP4, stateless signed URLs, bitrate ladders, byte-identical segment regeneration), async jobs with restart safety (transcode, analyze, and the gapless merge/split pair: a lossless split rejoins bit for bit, less any data track it skipped, and a split takes a CUE sheet or sample cut points), uploads, loudness analysis with ReplayGain tagging, metadata passthrough (tags, chapters, cover art, lyrics), admission control, Prometheus metrics, named client delivery profiles in /caps, and a full API contract in docs/api.md.
• Structure: the root module is the importable, dependency-free audio library; its go.mod has an empty require block, so importing codec/flac (or any public package) pulls in nothing. Beside the codecs and containers it holds the two text formats a host shares with the daemon, cue (CUE sheets) and m3u8 (HLS playlists). The CLI/daemon binary lives in the nested cli/ module (cobra + waxlabel), the tests that need third-party oracles in oracletest/, and a worked example of extending the CLI in examples/catalogcli/.

Performance

Measured by the committed harnesses on the reference dev box (12-core x86-64, Linux, loopback HTTP; server/load_soak_test.go, re-run per release with make soak):

• Time to first audio (TestTTFAPercentiles, n=50): cold transcode (also the cost of every t= seek, since each offset is its own cache entry) p50 0.6 ms / p95 0.9 ms; warm (completed cache entry) p50 0.21 ms / p95 0.24 ms; seek into a 60 s FLAC source (bisection seek + pipeline start) p50 1.7 ms / p95 2.7 ms. Targets: p95 <300 ms warm, <800 ms cold, met with three orders of headroom on this box; network and disk latency dominate in deployment.
• HLS seek-to-segment (variant-worker restart): worst 49 ms in the restart-heavy e2e fetch pattern, p95 well under the 1 s target.
• Load (TestLoadMixedTraffic): 6,350 req/s of mixed live transcodes, seeks, direct play, probes, and HLS over 8 concurrent workers with zero malformed responses (503s under saturation carry the honest overloaded envelope).
• Streaming soak (TestStreamingSoak, goroutine/heap leak watch): 918k requests over a sustained run with full reads, mid-body client disconnects, seeks, and HLS fetches; goroutines returned to baseline and heap stayed flat (~1.3 MiB). The nightly job runs a 20-minute soak; make soak runs 30 minutes locally.
• Codec throughput (per core, make bench): every codec clears its pinned floor in docs/quality-gates.md by a wide margin (Opus encode 55-67x realtime, decode 274-514x; MP3 encode 50-69x, decode ~200x; FLAC encode ~200x, decode 490-870x; AAC encode 20-68x, decode ~240x; Musepack decode 690-1900x).

Quick start

# Hardened standalone deployment, port 4418. Publishing a non-loopback
# port requires API keys (the daemon fails closed without them).
WAXFLOW_API_KEYS=$(openssl rand -hex 24) docker compose up -d
curl http://localhost:4418/ping
curl -H "X-API-Key: $WAXFLOW_API_KEYS" http://localhost:4418/caps

Put music under ./library, or point the compose bind mount elsewhere with WAXFLOW_LIBRARY=/path/to/music, and stream: mint a URL with POST /sign, or try the dev demo page (--demo). WAXFLOW_LIBRARY is compose's variable for that mount, not a daemon setting; the daemon reads roots / WAXFLOW_ROOTS. Or from source (Go 1.26.3):

make build
./bin/waxflow server --demo &   # loopback: keyless is allowed
./bin/waxflow ping
open http://127.0.0.1:4418/demo

Configuration

Precedence: flag > WAXFLOW_* env > JSON config file > default (config file via --config or WAXFLOW_CONFIG; unknown keys are rejected).

Key	Env	Default	Purpose
addr	WAXFLOW_ADDR	127.0.0.1:4418	listen address (compose widens to 0.0.0.0)
logLevel	WAXFLOW_LOG_LEVEL	info	debug|info|warn|error
roots	WAXFLOW_ROOTS	none	named library roots; JSON [{"name","path"}], env name=path,name2=path2; each opened via os.Root (no escape, symlinks confined), files validated regular and size-capped; a root whose folder does not open, or is empty, starts unmounted (see below)
catalogDB	WAXFLOW_CATALOG_DB	none	WaxBin catalog path for pid:<ULID> source references, read by a build that injects a catalog resolver (see pid: sources). No build here serves it: this server refuses to start with it set (one-shots on plain paths never read config, so they neither honor nor refuse it)
apiKeys	WAXFLOW_API_KEYS	none	control-API keys (comma-separated in env). Fail closed: required on a non-loopback addr unless allowUnauthenticated
allowUnauthenticated	WAXFLOW_ALLOW_UNAUTHENTICATED	false	explicit opt-in to keyless on non-loopback
sourceMaxBytes	WAXFLOW_SOURCE_MAX_BYTES	4 GiB	per-source open cap
metricsKey	WAXFLOW_METRICS_KEY	none	additionally unlocks GET /metrics
signingSecret	WAXFLOW_SIGNING_SECRET	auto-generated into dataDir (0600)	HMAC key for signed URLs; kid:hex,kid2:hex rotation list or a literal secret
allowedOrigins	WAXFLOW_ALLOWED_ORIGINS	none	CORS allowlist for playback endpoints
dataDir / cacheDir	WAXFLOW_DATA_DIR / WAXFLOW_CACHE_DIR	platform dirs	daemon state (signing secret, job store) / transcode cache
scratchDir	WAXFLOW_SCRATCH_DIR	temp dir + /waxflow	upload spool (the hardened container mounts a tmpfs)
uploadMaxBytes / uploadTTL	WAXFLOW_UPLOAD_MAX_BYTES / WAXFLOW_UPLOAD_TTL	2 GiB / 1h	one upload's size cap / spool eviction after creation (Go duration, 0 never)
scratchMaxBytes	WAXFLOW_SCRATCH_MAX_BYTES	8 GiB	aggregate spool cap (uploadMaxBytes only caps one upload)
cacheMaxBytes / cacheMaxAge	WAXFLOW_CACHE_MAX_*	10 GiB / off	LRU eviction policy (cacheMaxAge is a Go duration)
liveSlots / jobSlots	WAXFLOW_*_SLOTS	NumCPU-1 / 2	live admission pool (over limit means 503 + Retry-After: 2) / concurrent job workers (jobs queue and also pause while the live pool is saturated)
defaultGain	WAXFLOW_DEFAULT_GAIN	track	gain mode when gain= absent
resampleProfile	WAXFLOW_RESAMPLE_PROFILE	hq	hq or fast (constrained hosts)
tlsCert / tlsKey	WAXFLOW_TLS_*	none	native TLS; else put a terminating proxy in front (ADR-0007)
debugAddr	WAXFLOW_DEBUG_ADDR	off	loopback-only pprof listener
paceBurstSeconds / paceFactor	WAXFLOW_PACE_*	30 / 2.0	read-behind delivery pacing (factor 0 disables)
liveIdleGrace / hlsIdleGrace	WAXFLOW_LIVE_IDLE_GRACE / WAXFLOW_HLS_IDLE_GRACE	5s / 60s	stop a /stream encode once every client reading it has been gone this long / an HLS variant worker once nobody has fetched from it this long (Go durations, 0 never); see ADR-0012
demo	WAXFLOW_DEMO	false	serve the browser test page at /demo (dev only)

Runtime root reload. A daemon configured through the JSON roots (not WAXFLOW_ROOTS) serves POST /roots/reload, which re-reads the config and reconciles its live library roots so a root added at runtime streams without a restart. It reconciles only what the resolver owns (the roots and sourceMaxBytes); every other setting still needs a restart. It is wired only when a reload could do something, a config file is set and WAXFLOW_ROOTS is not pinning roots, so env-only and no-config deployments report delivery.rootsReload: false in /caps and keep serving new roots by direct download. The integration contract for a caller that adds a root (WaxDeck): write the root into the sidecar's JSON config atomically (write a temp file, then rename), then POST /roots/reload; the atomic write keeps a reload from reading a half-written file and 400ing.

Unmounted roots. A root whose folder does not open, or is empty (an unplugged drive, a mount point whose drive has not arrived, a mount that failed), does not stop the daemon or fail a reload: the other roots serve, requests on that one answer 503 root-unavailable with Retry-After: 2, and it mounts on the first request after its folder holds something, or within 10 s by the daemon's sweep, which also notices a root whose folder goes away later. A folder that stops answering before it mounts (a dead network mount) holds no request, reload or sweep for more than 5 s; one that hangs while mounted holds the requests reading it, as any blocked read does. The log names each change once, whoever finds it (Warn for a missing or empty folder, Error for anything else, a mistake that needs an operator), GET /roots lists every root with its state, a reload's unmounted field names the roots it leaves down, jobs on an unavailable root wait for it instead of failing, and waxflow doctor fails each one by name (an empty one is a warning).

CLI

• waxflow server: run the daemon (--demo for the browser test page)
• waxflow probe <file>: identify a file and print stream parameters (--json for the schemaVersion'd machine shape, identical to GET /probe; --strict to treat tolerated input damage as errors, which on a frame-indexed payload such as MP3 or ADTS also walks it to its end so the verdict covers the whole file; without it a probe reads headers, and damage past the head is reported by the read that reaches it)
• waxflow transcode <in> <out>: local one-shot file-to-file transcode through the same engine the daemon uses (--format wav|aiff|flac|mp3|aac|he-aac|alac|opus|vorbis|wavpack|ape, --flac-level, --wavpack-level, --ape-level, --mp3-bitrate, default from the output extension; --force to overwrite). An mp4-family output (.m4a, .m4b, alac) is written flat: a file can satisfy the header back-patch that form needs, and it is the shape players and taggers expect. The fragmented (CMAF) form is for delivery, and /stream and HLS keep it. Metadata (tags, chapters, cover art, lyrics) passes through onto the output automatically (--no-tags to skip); --loudness analyze measures the source, applies the exact gain to the ReplayGain reference, and writes measured RG tags on the output. Ogg-FLAC, WavPack, and Monkey's Audio outputs get a projection there rather than a measurement, and say so: their muxer takes its tags before the encode and cannot be patched afterward.
• waxflow split <in> <dir>: cut a single-file rip into one output per track, from a CUE sheet (--cue album.cue) or explicit source-sample offsets (--at). Cut points are samples either way: a sheet's MM:SS:FF times are CD frames of 1/75 s, which every CD-family rate divides exactly (44100/75 = 588), so a boundary converts with no rounding, where seconds would land a sample off and click at every join. The default FLAC output at the source's own rate makes the cut bit-exact, so the pieces rejoin into the original, less a mixed-mode disc's data track, which is skipped rather than written as audio, under the disc's own numbering. --dry-run prints the pieces and their ranges
• waxflow sign --src lib/a.flac: mint a signed playback URL offline (ADR-0003; uses the same secret and roots the daemon holds)
• waxflow cache stats|gc: inspect or evict a running daemon's cache
• waxflow doctor: check the local environment a daemon needs: config resolves, every root opens and reads, the cache/data/scratch dirs accept writes, the WaxBin catalog opens (builds with a catalog resolver), a quick self-bench transcodes faster than realtime, and the absence of ffmpeg is confirmed to be fine (--json for the machine shape)
• waxflow ping: liveness probe; the container HEALTHCHECK
• waxflow version: version and build info
• waxflow exit-codes: print the documented exit-code contract (0 ok, 1 internal, 2 invalid, 3 not-found, 4 io, 5 unsupported, 6 canceled, 7 unauthorized, 8 overloaded, 9 malformed)

The HTTP surface is documented in docs/api.md.

pid: sources

WaxFlow ships no catalog code and no database dependency, so every build in this repo answers pid: with 501 unsupported-source.

A build that wants them injects a catalog resolver through the cli.Flavor seam (cli/root.go), whose OpenResolver hook wraps the library roots with extra source schemes; catalogDB is the configuration field carried across it. examples/catalogcli/ is a working CLI built that way, and the module to copy from. Once such a build resolves pid:, every surface that takes a source reference accepts it (/stream, /probe, /sign, jobs, HLS, plus waxflow probe|transcode|sign), and /caps reports delivery.pid.

What holds regardless of who resolves them: signed URLs pin bytes, not locations, so a catalog rename or move does not kill a minted URL, while replaced content still dies with 410 source-changed.

Development

make check           # gofmt + vet + test + test-race + nested modules + depcheck
make test            # root-module suite, no race detector (the fast default loop)
make test-race       # race detector over the root module (heavy numeric suites self-skip)
make test-cli        # the cli module (cobra CLI + waxlabel mapper), race included
make test-oracle     # the third-party-oracle tests (waxlabel round trips, go-mp3)
make test-example    # examples/catalogcli: the out-of-prefix cli.Flavor canary
make soak            # 30m streaming soak + load + TTFA percentiles (nightly-scale)
make client-e2e      # browser client-matrix cells via Playwright (gated tooling)
make docker          # local image build
make verify-vectors  # fetch SHA-256-pinned conformance vectors (CI-cached)
make goldens         # regenerate muxer golden files (review the diff)

• Architecture invariants live in docs/adr/. Read ADR-0001 (clean-room policy) before touching codec code.
• Encoder/decoder acceptance thresholds are pinned in docs/quality-gates.md; gates only ratchet up.
• Defects that are understood but deliberately unfixed, with the reason, are in docs/deferred-work.md; what WaxFlow wants from the sibling Wax repos it depends on is in docs/upstream-requests.md.
• ffmpeg is a test oracle only (differential CI job), never a runtime dependency.
• Releases are tag-driven: pushing vX.Y.Z publishes binaries + SHA256SUMS and a multi-arch (amd64/arm64) image to ghcr.io/colespringer/waxflow.

License

MIT. Third-party attributions: THIRD-PARTY-NOTICES.md.