Open-source collaborations and apps

• vLLM (96K+ ⭐) — 10 commits merged to main, in the top ~9% of its ~3,700 contributors

  10 merged PRs: Canary, Parakeet and Nemotron diarization support, Gemma 4 GPU image normalization, fixes

  • Added NVIDIA Canary (canary-1b-v2): speech recognition and X↔English speech translation in 25 European languages, served through the OpenAI-compatible audio API; output is token-identical to Transformers, with 12.34 WER on earnings22 (#64215)
  • Made Canary transcription return a 400 when language is missing, instead of silently answering non-English audio in English, and added CI coverage for Canary translation (#65445)
  • Built vllm-parakeet-plugin for NVIDIA Parakeet TDT v3 (parakeet-tdt-0.6b-v3), now listed in vLLM's supported models (#65005): batched greedy TDT decoding on the GPU with its own CUDA graphs; 73/73 transcripts token-identical to Transformers, and LibriSpeech WER 1.93% / 3.61% through vllm serve (model card: 1.93% / 3.59%)
  • Built vllm-nemotron-diarization-plugin for NVIDIA Nemotron-3-Diarization, now listed in vLLM's supported models (#65199): speaker diarization of up to 8 speakers at 10 ms resolution through vLLM's LLM API with one pip install; logits within 2e-5 of Transformers, with identical speaker segments
  • Fixed main's failing pre-commit after two merged PRs collided on Muse Spark's weight-loading hook, which also restored its ROCm monokernel path (#65080)
  • Fixed chat derender misclassifying answers as reasoning: the reasoning and tool parsers never saw the prompt in the usual render → generate → derender flow, so they now fall back to the caller's prompt token IDs (#63434)
  • Moved Gemma 4's image normalization from the CPU processor onto the GPU (mm_device_do_normalize), with a perplexity test against the CPU path (#65577)
  • Extended GPU image normalization to Gemma 4 Unified (the 12B checkpoints), and turned back on 36 of its processor tests that CI had been skipping (#65724)
  • Fixed CLIP and SigLIP returning a different embedding for text sent as chat messages than for the same text sent as input: the fallback chat template dropped the tokenizer's BOS/EOS, so each model now gets a template that keeps them (#66417)
  • Fixed the same mismatch for SigLIP2: its checkpoints ship Gemma's chat template, which vLLM ranked above its own, so SigLIP models now use the SigLIP template and the NaFlex variants keep EOS too; also opened the matching fix on the Hugging Face Hub (#66624, Hub PR)

• SGLang — fixed the diffusion LoRA merge cache silently reusing stale merged weights after an adapter is rewritten in place, with regression tests (#44204), and made a diffusion worker exit after an unrecoverable CUDA error, instead of serving on a broken device while /health still reported ok (#45065) (40K+ ⭐)

• oh-my-pi — 5 PRs merged to main, in the top ~12% of its ~840 PR authors; 5 fixes to the coding agent: lsp reload now stops a superseded shared language server instead of leaving it running for 5 minutes (#18254), omp stats no longer re-parses every session after ZFS renumbers the filesystem's device (#18536), rendered SVGs no longer drop their text when a generic font family names a font that isn't installed (#18538), worktree add no longer fails on read-only files copied from the source checkout (#18537), and the browser tool records non-Error page rejections as JSON instead of a bare Object (#18251) (38K+ ⭐)

• PEFT — fixed FRoD and EVA initialization crashing on Apple Silicon, where MPS has no float64, and the 13 test failures Hugging Face's new macOS CI found (#4102) (21K+ ⭐)

• NVIDIA Dynamo — improved disaggregated serving and inference infrastructure: made the prefill router fail closed with a retryable 503 while a target activates (#16736), sized multimodal cache budgets for DeepStack embeddings (#17249), fixed DCGM metrics scrape connection handling (#17387), and made the profiler size Hub checkpoints from their weight files instead of a parameter-count estimate that overstated quantized models 2–3× (#17563) (8.4K+ ⭐)

  Featured in ai-dynamo.org updates

  https://ai-dynamo.org/#updates

  

Projects I work on — at Fluid Inference, which was acquired by Levenza

• FluidAudio — speech AI SDK for iOS and macOS (3.2K+ ⭐)

  • Supports dozens of commercial applications
  • Parakeet TDT V3 Core ML: nearly 3,000,000 total downloads, peaked at 500,000 monthly downloads

• FluidUse — Core ML runtimes for decision models in Swift, including Decision 3.0 (D3Manager)

  Some Experiment Highlights: laya, GLiNER2.5-Decide, EmbeddingGemma 2, clef-text-0.6B, Decision 3.0

  • laya — Convai's typed decision model with 99.5% of ops on the Apple Neural Engine, 3.7 ms per decision on an M5 Pro (demo) · 1,323 likes · 111 reposts · 120K views

    Reactions on X: jackzampolin, JiliJeanlouis, ClementDelangue, victormustar

    @jackzampolin

    @JiliJeanlouis

    @ClementDelangue reposts

    @victormustar

  • GLiNER2.5-Decide — Fastino's 340M decision model on Core ML: ~4× faster, ~5× less peak RAM, half the size (demo) · 880 likes · 83 reposts · 69K views

    Reactions on X: fastinoAI

    https://x.com/fastinoAI

    

  • EmbeddingGemma 2 — text encoder 100% on the Apple Neural Engine; sorts 10,080 posts into topics live at ~620/s on a MacBook, any topic splits into subtopics in under a second (demo) · 693 likes · 51 reposts · 34K views

    Reactions on X: SeraAndroid, googlegemma

    https://x.com/SeraAndroid/status/2108629662238961694?s=20

    https://x.com/googlegemma/status/2115150306032193954

    

  • clef-text-0.6B — distilled Cloudflare's 9B clef-flash into a 0.6B decision model that runs 100% on the Apple Neural Engine; 1,000 support tickets triaged in 57 s on an M5 Pro, 48 ms per ticket, within a few points of the 9B on classification (demo) · 494 likes · 45 reposts · 32K views

    Reactions on X: art_zucker, ClementDelangue

    @art_zucker reposts

    @ClementDelangue reposts

  • Decision 3.0 — d3-lite, nano and mini on Core ML; d3-lite triages 1,000 pull requests into vLLM-SR workgroups live on an M5 Pro at 0.12 s per PR, 8.3 PRs/s (demo) · 432 likes · 42 reposts · 28K views

    Reactions on X: ClementDelangue, victormustar

    @ClementDelangue

    @victormustar

  Its distilled & finetuned models brought 1,000+ Twitter followers in 1 month

  

• möbius — model conversion and deployment across accelerators

  • 500,000+ monthly model downloads across the Core ML models on FluidInference