low-bit

An innovative library for efficient LLM inference via low-bit quantization

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

QuantFace: Towards Lightweight Face Recognition by Synthetic Data Low-bit Quantization

admm for cnn layerwise weight low bit quantization

Memory-efficient PyTorch optimizers with Adaptive Log-Space quantization, fused CUDA kernels, and Adafactor, CAME, APOLLO support.

2.898-BPW Qwen3-8B with direct-packed CPU/CUDA inference

Efficient low-bit KV-cache compression research with honest metadata accounting.

Pure-Julia CPU inference engine for BitNet b1.58 ternary LLMs

Microscaling (MX) low-bit quantization formats, implemented and benchmarked on VLMs.

Quantization toolkit for large language models: run and store huge models on small VRAM. Inspired by Unsloth's dynamic quantization.

Some investigations I made with AMD cloud credits: training language models from scratch, exploring low-bit and byte-level architectures, and running ablation studies.

A local testbed for 1-bit and ternary neural networks, BitNet research, and low-bit LLM inference.

qpeft: quantization-aware training (QAT) and PEFT for LLMs in PyTorch. EfficientQAT, QA-LoRA and PEQA for Hugging Face models at 2/3/4 bits: train scales, zero-points or LoRA adapters, then merge into a low-bit integer model (GPTQ layout) that is tested to equal the trained one. Pure-torch and torchao backends.

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs.

저비트(3·4비트) 가중치 양자화 오차의 저랭크 복원(W ≈ Q + LR)을 NumPy로 단순화해 구현 — RTN·QER·보존-복원 랭크 분할·대리지표. SmolLM2-135M·TinyLlama-1.1B 실제 선형층 49개에서 분할의 이득은 ρ_r(W)가 거의 결정(첫 층 q/k는 QER 대비 오차 30~61%↓)하지만, perplexity(WikiText-2)는 거꾸로 QER이 앞서고(3비트 80.9 대 121.1) 같은 비트 예산에선 4비트 RTN(22.7)이 둘 다 이김. 논문 재현 아님.