low-bit

An innovative library for efficient LLM inference via low-bit quantization

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

QuantFace: Towards Lightweight Face Recognition by Synthetic Data Low-bit Quantization

admm for cnn layerwise weight low bit quantization

Memory-efficient PyTorch optimizers with Adaptive Log-Space quantization, fused CUDA kernels, and Adafactor, CAME, APOLLO support.

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs.

2.898-BPW Qwen3-8B with direct-packed CPU/CUDA inference

Efficient low-bit KV-cache compression research with honest metadata accounting.

Pure-Julia CPU inference engine for BitNet b1.58 ternary LLMs

Microscaling (MX) low-bit quantization formats, implemented and benchmarked on VLMs.

A local testbed for 1-bit and ternary neural networks, BitNet research, and low-bit LLM inference.

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs.

Quantization toolkit for large language models: run and store huge models on small VRAM. Inspired by Unsloth's dynamic quantization.

qpeft: quantization-aware training (QAT) and PEFT for LLMs in PyTorch. EfficientQAT, QA-LoRA and PEQA for Hugging Face models at 2/3/4 bits: train scales, zero-points or LoRA adapters, then merge into a low-bit integer model (GPTQ layout) that is tested to equal the trained one. Pure-torch and torchao backends.