sparsity

Iterative magnitude pruning in PyTorch for finding lottery tickets (Frankle & Carbin), with late rewinding, quantisation-aware training and resumable checkpoints. Started for my ML MSc.

Neural Network Compression Framework for enhanced OpenVINO™ inference

PyTorch native quantization for training and inference

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

[NeurIPS 2025] This is the official repository for VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set

Source code of ACL 2023 Main Conference Paper "PAD-Net: An Efficient Framework for Dynamic Networks".

Source code of EMNLP 2022 Findings paper "SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters"

The official implementation of the paper "Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity".

The official implementation of the paper "Understanding and Harnessing Sparsity in Unified Multimodal Models" (TMLR).

A research draft on sparsity-sensitive bounds for odd-power autoencoders; proof and reproducible checks.

How Divergence Becomes Decision Flips in Compressed Language Models

Structured sparsity in Transformer FFN layers under an equal-parameter constraint. Introduces mixing depth (tau), validated with pre-registered predictions. Code, 324 training runs, and paper sources.

Reproducible Apple M2 study of dynamic FFN block sparsity for LLM inference: quality vs. latency, packed vs. per-neuron kernels, with all raw data and validators

SinkSLOT: Sinkhorn via Sparse Lifted Optimal Transport

VGG Magnitude Pruning: PyTorch Implementation

SGHS: Dynamic Hybrid Sparse Pruning via Fine-grained Sensitivity Allocation for LLMs — one-shot N:M semi-structured pruning with sub-block heterogeneous patterns

Deterministic search over per-layer bit-widths to fit a model into a fixed VRAM budget. An LLM agent proposing allocations lost to a plain greedy loop by 4,386x at the same memory envelope; 2:4 structured sparsity lost to dense by 3,307x at equal memory.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

Sparse and one-bit steering vectors on the original GPT-1: how few residual dimensions does it take to flip sentiment? Paper + open-weight vectors.

Automatic Sparse Differentiation in JAX.