parameter-efficient

Toward controlled evolution of artificial intelligence through validated neural grafting.

ASSTF (Adaptive State-Space Transfer Function): A PyTorch framework for dynamic neural topology that reduces parameters by 5-10x, enables test-time adaptation, and outperforms static models on structure-sensitive tasks.

Various LoRA adapters. One shared basis. Up to 122× compression at scale.

Train the smallest LM you can that fits in 16MB. Best model wins!

How many parameters are needed to get 99% on MNIST? Personal record of 697 parameters.

Parameter-efficient fine-tuning of BERT for binary sentiment classification using QLoRA (4-bit NF4 quantization + LoRA adapters) on the IMDb 20k dataset. Reduces trainable parameters by ~99% and GPU memory by ~70% vs full fine-tuning. Runs on CPU locally and full QLoRA on GPU (Colab/Kaggle).

Reduce LLM inference compute by 4x with no accuracy loss. Oscillatory adapter for pretrained Transformers.

A modular and extensible LoRA fine-tuning framework for question-answering tasks with PEFT integration

BiDoRA: Bi-Level Optimization for Parameter-Efficient Fine-Tuning of LLMs - Optimized for 3D Code Generation

Parameter-efficient NLI: frozen BGE encoder + LoRA adapters (1.77M trainable params), F1 0.823 with OOD analysis

Official source code for the paper "Tailored Design of Audio-Visual Speech Recognition Models using Branchformers"

PEFT-compatible SingLoRA: single-matrix low-rank adaptation with half the adapter parameters of standard LoRA, drop-in for transformers workflows.

This repository contains the source code for the paper "Grouped Pointwise Convolutions Reduce Parameters in Convolutional Neural Networks".

ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, verified on llama.cpp GGUF (Qwen3.8-Flash-Next).

Code for AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP tasks

Frame Flexible Network (CVPR2023)

This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.12410).

A parameter-efficient mixture-of-experts module for computational pathology - ICLR

K-CAI NEURAL API - Keras based neural network API that will allow you to create parameter-efficient, memory-efficient, flops-efficient multipath models with new layer types. There are plenty of examples and documentation.

Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"