A 6.87 GiB GGUF build of Qwen 3.8 27B — full 27B quality on 8 GB RAM. 96.7% HumanEval, 92.9% MMLU retention, no distillation.
Self-quantized Gemma 4 26B A4B to 4-bit on a 36GB Mac Studio, benchmarked it vs Qwen 3.5 27B, and patched native local tool calling through MLX-VLM into a Hermes operator harness.
双击 install.bat，12 GB 笔记本跑 27B（三进制 1.76 bit · 73K 上下文 · 57–66 tok/s）+deepseekUI优化插件
Nexus ZLC is an experimental offline toolkit for lossless GGUF compression and compressed-weight inference. It preserves original model data, supports compatible models through a custom WebGPU engine, and includes restoration and before-and-after perplexity tests.
Native EXL3 (exllamav3) inference engine for Apple Silicon: MLX-backed C++ engine, CLI and OpenAI-compatible HTTP server
Edge AI Inference Project: Deploy FP32, dynamic int8 quantized, and static int8 quantized mobilnet-v3 model on the Raspberry Pi 5, and compare performance
Viggle-Animate in one ComfyUI node, on an INT8 checkpoint, with automatic attention routing. Weight + model card: https://huggingface.co/aireet/viggle-animate-workflow
RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication
Self-hosted Vietnamese text-to-speech web app powered by VieNeu-TTS 0.3B Q4 GGUF, CUDA-accelerated llama.cpp, ONNX Int8 codec, FastAPI and uv. Generates 24 kHz WAV locally, supports long-form Vietnamese text, and requires no cloud TTS API or API key.
Efficient fine-tuning of Phi-3.5-Mini-Instruct using Unsloth and LoRA for Medical Question Answering. Demonstrates memory-efficient training, faster inference, and domain-specific adaptation to build an accurate healthcare AI assistant.
Medical QA fine-tuning of Qwen2.5 using Unsloth and LoRA for medical question answering. Features optimized training, reduced GPU memory usage, accelerated inference, and robust clinical knowledge adaptation for healthcare-focused conversational AI.
Official library of pre-optimized Tensorbit models. Ready-to-deploy LLMs and Vision Transformers for edge hardware, optimized via the Tensorbit P-D-Q pipeline.
Mistral Legal Advisor is a full-stack AI application that connects a fine-tuned Mistral language model running on Google Colab (GPU) with a local Flask-based frontend. The system generates comprehensive legal document requirements for startups based on user inputs, using an ngrok tunnel to securely bridge local and cloud environments.
CLI for running prism-ml's 1-bit Bonsai models locally. Auto-manages llama.cpp server, downloads from HuggingFace, exposes OpenAI-compatible API.
Helm charts for deploying quantized LLMs on Kubernetes via llama.cpp. Powers the AI agent layer of the Citation-Cosmograph ecosystem.
Fine tuned Llama 1B model using Qlora technique on a medical dataset and demonstrated a simple before and after fine tuning performance of the model.
BitNet-inspired 1-bit Quantized Transformer for efficient protein function prediction and biological sequence modeling on low-power devices.
This paper proposes evaluating pruning and quantization techniques to reduce model size and inference cost while maintaining classification accuracy on ecological datasets such as iNaturalist.
A complete, menu-driven AI model interface for Windows that simplifies running local GGUF language models with llama.cpp. This tool automatically manages dependencies, provides multiple interaction modes, and prioritizes user privacy through fully offline operation.
A privacy-first, fully offline neural machine translation application supporting 200+ languages. Built on Meta's NLLB-200 model, this tool runs entirely in your browser without requiring server infrastructure or internet connectivity after initial model download.
To associate your repository with the quantized-models topic, visit your repo's landing page and select "manage topics."