quantization

RTX 5090 Laptop (24GB) + ComfyUI: 12 model lines / 21 measured configs — MiniMax H3, Wan 2.2, Qwen-Image 2512, HunyuanVideo 1.5, FLUX.2 Klein, Z-Image-Turbo, LTX-Video, ACE-Step 1.5, YuE2, Stable Audio 3, Hunyuan3D 2.1, ERNIE-Image. A controlled A/B proves NVFP4 beats INT8. Bilingual 中文/English, licence-verified.

AIMET GitHub pages documentation

AI-powered Quantitative Investment Research Platform.

📱 手机端 AI 操作系统全景知识库 — 334+ 篇深度页面，覆盖端侧大模型、AI Agent、芯片适配、推理优化 | 自动更新

🌐 Run GGUF models directly in your web browser using JavaScript and WebAssembly for a seamless and flexible AI experience.

ThaiLLM-30B NVFP4 quantization study on NVIDIA DGX Spark: 3.4x smaller, 2.3-2.5x faster, Thai capability statistically unchanged. Full reproducible pipeline + benchmark report by AGIcafet.

Interactive real-time viewer and search engine across 100 million images, 50 million comments, and 20 million websites, built on ClickHouse.

[DAC'24] EmMark: Robust Watermarks for IP Protection of Embedded Quantized Large Language Models

Interactive deep dive into LLMs — built from Andrej Karpathy & Julia Turc's lectures. 8 videos, 49 direct quotes, 21 interactive demos, zero dependencies.

A benchmark for evaluating local LLMs on Gaokao (China college entrance exam) math problems.

A self-contained AI project that runs a quantized Large Language Model (Qwen2.5-0.5B) entirely on your local machine. Built with FastAPI and llama-cpp-python, this agent intelligently switches between standard chat and "Search Mode" to fetch real-time data from the internet. The project features a responsive HTML/CSS/JS frontend and is fully Docker

An AI-powered MLOps assistant for effortless model compression. Upload PyTorch models to chat with a local LLM expert, receive hardware-aware optimization advice, and perform one-click FP16/INT8 quantization to reduce model size and latency.

Mac/Apple Silicon 本地部署 Qwen3.8-27B 指南：Ollama + MLX/GGUF 量化选型、MTP 加速、国内镜像与实测避坑（M5 Pro 48GB）。

Auditable multimodal compression experiments with task-specific regressions and measured calibration budgets. P0 research prototype.

"Memory-efficient, trimodal AI tutoring kernel designed for offline use on android tablets and phones."

Estimate the GPU memory a local LLM needs, with the arithmetic shown, an honest range instead of a fake-precise number, and a fits-or-not verdict for common GPUs.

Measured guide to running a 28B LLM offline on an M1 Pro with MLX, plus a primer on reasoning models

A bilingual field guide to Inference Engineering: the path from product constraints and model mechanics through hardware, runtimes, optimization techniques, modalities, and production operation.

Measure and visualize why LLM inference is slow: bottleneck analysis, model dissection, KV-cache, GEMM/GEMV, quantization, and memory-bound decoding.

Reproducible benchmark scoring 12 local GGUF models on end-to-end code generation (2D + 3D Breakout) — playable demos, per-model reports and machine-readable results.