DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.

Nano vLLM

The fastest repo in history to surpass 100K stars ⭐. Better Harness Tools that make real things done. Built in Rust using oh-my-codex.

A high-throughput and memory-efficient inference and serving engine for LLMs

CUDA Core Compute Libraries

Draft-Target Disaggregation LLM Serving System via Parallel Speculative Decoding.

A curated list of awesome C++ (or C) frameworks, libraries, resources, and shiny things. Inspired by awesome-... stuff.

Modern C++ actor framework based on std::execution, only requires C++20.

A curated list of awesome header-only C++ libraries

A fast single-producer, single-consumer lock-free queue for C++

disruptor concurency pattern in c++

A disruptor thread-synchronisation data structure for C++11.

Port of LMAX Disruptor to C++