CUDA checkpoint and restore utility

High Performance LLM Inference Operator Library

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

DeepGEMM: clean and efficient BLAS kernel library on GPU

DeepEP: an efficient expert-parallel communication library

Fast and memory-efficient exact attention