A high-throughput and memory-efficient inference and serving engine for LLMs

A high-performance and light-weight router for vLLM large scale deployment

A framework for efficient model inference with omni-modality models

Rust multimodal preprocessing library from SMG

The source of LMSYS website and blogs

SGLang is a fast serving framework for large language models and vision language models.

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

A parallelism VAE avoids OOM for high resolution image generation

Code for the paper "Efficient Training of Language Models to Fill in the Middle"

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

Open deep learning compiler stack for cpu, gpu and specialized accelerators

An elegant \LaTeX\ résumé template. 大陆镜像 https://gods.coding.net/p/resume/git

GPGPU-Sim provides a detailed simulation model of a contemporary GPU running CUDA and/or OpenCL workloads and now includes an integrated (and validated) energy model, GPUWattch.

This is the top-level repository for the Accel-Sim framework.