SGLang is a fast serving framework for large language models and vision language models.

A workload for deploying LLM inference services on Kubernetes

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

LeaderWorkerSet: An API for deploying a group of pods as a unit of replication

Run OpenClaw more securely inside NVIDIA OpenShell with managed inference

Client for OpenClaw — Connect ClawWork to your own OpenClaw and unlock 10x multi-session productivity.

Community Repo for Nowledge Labs Products

Offline optimization of your disaggregated Dynamo graph

Bub it. Build it.

xURL: Client for AI Agents URLs

A Datacenter Scale Distributed Inference Serving Framework

Variant optimization autoscaler for distributed inference workloads

A framework for efficient model inference with omni-modality models

a CLI tool to compare CustomResourceDefinitions and identify breaking changes

AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

A light weight vLLM simulator, for mocking out replicas.

agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.

Kubeflow SDK for ML Experience

Distributed ML Training and Fine-Tuning on Kubernetes

Benchmark and optimize LLM inference across frameworks with ease

Cost-efficient and pluggable Infrastructure components for GenAI inference

Apache OpenDAL: access data freely.

A lightweight, local-first, and free experiment tracking Python library built on top of 🤗 Datasets and Spaces.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.