Skip to content

@EmbeddedLLM EmbeddedLLM

All

    Repositories list

    87 repositories

    • agentic-api

      Public
      Stateful API logic for agentic applications using vLLM
      Rust
      •
      Apache License 2.0
      •82•1•0•0•Updated Oct 6, 2026Oct 6, 2026

    • recipes

      Public
      Common recipes to run vLLM
      JavaScript
      •
      Apache License 2.0
      •442•0•0•0•Updated Oct 6, 2026Oct 6, 2026

    • vllm

      Public
      vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
      inferencepytorchtransformer
      inferencepytorchtransformergptamdgpurocmmodel-servingllmllm-inference
      Python
      •
      Apache License 2.0
      •23k•94•1•10•Updated Oct 6, 2026Oct 6, 2026

    • TokenVisor-recipes

      Public
      Inference recipes
      Python
      •0•0•0•0•Updated Oct 6, 2026Oct 6, 2026

    • litellm

      Public
      Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Re…
      Python
      •
      Other
      •12k•0•0•0•Updated Oct 6, 2026Oct 6, 2026

    • skypilot

      Public
      SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
      Python
      •
      Apache License 2.0
      •1.3k•2•0•0•Updated Sep 29, 2026Sep 29, 2026

    • pi-cliffcompaction

      Public
      CliffCompaction for Pi: mechanical, truncate-only context compaction
      TypeScript
      •
      MIT License
      •0•0•0•0•Updated Sep 27, 2026Sep 27, 2026

    • JamAIBase

      Public
      The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work tog…
      pythonworkflowai
      pythonworkflowaiserverlesschatbotspreadsheetsvelteorchestrationbaasagents
      Python
      •
      Apache License 2.0
      •48•1.1k•1•1•Updated Sep 24, 2026Sep 24, 2026

    • Infera

      Public
      More token goodput from frontier models. A distributed, SLA-aware serving mesh — disaggregated prefill/decode, KV-aware routing, and cache offload, tuned to you…
      Python
      •
      MIT License
      •6•0•0•0•Updated Sep 23, 2026Sep 23, 2026

    • fastsafetensors

      Public
      High-performance safetensors model loader
      Python
      •
      Apache License 2.0
      •37•0•0•0•Updated Sep 22, 2026Sep 22, 2026

    • wasm-tools

      Public
      CLI and Rust libraries for low-level manipulation of WebAssembly modules
      Rust
      •
      Apache License 2.0
      •360•0•0•0•Updated Sep 15, 2026Sep 15, 2026

    • eryx

      Public
      A Python sandbox using Wasmtime
      Rust
      •
      Apache License 2.0
      •14•0•0•0•Updated Sep 15, 2026Sep 15, 2026

    • mori

      Public
      Modular RDMA Interface
      C++
      •
      MIT License
      •101•0•0•0•Updated Sep 14, 2026Sep 14, 2026

    • TokenVisor-charts

      Public
      Shell
      •0•1•0•0•Updated Sep 8, 2026Sep 8, 2026

    • llm-d-router

      Public
      llm-d Router: The intelligent entry point for inference requests
      Go
      •
      Apache License 2.0
      •417•0•0•0•Updated Sep 1, 2026Sep 1, 2026

    • llm-d

      Public
      Achieve state of the art inference performance with modern accelerators on Kubernetes
      Shell
      •
      Apache License 2.0
      •822•0•0•0•Updated Aug 23, 2026Aug 23, 2026

    • Mooncake

      Public
      Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
      C++
      •
      Apache License 2.0
      •1.3k•0•0•0•Updated Aug 23, 2026Aug 23, 2026

    • tokenvisor-pi

      Public
      TypeScript
      •0•0•0•0•Updated Aug 20, 2026Aug 20, 2026

    • aiter

      Public
      AI Tensor Engine for ROCm
      Python
      •
      MIT License
      •615•0•0•3•Updated Aug 19, 2026Aug 19, 2026

    • vllm-rocm-wheel

      Public
      Python
      •
      Apache License 2.0
      •0•0•0•4•Updated Aug 13, 2026Aug 13, 2026

    • wasi-python-layer

      Public
      wasm32-wasip2 Python site-packages layer: build pipelines + CI + releases
      Shell
      •0•0•0•1•Updated Aug 12, 2026Aug 12, 2026

    • LMCache

      Public
      ROCm support of Ultra-Fast and Cheaper Long-Context LLM Inference
      Python
      •
      Apache License 2.0
      •2k•0•0•0•Updated Jul 21, 2026Jul 21, 2026

    • vllm-omni

      Public
      A high-throughput and memory efficient inference and serving engine for Omni-modality models
      Python
      •
      Apache License 2.0
      •1.9k•0•0•0•Updated Jun 30, 2026Jun 30, 2026

    • northwind-demo

      Public
      Agentic demo for the northwind dataset. to levels of capabilities increasing with number of tools provided.
      Python
      •1•0•0•0•Updated Jun 22, 2026Jun 22, 2026

    • vllm-responses

      Public
      Python
      •
      Apache License 2.0
      •0•0•0•0•Updated Jun 9, 2026Jun 9, 2026

    • tilelang

      Public
      Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels
      Python
      •
      Other
      •854•0•0•0•Updated Jun 1, 2026Jun 1, 2026

    • tilelang-to-flydsl-skills

      Public
      Python
      •0•1•0•0•Updated May 6, 2026May 6, 2026

    • apidayworkshop2026

      Public
      Python
      •
      Apache License 2.0
      •4•7•0•0•Updated Apr 25, 2026Apr 25, 2026

    • pytorch3d

      Public
      PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
      Python
      •
      Other
      •1.5k•0•0•0•Updated Apr 17, 2026Apr 17, 2026

    • ATOM

      Public
      AiTer Optimized Model
      Python
      •
      MIT License
      •156•0•0•1•Updated Apr 10, 2026Apr 10, 2026
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.