Skip to content

@neuralmagic Neural Magic

All

    Repositories list

    111 repositories

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •23k•17•0•38•Updated Oct 6, 2026Oct 6, 2026

    • DeepEP

      Public
      DeepEP: an efficient expert-parallel communication library
      Cuda
      •
      MIT License
      •1.5k•1•0•1•Updated Oct 6, 2026Oct 6, 2026

    • crucible

      Public
      Crucible: an autonomous goal-directed research loop engine
      kubernetesrustagents
      kubernetesrustagentsautonomous-researchllm
      Rust
      •
      Apache License 2.0
      •3•10•7•2•Updated Oct 6, 2026Oct 6, 2026

    • vllm-openshift-recipes

      Public
      0•0•0•0•Updated Oct 6, 2026Oct 6, 2026

    • vllm-vcr

      Public
      vLLM inference simulator and replay machine
      rustinferencevllm
      rustinferencevllm
      Rust
      •
      Apache License 2.0
      •2•8•15•14•Updated Oct 6, 2026Oct 6, 2026

    • eval-hub-contrib

      Public
      Community-contributed evaluation framework adapters for eval-hub
      Python
      •
      Apache License 2.0
      •26•0•0•1•Updated Oct 6, 2026Oct 6, 2026

    • nyann-bench

      Public
      Go
      •
      Apache License 2.0
      •2•1•0•12•Updated Oct 5, 2026Oct 5, 2026

    • model-validation-configs

      Public
      2•2•1•13•Updated Oct 5, 2026Oct 5, 2026

    • nm-vllm-omni-ent

      Public
      A framework for efficient model inference with omni-modality models
      Python
      •
      Apache License 2.0
      •1.9k•3•0•1•Updated Oct 5, 2026Oct 5, 2026

    • inspect_k8s_sandbox

      Public
      A Kubernetes sandbox environment for use with inspect_ai
      Python
      •
      MIT License
      •35•0•0•6•Updated Oct 5, 2026Oct 5, 2026

    • axolotl

      Public
      Go ahead and axolotl questions
      Python
      •
      Apache License 2.0
      •1.5k•0•0•6•Updated Oct 4, 2026Oct 4, 2026

    • llm-manifesto

      Public
      Declarative Kubernetes renderer for llm-d and vLLM deployments
      Python
      •
      Apache License 2.0
      •0•2•0•0•Updated Oct 1, 2026Oct 1, 2026

    • rh-custom-models-plugin

      Public
      Python
      •
      Apache License 2.0
      •0•0•0•0•Updated Oct 1, 2026Oct 1, 2026

    • inspect_evals

      Public
      Collection of evals for Inspect AI
      Python
      •
      MIT License
      •454•0•0•9•Updated Sep 24, 2026Sep 24, 2026

    • speculators-dashboard

      Public
      HTML
      •0•1•31•0•Updated Sep 24, 2026Sep 24, 2026

    • research

      Public
      Repository to enable research flows
      Python
      •2•4•0•4•Updated Sep 17, 2026Sep 17, 2026

    • eval-hub

      Public
      Go
      •
      Apache License 2.0
      •27•0•0•0•Updated Sep 16, 2026Sep 16, 2026

    • vllm-beam-search-plugin

      Public
      Beam search scheduler plugin for vLLM v1 with CoW block table forking
      Python
      •
      Apache License 2.0
      •2•3•1•0•Updated Sep 15, 2026Sep 15, 2026

    • gpu-pruner

      Public
      Non-destructive GPU based idle-culler for RHOAI/Kubeflow workloads
      kubernetesrustopenshift
      kubernetesrustopenshiftgpuk8skubernetes-operatorkubeflowrhoaikubeflow-notebooks
      Rust
      •
      MIT License
      •4•6•0•3•Updated Sep 11, 2026Sep 11, 2026

    • DeepGEMM

      Public
      DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
      Cuda
      •
      MIT License
      •1.4k•0•0•5•Updated Sep 4, 2026Sep 4, 2026

    • fs-offload-experiments

      Public
      Benchmark sweep experiments for FS Offloading
      HTML
      •0•0•0•0•Updated Sep 3, 2026Sep 3, 2026

    • nm-hard-tools

      Public
      MCP services for rigorous inference evaluation and benchmarking
      Python
      •
      Apache License 2.0
      •0•0•0•2•Updated Sep 1, 2026Sep 1, 2026

    • vllm-envs

      Public
      Fast disposable vLLM dev environments from any branch/commit, built on content-addressed layer caches
      Python
      •0•3•2•3•Updated Aug 26, 2026Aug 26, 2026

    • tokenspeed

      Public
      TokenSpeed is a speed-of-light LLM inference engine.
      Python
      •
      MIT License
      •310•0•0•3•Updated Aug 19, 2026Aug 19, 2026

    • tpu-inference

      Public
      TPU inference for vLLM, with unified JAX and PyTorch support.
      Python
      •
      Apache License 2.0
      •330•0•0•0•Updated Aug 18, 2026Aug 18, 2026

    • speculators

      Public
      A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
      Python
      •
      Apache License 2.0
      •242•0•0•1•Updated Aug 13, 2026Aug 13, 2026

    • gorilla

      Public
      Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
      Python
      •
      Apache License 2.0
      •1.4k•0•0•18•Updated Aug 4, 2026Aug 4, 2026

    • LMCache

      Public
      Redis for LLMs
      Python
      •
      Apache License 2.0
      •2k•0•0•1•Updated Jul 17, 2026Jul 17, 2026

    • every_eval_ever

      Public
      Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboar…
      Python
      •
      MIT License
      •50•0•0•17•Updated Jul 10, 2026Jul 10, 2026

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •3.6k•7•0•4•Updated Jul 10, 2026Jul 10, 2026
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.