Dynamo is NVIDIA's open-source, datacenter-scale distributed inference framework. It is the orchestration layer above inference engines (SGLang, TensorRT-LLM, vLLM), not a replacement for them: it turns a cluster of GPUs into one coordinated inference system. Core capabilities are disaggregated prefill/decode serving, KV-aware routing, multi-tier KV cache management (KVBM: GPU → CPU → SSD → remote), SLA-driven autoscaling (Planner), in-flight fault tolerance, and a Kubernetes operator for deployment.

The stack is deliberately layered and large. A Rust core (a Cargo workspace of twenty-plus crates, mostly under lib/) holds the runtime, LLM, routing, and KV-block-manager engines. A Python extensibility layer (the ai-dynamo wheel, bound to the Rust core through PyO3/maturin) holds the frontend, backends, planner, and profiler. A Kubernetes layer (deploy/) holds the operator, Helm charts, and gateway integration. Treat any change that crosses these boundaries as non-trivial. Dynamo also sits inside a wider ai-dynamo ecosystem of sibling repos (below) that it integrates with rather than vendors.

• AGENTS.md is the canonical and only source of agent instructions at every repository scope.
• Every AGENTS.md must have a sibling CLAUDE.md containing exactly @AGENTS.md so Claude loads the same instructions.
• Do not put independent instructions in CLAUDE.md, and do not make AGENTS.md a symlink to CLAUDE.md.
• Add, move, or remove each AGENTS.md and CLAUDE.md pair together.

Skills live canonically in .agents/skills/; skills/ and .claude/skills/ are symlinks to it — edit only the canonical copy. Reach for the right group first:

For developing Dynamo:

• debug-session — structured bug investigation with a persistent worklog
• dep-create — create or update Dynamo Enhancement Proposals as GitHub issues
• dep-status — check DEP status and list DEPs by lifecycle state or area
• dep-update — advance DEP lifecycle: triage, PIC assignment, review, approval
• dynamo-clone-hotpath-audit — audit Rust hot-path .clone() calls
• dynamo-docs — Fern docs-site content per the style guide
• dynamo-frontend-benchmark — benchmark/profile the frontend against mock workers
• fern-components — Fern MDX component library and usage guidance
• fern-navigation — Fern navigation and site-structure configuration guidance
• dynamo-kv-replay-parity — validate offline KV replay parity and performance
• dynamo-agent-harness — drive persistent Claude Code, Codex, or OpenCode sessions through Dynamo over ACP
• graham-code-review — strict Rust/systems review in Graham King's style
• issue-first — start from a tracked issue and reference it from the PR
• pr-monitor — CI health check, failure root-cause, and skip analysis
• repo-codeowners — who reviews a change, fixing a failing codeowners check, changing review routing
• visual-review — interactive HTML code-review dashboards with diagrams and annotated diffs

When reviewing frontend or runtime changes, also read the corresponding frontend review prompt or runtime review prompt for additional CODEOWNERS guidance.

For deploying and operating Dynamo:

• synthesize-user-workload — interview the user, capture their confirmed baseline DGD, and create the canonical workload contract
• author-baseline-dgd — draft a baseline DGD from interview requirements when no recipe matches, for the user's confirmation
• consult-perf-knowledge — select one evidence-backed optimization proposal and write its reasoning record
• create-optimization-hypothesis — materialize a performance consultation as a challenger-ready DGD draft
• perform-adversarial-review — challenge a generated DGD candidate before it consumes GPU time
• deploy-dynamo-recipe — deploy one assigned Kubernetes DGD and verify it with an API smoke test
• configure-aiperf-benchmark — freeze and render a comparable AIPerf workload for a deployed candidate
• run-aiperf-benchmark — execute and collect one run-scoped AIPerf Kubernetes benchmark
• analyze-aiperf-results — validate AIPerf evidence, evaluate SLOs, and compare valid same-series runs
• find-serving-recipe — walk the ordered recipe catalogs with provenance gates and write a recipe dossier
• report-skillpack-issue — file a sanitized, operator-approved GitHub issue for a defect in the pack itself
• dynamo-router-starter — start/patch router modes with smoke checks
• dynamo-interconnect-check — validate NIXL/UCX/NCCL readiness for disaggregation
• troubleshoot-dynamo — diagnose failed or unhealthy deployments

Adding a skill: the folder name must equal the frontmatter name (kebab-case); the description is third person, states what the skill does and when to use it, and is at most 1024 characters; include license: Apache-2.0 and a metadata: block with author and tags. List the skill in this section — the index must match .agents/skills/ exactly. All of this is enforced by scripts/validate_skills.py (pre-commit hook validate-skills). Changes under .agents/skills/ are also validated by NVSkills CI — a maintainer comments /nvskills-ci on the PR.

If these skills, instructions, role contracts, or this file misled you, blocked you, contradicted what you verified live, or left a component or situation uncovered, do not route around it silently: invoke the report-skillpack-issue skill (.agents/skills/report-skillpack-issue/), which every role contract declares and which owns the full procedure. Dispatched roles record drafts in <EXP_ROOT>/analysis/skillpack-defects.md and return them; only the top-level agent, with operator approval, files, at most one new issue per session with findings batched, plus comments on duplicates. If your harness cannot surface the skill, follow this minimum, which the skill also enforces: search existing reports by title, body, and comments first; identify yourself as an AI agent with your driver model and the skills commit; sanitize every emitted string (title, body, comment, search term: no workload details, traffic numbers, cluster or namespace names, company names, or credentials); keep drafts as run-scoped files, never in a shared temp path; and show the operator the exact draft and file only on their approval, using the [AGENT]: title prefix and verifying the label landed.

When the first user message starts a new Dynamo recipe optimization run, FIRST read agent-docs/guides/optimization/optimize-loop.md end to end - the individual SKILL.md files are auto-discoverable, but the loop's sequencing, state machine, and stopping rules live only in that guide - then dispatch user_interviewer before any other specialized role. It must invoke synthesize-user-workload and produce a validated <EXP_ROOT>/user_workload.yaml plus an immutable <EXP_ROOT>/inputs/user_provided_dgd.yaml copied from the baseline the user supplied or explicitly confirmed. Do not dispatch recipe_deployer, perf_analyzer, hypothesis_generator, or hypothesis_challenger until both exact paths and SHA256 values are available. Pass both inputs directly to recipe_deployer; pass the same immutable workload path and hash to every later role. Do not insert a recipe exploration or selection step after the interview: the baseline-source ladder (agents/user-interviewer/AGENTS.md) is the only place selection or authoring happens, always with the user's explicit confirmation, and always before the loop starts.

An optimization loop is long-running, unattended work. Know which harness you are in: in a SINGLE-SHOT harness (headless -p/print mode, one-turn API calls), background-job completion notifications can never reach you - the session is gone when your turn ends. There, poll synchronously with bounded loops and never park the engagement on a wake-up you cannot receive; parking is only valid where the harness can re-invoke you (interactive sessions, goal mode). An interactive harness ends its turn whenever the agent stops calling tools — a turn that ends on narrated intent ("now I'll test disagg") silently stalls the loop until a human notices. Two rules:

1. Operators: launch unattended runs inside your harness's goal mode. Goal mode is an operator action at launch, not something these instructions can enable mid-run. On Codex CLI, wrap the run in /goal with a token budget. On Claude Code (v2.1.139+), wrap it in /goal; its completion condition is model-evaluated and may include a bound such as "or stop after N turns" as part of the condition text (a soft limit, not a hard budget). A validated condition template: "Test every lever family that is testable within the authorized budget. Never stop because a report exists. Parked on pending asks with nothing else testable is a valid pause, not completion. Valid stops: an operator-granted stop-request, the authorized budget exhausted, access lost, or operator interrupt." Always name the budget (GPU-hours, wall-clock, failed-deployment limit) in the condition; a bare "never stop" silently relies on credential expiry as its budget. Tell the operator at the START of any optimization engagement — not only when they say "unattended" — that this is long-running work and how to arm goal mode; the user-interviewer's contract handoff is the natural moment. Arm goal mode only AFTER the contract questions are answered: a goal hook armed while questions are outstanding forces the run past them onto its own defaults. The template's parked-on-asks pause assumes a reachable operator: for runs where the operator will be away, instruct the agent not to park on asks (asks are logged and the loop continues) and keep only the hard stops. Blocking question tools suspend the turn BEFORE the goal hook can evaluate, so one blocking question can hang an unattended run for hours; harnesses that support tool restrictions should disallow blocking question tools in goal mode.
2. Never end a turn on narrated intent during a loop. Either perform the next step in the same turn, launch it as background work that will re-invoke you, or return the specific blocking question you need answered.

Harness tiers. These roles and skills are developed and tested on Claude Code and Codex CLI. Isolated role configurations currently ship for Codex only (.codex/agents/*.toml); on Claude Code the roles run in-context within one session (no .claude/agents/ configurations yet), so adversarial review there is same-context review, not an independent reviewer. The skills follow the Agent Skills open standard and load on other compliant harnesses (for example, OpenCode includes .agents/skills/ among its standard skill search paths), with the same in-context role caveat plus two more degradations: no native goal mode (run lights-out sessions under an external loop), and every rule in this pack is prompt-enforced, so discipline depends on the driver model. If you hit an instruction gap on any harness, prepare a sanitized issue describing the gap and ask your operator to approve filing it on this repository.

Sibling repositories this repo integrates with:

System prerequisites (Rust toolchain, uv, system libraries) and the VS Code / Cursor devcontainer are covered in the contribution guide.

Python dev build (bindings + wheel, editable):

uv venv .venv && source .venv/bin/activate
uv pip install pip 'maturin[patchelf]'
cd lib/bindings/python && maturin develop --uv && cd -
uv pip install -e lib/gpu_memory_service
uv pip install -e .
python3 -m dynamo.frontend --help   # verify

Rust-only:

cargo build                 # whole workspace
cargo build -p dynamo-llm   # one crate

cargo test                  # Rust
pytest -m unit tests/       # Python unit tests

On macOS, run dynamo-llm checks and targeted tests with --no-default-features unless the target explicitly requires block-manager. The default feature enables Linux/CUDA-oriented NIXL, NUMA, and O_DIRECT code that is not a valid general-purpose macOS validation path.

Markers are strict (--strict-markers); the full marker list lives in pyproject.toml [tool.pytest.ini_options], including GPU gating (gpu_0 … gpu_8). Read .ai/pytest-guidelines.md and .ai/test-model-size-guardrails.md before writing tests.

pre-commit run --all-files            # all hooks (run `pre-commit install` first; it also installs the DCO commit-msg hook)
cargo fmt --all && cargo clippy --workspace

• Keep changes focused and reviewable.
• Use Conventional Commit PR titles: type(scope): summary. Accepted types: feat, fix, docs, test, ci, refactor, perf, chore, revert, style, and build.
• PR descriptions must include Summary and Validation.
• Every PR references the tracked issue it implements, in the title, description, or branch name (Closes #123, Closes DYN-1234, user/dyn-1234-description). The PR Issue Link check verifies the reference; it is advisory until 2026-10-21, then required. Reference forms and remediation are in the issue-first skill.
• Sign every commit with DCO: git commit -s.
• For fork PRs that qualify for automatic CI approval, GitHub must report every commit's signature as Verified. A DCO sign-off alone is not enough. Signing commits does not grant automatic approval; a maintainer can approve the current head manually with /ok to test <sha>. The only signature exception is a full-CI request from glamr-agent on its own PR. The commenter, PR author, and head repository owner must all be glamr-agent, checked by GitHub account ID. The request's full SHA must match the PR's current head. Other automatic approvals still require verified signatures.
• Do not hand-edit a generated artifact — change its source and regenerate. A generated file says so in a do not edit marker, and its generator has a --check mode that fails when the committed output is stale. Resolve a conflict in a generated file by regenerating rather than editing the conflict — a hand-resolved artifact passes review and then fails the next --check — and resolve one in an aggregate list, such as a coverage set or a filter list, as the union of both sides.
• Do not hand-edit the root CODEOWNERS — it is generated. To change review routing, edit .github/codeowners/areas.yaml and regenerate; CI gates 100% coverage and CODEOWNERS↔areas.yaml drift. See .github/codeowners/README.md. To check who reviews your PR: python .github/codeowners/who_owns.py --codeowners CODEOWNERS --changed (--people expands teams to members for org members). If the codeowners check fails after adding a new directory, claim it with one line under the owning area in areas.yaml, regenerate, and commit both files together. External contributors earn area-scoped co-ownership via .github/codeowners/external_contributors.yaml. The repo-codeowners skill automates all of this.
• Full CI on a PR runs only after a maintainer comments /ok to test <sha> with the short SHA of the latest commit; copy-pr-bot then creates the pull-request/N branch that triggers it. For an eligible fork PR, automatic approval posts this command after the checks above pass. Fix CI failures before requesting human review.
• Architecture changes require a Dynamo Enhancement Proposal (DEP), filed as a GitHub issue on ai-dynamo/dynamo with dep:* labels (the dep-create skill automates this).

See the contribution guide for the full workflow (issue sizing, CODEOWNERS, review process).

Any change under docs/, examples/, or recipes/ must follow docs/fern/AGENTS.md and the documentation style guide: SPDX headers, Fern frontmatter (no body # H1), GitHub-style admonitions, and backend casing (vLLM / SGLang / TensorRT-LLM). The deterministic subset is enforced pre-merge.

The docs site is tab-based: docs/fern/pages/ splits into kubernetes/ and cli/ (parallel guides for two readers), plus use-cases/, recipes/, developer-guide/, reference/, blog/, and community/. Read docs/fern/pages/AGENTS.md to pick the right tab before adding a page — a misplaced page costs a move plus a redirect.