All pull requests

Pull requests

• cuda: grow the CUDA graph cache cap while it evicts graphs still in use

  #238·LeoBorcherding opened Oct 2, 2026·

  ·

• Pin unslothai/llama.cpp#235 (keep unparsed Qwen3-Coder output as content)

  #236·danielhanchen opened Sep 28, 2026Member·

  ·
  3

• chat : keep unparsed Qwen3-Coder output as content instead of dropping it

  #235·danielhanchen opened Sep 28, 2026Member·

  ·
  4

• cuda: raise CUDA graph cache cap from 64 to 512 for tensor parallel

  #232·angliubc opened Sep 23, 2026·

  ·

• chat : accept packed XML tool calls in the Qwen3-Coder parser

  #231·danielhanchen opened Sep 22, 2026Member·

  ·
  2

• fit: measure a draft head that borrows the target's embeddings

  #227·danielhanchen opened Sep 22, 2026Member·

  ·
  3

• feat(qwen35): support d2t draft-vocabulary trim for MTP speculative decoding

  #225·Shreesh-Sree opened Sep 19, 2026·

  ·
  4

• examples : add speculative-tree, a tree-structured speculative decoding prototype

  #224·danielhanchen opened Sep 18, 2026Member·

  ·
  13

• Pin unslothai/llama.cpp#210 (server-side preemption and exact concurrency) at afd3248b82

  #211·danielhanchen opened Sep 9, 2026Member·

  ·
  5

• server: preemption notices, asynchronous parks and exact concurrency (pin branch on b10869)

  #210·danielhanchen opened Sep 9, 2026Member·

  ·
  32

• rpc: do not abort on shutdown when a teardown round trip fails

  #203·danielhanchen opened Sep 6, 2026Member·

  ·
  7

• server: do not abort a completion when the model emits invalid UTF-8

  #202·danielhanchen opened Sep 6, 2026Member·

  ·
  19

• server: reuse checkpoint state buffers from a bounded pool

  #201·danielhanchen opened Sep 6, 2026Member·

  ·
  17

• server: remove two host side stalls in the decode loop at high concurrency

  #200·danielhanchen opened Sep 6, 2026Member·

  ·
  14

• rpc: cut the protocol cost of uploading weights to a remote server

  #199·danielhanchen opened Sep 6, 2026Member·

  ·
  13

• CUDA: MMQ tile configuration and FP4 accumulation for GB10 (sm_121)

  #198·danielhanchen opened Sep 6, 2026Member·

  ·
  9

• server: preemption notices, asynchronous parks and exact concurrency together

  #197·danielhanchen opened Sep 6, 2026Member·

  ·
  47

• rpc: direct server-to-server tensor transfer for a layer split

  #196·danielhanchen opened Sep 6, 2026Member·

  ·
  14

• convert: support compressed-tensors mixed-precision NVFP4 checkpoints

  #195·danielhanchen opened Sep 6, 2026Member·

  ·
  7

• cuda: opt-in exact concurrency, byte-identical output regardless of neighbours or parks

  #194·danielhanchen opened Sep 5, 2026Member·

  ·
  30

• rpc: one tensor read per decode step and asynchronous split copies

  #193·danielhanchen opened Sep 5, 2026Member·

  ·
  42

• server: take the park and restore copies off the decode loop

  #192·danielhanchen opened Sep 5, 2026Member·

  ·
  24

• trace: event tracer for the RPC backend, rpc-server and llama-server

  #191·danielhanchen opened Sep 5, 2026Member·

  ·
  46

• server: tell a streaming client when its slot is parked and restored

  #190·danielhanchen opened Sep 5, 2026Member·

  ·
  19

• server: priority-aware preemption with an explicit slot park and unpark API

  #189·danielhanchen opened Sep 5, 2026Member·

  ·
  1