Notable changes to vla.cpp. Format loosely follows Keep a Changelog.
0.2.0 - 2026-08-12
- SYCL backend for Intel GPUs (Arc, Flex, Data Center Max, Xe iGPU).
VLA_DEVICEpicks the ordinal on CUDA and SYCL alike. Seedocs/backend/sycl.md. - Stable C ABI (
include/vla.h,libvla) and Python bindings over it (bindings/python). - Four more architectures: π0.5, VLA-Adapter, OpenVLA-OFT and VLA-JEPA.
vla-benchfor engine-only latency, and-hf user/repo[:file.gguf]to fetch a checkpoint on first use.vla-cli --text, tokenized byscripts/tokenize_prompt.pywith the tokenizer the architecture was trained on.- Release workflow publishing Linux x86-64 (CPU and CUDA), Linux aarch64 (CPU), macOS Metal and a GHCR image.
- Opt-in BF16 activations for Evo-1 and pi0 (
VLA_EVO1_BF16_ACT,VLA_PI0_BF16_ACT). Weight GEMMs, bias adds, residuals, norms and activations carry BF16; attention scores, softmax, RoPE and the flow-matching integrator stay F32. Needs CUDA and a BF16 checkpoint, and is ignored otherwise. - Opt-in fused attention for Evo-1, pi0 and SmolVLA (
VLA_EVO1_FA,VLA_PI0_FA,VLA_SMOLVLA_FA). Off by default: ggml's CUDA flash attention computes K/V at F16 whatever the input type, which measured 4 to 5 points lower on libero_object.
- One shared backend ladder (
src/backend.h) instead of a copy per arch. CMake rejects two accelerators in one build directory. - Shared headers for the Qwen3-VL tower, the DINOv2+SigLIP dual tower, the DiT time embeddings, the causal mask and CHW image preprocessing.
vla::graph_cachekeeps the compute graph acrosspredictcalls in nine architectures, not just GR00T N1.7. Output is unchanged.- llama.cpp pinned at b10331. GR00T N1.5 and N1.6 shift by up to 4.6e-4 on actions peaking near 0.87, from an upstream ggml kernel change in the SigLIP tower they share. The other nine architectures are bit-identical.
- Evo-1 encodes every camera view in one vision graph and one compute, rather than a
graph_computeper view. The arithmetic per view is unchanged, only the submission pattern. - BitVLA's ternary GEMM feeds four column tiles per CTA from one shared activation block, and pads its shared-memory row stride to 144 B to break 16-way bank conflicts.
VLA_BITVLA_NARROW_GEMM=1selects the previous one-tile kernel. - The CUDA BF16 kernels live in
src/cuda/and depend only on the public ggml header. The fetched ggml gets one addition to carry them: a function-pointer hook at the top of its CUDA op dispatch (scripts/patch_ggml_cuda_ext_hook.py). Left unregistered the pointer is null and ggml behaves exactly as shipped. This is the first llama.cpp patch since 0.1.0 removed the oldpatches/script, and it makes Python 3 a requirement for configuring any build. - llama.cpp's tool binaries are no longer part of the default build. Ask for one by name when you want it:
cmake --build build --target llama-mtmd-cli. vla_corelinks ggml alone; the VLA architectures call nollama_*API. Only the VLM path links llama.- Build snippets no longer pass
-DGGML_CUDA_GRAPHS=ON, which llama.cpp already defaults on.
- Reject checkpoint geometry that contradicts itself before it sizes a buffer, in smolvla, bitvla, gr00tn1d6, vla_adapter and the Qwen3-VL position resample.
- A peer that stalls mid-message no longer parks either server.
- Treat a missing state vector as zeros in every architecture rather than dereferencing it.
- Build every registered test before
ctest, so the four that were never built stop reporting as not run.
0.1.1 - 2026-07-04
vla-cli: one-shot inference from the command line (image + tokens to action), no server needed.scripts/quantize_gguf.py: repack LM weights to Q8_0/Q4_0. The loader runs quantized GGUFs directly (Q8_0 roughly halves the LM, near-lossless).
- One shared GGUF reader across the model loaders, replacing the per-arch copies.
- Cap inbound message size (256 MiB) and image count (16) on
vla-serverandvlm-server. - Scale CPU threads to the machine core count across all loaders instead of a fixed 4.
- Read GGUF file offsets as 64-bit and reject non-float embedding tensors in row-fetch.
- Reject out-of-range language tokens in OpenVLA-OFT and VLA-Adapter.
- Zero the padded action dimensions so only real action dims carry values.
- Reject images that do not match the model input size in VLA-Adapter and OpenVLA-OFT (out-of-bounds read on a smaller view).
- Validate Evo-1 action dims at load so a client-supplied noise buffer cannot underrun.
- Only enable the BitVLA CUDA path once every device buffer allocates.
0.1.0 - 2026-07-03
First tagged release. One self-contained GGUF per model (vision tower + LM + action expert + dataset stats), CPU or CUDA, no external mmproj and no patch to llama.cpp.
- Seven VLA policies auto-detected from the GGUF: SmolVLA, pi0, BitVLA, Evo-1, GR00T N1.5/N1.6/N1.7.
- In-tree vision towers (SigLIP, BitSigLIP, InternViT, RADIO) on stable public ggml/llama APIs.
- ZeroMQ + protobuf
vla-server; a separatevlm-serverfor VLM chat. - BitVLA 1.58-bit custom ternary CUDA kernels.
- Per-arch HuggingFace -> GGUF converters and mmproj-merge helpers (
scripts/). - Robot eval harness for LIBERO, SimplerEnv, and ALOHA (
eval/), with device benchmark reports. - Minimal CI: pixel-shuffle unit test, converter-remap test, CPU build gate.
pyproject.tomlfor the Python tooling and a CUDADockerfileforvla-server.
- llama.cpp is fetched + pinned via CMake
FetchContent(tagb9866); bumping is a one-lineGIT_TAGchange. Removed thepatches/fetch script.