Measured 2026-09 against llama.cpp build 11201 (2145525a4), passed in with -LlamaDir. The b11223 tag that CMakeLists.txt pins was not tested.

vla.cpp now builds natively on a Snapdragon X laptop in three flavours:

• the Oryon CPU;
• the Hexagon NPU (HTP), with ops the NPU rejects running on the CPU;
• the Adreno GPU (OpenCL), with the same CPU fallback.

vla-server, vla-cli, vla-bench and the tests all build.

Eleven of the twelve published checkpoints ran on all three. BitVLA does not: its only published GGUF is int2-packed, which only CUDA builds load. OpenVLA-OFT was not attempted, because at F16 it needs about 14 GB (more than the 8 GB budget).

Every accelerator result below was checked against a CPU-backend reference on identical inputs before its latency was recorded.

• SmolVLA runs fastest on the NPU: 1.23 s per action chunk against 2.57 s on the CPU and 3.05 s on the GPU. It is within 1.5e-3 of the CPU reference.

• The NPU also wins on π0 (5.6 s vs 10.4 s CPU) and VLA-JEPA (0.85 s vs 1.6 s). Both models are sensitive to precision, and both land 4.8e-3 to 5.6e-3 from an F32 reference, above the 2.9e-3 bar the other backends are held to. Running the same F16 weights on the CPU moves them further (8.5e-3 and 1.2e-2).

• On the other eight models the CPU is fastest, once it runs F16 weights instead of the BF16 it defaults to. Oryon has no BF16 matmul, so the shipped default is 3-4x slower than it needs to be. Both accelerators still agree with the reference on these models, most to within 1e-3.

• Five ggml-hexagon kernels give wrong answers for shapes vla.cpp uses. The first three can flip the gripper channel (max |Δ| 0.7 to 1.9):

  • GELU runs as GELU_QUICK.
  • A broadcast with a gap is mis-indexed.
  • IM2COL is wrong unless the conv is a patch embedding.
  • F16 copies read back wrong on the host.
  • Op fusion costs precision.

  The CPU fallback now routes around each of them, and each is described below.

The NPU path needs the same one-off setup as llama.cpp's Hexagon backend: a newer NPU driver, test-signing, and a self-signed certificate. See NPU prerequisites.

vcpkg needs no admin rights. protobuf has to be compiled by clang, like vla.cpp. Otherwise clang's and MSVC's different mangling of __restrict leaves the servers with unresolved protobuf symbols. The overlay triplet in cmake/vcpkg-triplets builds it with Visual Studio's clang-cl. ZeroMQ is a C API, so the stock MSVC build serves. Run these from a Visual Studio developer shell:

git clone --depth 1 https://github.com/microsoft/vcpkg C:\vcpkg
C:\vcpkg\bootstrap-vcpkg.bat -disableMetrics
C:\vcpkg\vcpkg install zeromq cppzmq --triplet arm64-windows
C:\vcpkg\vcpkg install protobuf --triplet arm64-windows-clangcl --overlay-triplets=<vla.cpp>\cmake\vcpkg-triplets

The build script sets up the Visual Studio shell, the compiler flags llama.cpp's Snapdragon preset uses, vcpkg, and (for the NPU) skel signing:

.\scripts\build_windows_snapdragon.ps1 -Backend htp    -LlamaDir <llama.cpp> -HtpCert <cert.pfx>
.\scripts\build_windows_snapdragon.ps1 -Backend opencl -LlamaDir <llama.cpp>
.\scripts\build_windows_snapdragon.ps1 -Backend cpu    -LlamaDir <llama.cpp>

Each build goes into build-wos-<backend>, with every binary and DLL in build-wos-<backend>\bin. -NoServer skips vla-server, SentencePiece and their protobuf and ZeroMQ dependencies, so Octo --text needs --tokens there.

-LlamaDir points the build at an existing llama.cpp checkout through FETCHCONTENT_SOURCE_DIR_LLAMA. Without it, the b11223 pin in CMakeLists.txt applies, which was not tested here.

The HTP build also signs libggml-htp-v*.so with the certificate and copies the skels and their catalog next to the binaries. At startup the Hexagon backend points ADSP_LIBRARY_PATH at the executable's own folder, but only if the variable is unset. If it is already set, for example by a llama.cpp install, the skels it names must come from the same llama.cpp commit.

These follow llama.cpp's Windows guide. They are one-off per machine, and the steps that need admin rights are marked.

1. Update the NPU driver. Install Qualcomm's HND package from the Qualcomm Software Center (driver 30.0.220 or later; admin and a reboot). The driver shipped with the laptop (30.0.143) lacks the dspqueue_* functions, and the backend fails with failed to dlsym dspqueue_create.

2. Enable test-signing (admin and a reboot). Secure Boot must be off first, in the firmware setup. Then run bcdedit /set TESTSIGNING ON.

3. Create and trust a code-signing certificate (admin, for the trust step):

   New-SelfSignedCertificate -Subject "CN=GGML.HTP.v1" -Type CodeSigningCert -CertStoreLocation Cert:\CurrentUser\My
   # export it to <cert.pfx> and <cert.cer>, then:
   certutil -addstore Root <cert.cer>
   certutil -addstore TrustedPublisher <cert.cer>
   

4. Pass the .pfx to the build with -HtpCert. Without it the skels are unsigned, and opening a session fails with error 0x80000406.

$env:VLA_PYTHON = "$PWD\.venv\Scripts\python.exe"   # tokenizer for --text
.\build-wos-htp\bin\vla-cli.exe --ckpt models\smolvla\smolvla-libero.gguf --image assets\front.jpg --text "pick up the black bowl" --pretty
.\build-wos-htp\bin\vla-server.exe models\smolvla\smolvla-libero.gguf

vla-server on the NPU answered a two-view SmolVLA request over ZeroMQ in 1.25 s server-side (1.67 s for the first, cold request), matching vla_predict_check. The LIBERO client could not be run, because it needs PyTorch and LIBERO; the smoke test used a 20-line pyzmq client instead.

Each process prints which ops the accelerator gave to the CPU, once per op type, and a summary when it exits:

vla: HTP0+CPU rejects GET_ROWS (f32 <- bf16, [960,48,1,1], e.g. 'node_15'); running it on CPU
vla: HTP0+CPU: 7 of 35 graphs split; 7 nodes in 7 runs went to CPU, 90.2 MiB copied in, 1.2 MiB out, 90.2 MiB of weights mirrored on host

The engine drives one backend per model through gallocr, with no scheduler. That works for CUDA, Metal, SYCL and OpenVINO, which run every op the archs build. Hexagon and OpenCL do not. An NPU rejects an op when it has no kernel for it (GELU_ERF), or when the tile does not fit VTCM (softmax rows that are not a multiple of 32). Neither can be known until the graph exists.

src/backend_fallback.cpp wraps the accelerator in a backend that looks like it to the arch: same device, same buffer type, so weights and activations still live on the accelerator. Its graph_compute splits each graph:

• Runs of ops the accelerator accepts go to it unchanged.
• Runs it rejects are copied to host memory, computed on a CPU backend, and copied back.
• Weights that a CPU op reads are copied once and kept.

No arch's call sites changed, and CUDA, Metal, SYCL and OpenVINO builds do not compile the wrapper at all. The Hexagon- and OpenCL-specific graph changes (fa_kv, gelu, the default weight dtype and flash attention) sit behind the same build flags and reduce to the old calls elsewhere. SmolVLA's CPU BF16 output was checked bit-identical before and after them. The other archs were not re-checked against a pre-change build, because the unmodified tree does not build on Windows.

The wrapper is also where Hexagon's wrong kernels are refused (next section). Forcing every op through the CPU path (GGML_HEXAGON_OPFILTER=.*) reproduces a pure-CPU run bit for bit.

Each issue below was found by moving op families to the CPU with GGML_HEXAGON_OPFILTER until the action chunk matched the reference. None are vla.cpp bugs; all should go upstream.

Two further changes keep work on the NPU:

• F16 K/V for flash attention. The HTP kernel takes F16 or Q8_0 K/V only. The archs pass F32, which sent every SmolVLA attention through the CPU: 352 runs and 348 MiB of copies per chunk. vla::fa_kv() casts K/V before the permute in Hexagon builds, and is a no-op elsewhere. Evo-1 keeps F32 K/V (its attention runs on the CPU), because the cast measured 3.9e-2.
• Flash attention on by default. In the towers, the explicit path's 1024x1024 F32 score matmuls took SmolVLA's vision stage to 3.8 s. Flash attention does it in 0.44 s.

Not fixed, left on the CPU:

• GELU_ERF has no HTP kernel (DINOv2 and SigLIP-so400m towers).
• SOFT_MAX needs rows that are a multiple of 32.
• GROUP_NORM, SIN and COS are also unsupported.

Evo-1 is where this costs the most: 3,900 CPU runs and 11 GB of copies per chunk, which is why the NPU is slower than the CPU there.

The shipped GGUFs are BF16, and most archs keep them BF16 in memory. HTP has no BF16 kernel for any op, and Adreno none for MUL_MAT, so both accelerator builds default to F16 (vla::default_weight_dtype). F16 holds every normal-range BF16 value exactly, and --weight-dtype still overrides.

SmolVLA used to convert every weight to its resident dtype, so a quantize_gguf.py file would not load for it on any platform. It now keeps a GEMM weight the file stores packed, as the other archs already do, and it dequantizes any packed tensor it keeps as float.

CPU BF16 is what a stock build does today. CPU F16 is the fair CPU baseline. HTP and OpenCL use their defaults (F16 weights).

TurboVLA and Octo ship F32 GGUFs, so their "CPU BF16" column is the F32 default.

The GR00T N1.5 and N1.6 rows predate flash attention reaching their towers. Hexagon turns it on by default, so both now run it there; they were not re-measured.

† Against the CPU BF16 run. An F32 reference for π0.5 needs more memory than the 8 GB budget.

The accelerators are often closer to F32 than the CPU running the same F16 weights. ggml's CPU matmul converts activations to the weight's type (F16 here), while HTP and Adreno keep them in F32.

Bold marks the two results above the 2.9e-3 bar. Both models are precision-sensitive: their own CPU F16 run is worse. The fix, F32 weights, would need 12 GB (π0) and 8.6 GB (VLA-JEPA). Turning flash attention off only brings them to 4.9e-3 and 3.4e-3, at 1.6-2x the latency.

Peak working set of the whole process:

Every configuration stays under 8 GB. The NPU's weights live in FastRPC shared memory: they count toward the working set, but the process's private commit stays under 1.4 GB. The GPU build is the largest, because the OpenCL driver's buffers count on top of the host copy made during loading.

scripts/quantize_gguf.py --type Q8_0 packs the LM backbones. It saves memory; it does not preserve fidelity. The CPU itself moves π0 by 4.9e-2 and the GR00Ts by 7e-3 to 4e-2 on the Q8_0 files. Latency and fidelity on the Q8_0 files (fidelity against the CPU F32 run of the original):

SmolVLA's CPU Q8_0 run keeps its float weights at F16; the other rows keep the CPU default, BF16. The π0 Q8_0 result on the GPU (0.27) is far worse than the same file on the CPU, and was not investigated. Evo-1's Q8_0 file failed to load on every backend with a ggml_view assertion; it now loads on the CPU and the NPU. The Adreno GPU refuses one made by the old quantizer, because ggml-opencl ignores view offsets on quantized weights; requantize with the current scripts/quantize_gguf.py, which keeps the action expert float.

• The NPU wins where the graph stays on it.
  • SmolVLA's whole graph runs on the NPU except 7 embedding row fetches, and its vision tower is 4x faster there than on the CPU.
  • π0 and π0.5 lose only their odd-width softmaxes to the CPU.
  • Where GELU_ERF or a 1025-wide softmax sits in every tower layer (Evo-1, TurboVLA, VLA-Adapter), the copies cost more than the NPU saves.
• The CPU's shipped default is its slow path. BF16 → F16 alone makes every BF16 model 2.5-3.5x faster on Oryon, with fidelity inside or near the bar. That is a one-flag win (--weight-dtype f16) for any ARM CPU without BF16 matmul.
• The Adreno GPU is never the fastest here, but it is the most faithful accelerator: every model is within 1.4e-3 of F32.
• The NPU's vision is slow on SigLIP-so400m. π0's tower takes 3.5 s on the NPU against 1.9 s on the CPU, with nothing falling back. So400m's 4304-wide MLP is not a multiple of 32, which likely keeps it off the matrix unit; this was not profiled.

• No task-success numbers. LIBERO does not run on Windows on Arm here, so fidelity is the only check that actions are right.
• One device, one driver, one llama.cpp commit. ggml-hexagon changes fast, and several of the bugs above may already be fixed upstream.
• Few repetitions, no power or thermal control. The minimum of 3 runs is reported, and the laptop was on its default power plan.
• BitVLA and OpenVLA-OFT were not run. BitVLA's published GGUF is int2 for CUDA; a BF16 GGUF would have to be converted from the PyTorch checkpoint. OpenVLA-OFT exceeds the 8 GB budget at F16 and was not downloaded.
• Q8_0 on the NPU uses dynamically quantized activations, which is part of its error; not separated here.

• The NPU prints ggml-hex: FASTRPC_GET_DOMAINS query failed (0x6c), using static CDSP domains on every run. It works regardless.
• The Adreno driver refuses GGML_OPENCL_ADRENO_USE_LARGE_BUFFER ("not supported by driver"), so no single allocation can exceed about 1 GB. F16 weights keep every model here under that.
• Launch-VsDevShell.ps1 prints 'vswhere.exe' is not recognized; the build is not affected.

Using the NPU leaves the machine with Secure Boot off, test-signing on, and a self-signed certificate trusted as a root authority. Anyone who can read the .pfx can sign code that machine will trust, so keep it private. To undo, run the following; the NPU backend then stops loading, while the CPU and GPU are unaffected:

certutil -delstore Root <thumbprint>
certutil -delstore TrustedPublisher <thumbprint>
bcdedit /set TESTSIGNING OFF

Then turn Secure Boot back on in the firmware setup.