You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ACT is probably the most common policy on SO-100/SO-101 arms, and it isn't in vla.cpp yet. I'd like to add it, but since it isn't a VLA (no language input), I wanted to check it's in scope first.
Why it could be worth it: PyTorch ACT is already fast on a Jetson (~59 ms per 100-action chunk on my Orin NX), but it still needs a CUDA PyTorch build for the board. On CPU-only boards like the one in #31, vla.cpp would be the lighter option.
What I have in mind, following "Adding an architecture" in CONTRIBUTING:
scripts/convert_act_to_gguf.py for LeRobot ACT checkpoints: ResNet18 with its batch norms folded into the convs, the normalizer stats from the processor files, and no VAE encoder (it's training-only, so the latent is zero at inference). That leaves ~34M of the 52M parameters.
src/models/act.cpp: ResNet18 on the conv path Octo uses, the 4-layer encoder, and the 1-layer decoder with 100 learned queries, similar to TurboVLA's ACT decoder but post-norm. No flow-matching steps.
The plan is absolutely fine to me. Thanks for raising the the topic. Please open the PR when your implementation is ready. We will review and merge it.
Feel free to add act option for --arch.
One thing I would like to add on top of your development suggestion: when checking against PyTorch with vla_predict_check, could you also check with the weight of lerobot lerobot/act_aloha_sim_transfer_cube_human along side with your SO-101 checkpoint?
Thanks! Will do. I'll check act_aloha_sim_transfer_cube_human against PyTorch too, alongside my SO-101 checkpoint, and add act to --arch in the client. I'll open the PR when it's ready.
Activity
khanhnd61-vr commentedon Oct 5, 2026
Hi @ravediamond ,
The plan is absolutely fine to me. Thanks for raising the the topic. Please open the PR when your implementation is ready. We will review and merge it.
Feel free to add
actoption for--arch.One thing I would like to add on top of your development suggestion: when checking against PyTorch with
vla_predict_check, could you also check with the weight of lerobotlerobot/act_aloha_sim_transfer_cube_humanalong side with your SO-101 checkpoint?Thanks for your contribution to the project!
ravediamond commentedon Oct 6, 2026
Thanks! Will do. I'll check
act_aloha_sim_transfer_cube_humanagainst PyTorch too, alongside my SO-101 checkpoint, and addactto--archin the client. I'll open the PR when it's ready.ravediamond commentedon Oct 19, 2026
Done in #38, closing.