PSA for dual-socket CPUs owners: So check your BIOS and enable NUMA(some boards label it “Node Interleaving”). With NUMA enabled, Strata gets another long-context boost: prefill up to +19%, decode up to +20%.
- PSA for dual-socket CPUs owners: So check your BIOS and enable NUMA(some boards label it “Node Interleaving”). With NUMA enabled, Strata gets another long-context boost: prefill up to +19%, decode up to +20%.
You can furthermore control numa interleaving which is especially important depending on how your ram is socketed to your CPUs, and how your experts, etc. are distributed. have your agent try and optimise that.
NUMA as a free context boost is the kind of BIOS tip that should be pinned. Dual-socket setups leave a lot on the table if that switch is off.
Be careful, though. On many systems you want to Pina model's RAM usage to a single NUMA and if your CPU is not fast enough for the movement between CPU bridge it will slow things down not speed up.