Stas Bekman

@StasBekman

Toolmaker. Software creator, optimizer and harmonizer. Makes ML systems work and fly @ Snowflake.

Nanaimo, BC, Canada

Stas Bekman
@StasBekman
14h
To get much faster NCCL comms at small-to-medium payloads, start using symmetric memory, which has been added recently to NCCL and PyTorch. If you struggle to overlap comms and compute this could be your salvation in a few lines of code! Details github.com/stas00/ml-engi…
8
Stas Bekman
@StasBekman
Oct 2
mamf-finder has been updated to support the following formats: --dtype bfloat16, float16, float32, float8_e4m3fn, float8_e4m3fnuz, mxfp8, mxfp4, nvfp4 So now you can benchmark your real 100% tflops for almost any workload github.com/stas00/ml-engi…
1
Stas Bekman
@StasBekman
Oct 2
I measured nvfp4 to be about 9% more efficient than mxfp4 on B200 using MAMF metrics Plus you get a higher accuracy with nvfp4 So Blackwell and up, choose nvfp4 over mxfp4 fp4 formats explained: github.com/stas00/ml-engi…
7
Stas Bekman
@StasBekman
Oct 2
I present to you a new AI skill that evaluates any cluster's compute, network and storage - all based on tools in the Machine Learning engineering book. github.com/stas00/ml-engi… You just tell your agent to use this skill, how it can access your cluster and it does all the work
6
Stas Bekman
@StasBekman
Oct 2
I present to you a new performance metric to measure a sustainable matmul FLOPS, so we have: - MSMF: Maximum Sustainable Matmul FLOPS — the power-saturated, sustained rate - MAMF: Maximum Achievable Matmul FLOPS — the boost-clock burst ceiling github.com/stas00/ml-engi… I have
7