Repository navigation
Conversation
As of Go 1.27 the compiler merges the byte loads into one MOVQ and compiles the tail switch to a jump table, so the asm's remaining edge was the tail: the byte at a time fallthrough switch left 12 bounds checks and up to 15 single byte loads on a 15 byte input. The tail is now a flat switch with one case per length, so every index is proven in range and each case is two or three word loads with no checks left. One sum128[T string | []byte] replaces the four duplicated copies. The compiler stencils one per type, so strings still hash with no copy and no allocation. The streaming digests reuse the same block mixer and finalizer rather than carrying their own tail switch. Measured interleaved on Go 1.27.1 amd64: 8 to 26 percent faster than the asm for 128 bit inputs under 64 bytes, equal at 32 bytes, 10 to 20 percent slower from 128 bytes up. The 32 bit sums gain 6 to 11 percent at 32 to 128 bytes from the two block unroll. Also: go.mod to 1.25 and the digests implement hash.Cloner; TestQuickSeedSum128 discarded the C++ result and compared nothing; TestBoundaries covers lengths 0 through 100; GitHub Actions replaces the dead Travis badge.
Wall clock numbers on the laptop this was developed on moved 40 percent with thermal throttling, so the tables are now user space cycles per call from perf stat at a fixed iteration count. Per 16 byte block the old assembly is 8.2 cycles and the Go loop about 9.2; under 16 bytes the Go code is 12 to 19 percent faster than the assembly was. Also records why compiler hints do not help: the loop is bound by the multiply port and the h1 to h2 dependency chain, not by instruction count. Hoisting the constants into registers or dropping the reslice guard cut instructions and cost cycles, and the compiler's own folding of 5*c1 + c2 is what keeps the chain short.
A reslice that can leave zero bytes makes the compiler guard the pointer advance with a NEG/SAR/AND mask so a slice never points past its allocation. Looping while len(data) > 16 and taking the last block after the loop makes that case impossible, and the guard is gone from the hot loop. Measured with perf in cycles per 16 byte block: 9.2 for the two block unroll, 8.5 now, 8.2 for the old assembly. The unroll stopped paying once the guard was gone; the loop is bound by the multiply port and the h1 to h2 chain, not by instruction count. Same change for the 32 bit sum and both streaming block mixers.
A switch on the tail length compiles to a jump table, and on varying key lengths that indirect branch mispredicts nearly every time: perf counted 0.94 mispredicts per hash against the assembly's 0.30 on a mixed length stream, and the Go code was 34 percent slower there. Once an input has had a full block its last 16 bytes are always in bounds, so the tail is read out of them with two overlapping loads and shifts chosen by the length, before the block loop so only two words stay live across it. Inputs shorter than a block keep the switch. Random lengths in [0, 64) now cost 89 cycles a hash against the assembly's 110 and the previous code's 98, and every fixed length is equal or better. Sum64 and friends call sum128 directly: going through the Sum128 wrapper put them at inline cost 86 against a budget of 80, one extra call per hash, and entering the loop through that call was 6 percent slower at 8 KiB.
The block loop indexes data[i:i+16] instead of reslicing, with the bound hoisted by hand because Go 1.26 recomputes it every iteration. The compiler proves the window from the bound and folds the index into the loads, so per block that is one counter add instead of pointer, length and capacity updates: 23 instructions and 5 LEAs against 26 and 7. The two multipliers are read once from variables so they stay in registers rather than being rematerialized as ten byte immediates. perf, cycles per 16 byte block at 8 KiB on Go 1.27: 8.6 before, 8.1 now, 8.2 for the old assembly. Every other size is equal or better, random lengths included. On Go 1.26 the same instructions with one register named R13 instead of CX bind one more micro op per block to port 1 and run at 9.3; Go 1.25 cannot prove the window and spills.
With the index loop, reading the multipliers from variables so they stay in registers measured 8.1 cycles per block on Go 1.27 and 9.3 on Go 1.26 against 8.5 on both as constants: the two extra live registers push the 1.26 allocator onto R13, whose REX prefix regroups the loop in the micro op cache and binds one more micro op per block to the multiply port. A stable 8.5 beats a coin flip between 8.1 and 9.3.
The block loop indexes data[i:i+16]. Go 1.25 cannot prove that window in bounds and emits bounds checks, the reslice guard and five spills into the loop, 32 to 50 percent behind the old assembly on 128 bit inputs of 16 bytes and more, where 1.26 and 1.27 are within 7 percent and ahead below 64 bytes. 1.25 has been out of support since 1.27.
twmb
commented
Sep 7, 2026
| runs-on: ubuntu-latest | ||
| strategy: | ||
| matrix: | ||
| go: ['1.26', oldstable, stable] |
Owner
Author
There was a problem hiding this comment.
stable, oldstable. Why 1.26 called out specifically?
Owner
Author
There was a problem hiding this comment.
It is the go.mod minimum. Today oldstable is 1.26 so the entry is redundant, but when 1.28 ships in February oldstable becomes 1.27 and 1.26 would silently stop being tested while still being what the module declares. The explicit entry keeps the declared minimum under test until go.mod moves. If you would rather run only stable and oldstable and bump go.mod when 1.26 falls out, I will drop it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Evaluates whether the amd64 assembly is still needed on Go 1.27 and concludes it is not: removes it, and rewrites the one shot sums as a single generic implementation over
string | []byte.Why the asm lost its edge. The compiler now merges the eight byte loads into one
MOVQand compiles the tail switch to a jump table. What the asm still beat was the tail: the byte at a time fallthrough switch left 12 bounds checks and up to 15 single byte loads on a 15 byte input. The tail is now a flat switch with one case per length, so the compiler proves every index and each case is two or three word sized loads. The generatedsum128has zero bounds check panics (verified by grepping the assembly), no allocations, and every helper inlines.Measured in cycles with perf (Go 1.27.1, Comet Lake i7-10710U,
cycles:uat a fixed iteration count, min of 5, so frequency and throttling drop out).asmis HEAD's assembly build,old Gois HEAD's pure Go path,new Gois this PR:Per 16 byte block: asm 8.2 cycles, new Go 8.5 on both go1.26.0 and go1.27.1. The Go code is faster than the asm was for everything under 64 bytes (14 to 18% on 3 to 13 byte inputs), on random lengths in [0, 64) (18% for 128 bit, 12% for 32 bit), and for every 32 bit sum (14 to 18% at 32 to 64 bytes); it is 1 to 7% slower on fixed inputs of 64 bytes and more.
The loop, with perf counters. Port counters show the loop is bound by issue width and port 1 (the four 64 bit multiplies, plus LEAs which share it with port 5), with the h1 to h2 chain at 5 cycles thanks to the compiler folding
5*c1 + c2. Of the loop shapes measured in the real package, the one kept is an index loop overdata[i:i+16]with the bound hoisted by hand (Go 1.26 does not hoist it itself): 23 instructions per block against 26 for a reslicing loop, zero bounds checks, and 8.5 cycles per block on both supported compilers. Reading the two multipliers from variables so they stay in registers instead of being rematerialized as 10 byte immediates measured 8.1 on go1.27.1 and 9.3 on go1.26.0: the two extra live registers push the 1.26 allocator to name one register R13 instead of CX, the REX prefix regroups the loop in the µop cache, and port binding puts one more µop per block on port 1. A stable 8.5 was chosen over that coin flip. Unrolling, hand folding the adds, an out of line loop, and unsafe pointer walks (raw and checkptr clean) were all measured worse. go1.25.0 spills on the index loop (10.7), which is why the module now requires 1.26. GOAMD64=v3 changes nothing.The tail, measured on varying lengths. Fixed length benchmarks predict every tail branch perfectly and hid a real problem: the flat switch is a jump table, and on a mixed length stream it took 0.94 mispredicts per hash to the asm's 0.30, 34% slower. Once an input has had a full block its last 16 bytes are always in bounds, so the tail is now read out of them with two overlapping loads and shifts selected by the length, computed before the block loop so only two words stay live across it (keeping the whole slice live cost 13 instructions per call in register pressure). Inputs shorter than a block keep the switch. Random lengths went from 98 to 89 cycles per hash (asm: 110) with every fixed length equal or better.
BenchmarkRandomLengths32/128keep this case measured. Sum64 and friends now callsum128directly, which puts them under the inliner's budget; going throughSum128cost a call per hash.Also in this PR
go 1.26: the streaming digests implementhash.Cloner(1.25), and 1.25's prove pass cannot verify the index loop's window and spills, 32 to 50% behind the asm at 16 bytes and up.TestQuickSeedSum128discarded the C++ reference result and so compared nothing. Fixed. Strayprintdebugging removed.TestBoundariescovers lengths 0 through 100 so the block loop, the trailing block, and every tail are exercised.