Repository navigation
Conversation
Pre-release look at SE.Redis v4. The existing Execute-based code compiles against it unchanged (0 warnings on all TFMs), so old and new styles can coexist while modules are migrated one group at a time.
A spike to see what migrating a module to SE.Redis v4 looks like, using the smallest one. The shape follows docs/Extending.md and SE.Redis's own groups: - RespCountMinSketch: a readonly struct over a RespContext, reached via one generic extension property (db.CountMinSketch, batch.CountMinSketch, db.Context.AppendKeyPrefix(...).CountMinSketch). - CountMinSketchCommands: the six commands as ValueTask extension methods. Four are a single interpolated string; INCRBY (multi) and MERGE use Compose for the variable tail, with WEIGHTS as a [Resp] generated fragment. Two custom IRespHandler<T>s: CMS.INFO (pairs by label, so RESP2 array and RESP3 map read identically) and single-item INCRBY. - Every command declares its retry category via WithRetryCategory, which is also what makes QUERY and INFO eligible for client-side caching. The original surface is kept and served from the group, as SE.Redis does for IDatabase: CmsCommands (sync) sends through db.Context.Blocking(); CmsCommandsAsync (Task) bridges with db.AsTask(...). Both are the public v4 APIs for exactly this. CmsCommandBuilder is [Obsolete]; the builder surface has no equivalent in v4 and nothing uses it any more. Behaviour changes in the legacy surface: - Merge: destination and sources were typed as RedisValue, so no key ever reached cluster routing. The group types them as RedisKey; the proxy converts, and now validates numKeys against the source count. - InitByDim/InitByProb/Merge on the new surface return ValueTask rather than a bool that could only ever be true. Tests: CountMinSketchGroupTests (new surface; RESP2+RESP3, standalone and cluster, prefixed context, batch) and CountMinSketchCachingTests (RESP3; proves the read-only opt-in is cached, the cache key covers value args, and both local and remote invalidation reach a module key). The existing CmsTests exercise the proxied legacy surface unchanged.
Blocking() allocates a new context each time; a database's context is immutable and does not change, so hold the blocking group in a field, as RedisDatabase does internally. The async side's group is a free struct but is cached too, for symmetry.
mgravell
added a commit
to StackExchange/StackExchange.Redis
that referenced
this pull request
Oct 9, 2026
…s, and read-only means cacheable Gaps found reviewing the NRedisStack port (redis/NRedisStack#577), which followed the page faithfully and still ended up with: - an extension-property accessor only, invisible below C# 14 (the .NET 8 SDK's default, .NET Framework): the page now says to ship the property and a method in a separate Downlevel namespace, as this client does, and why apart (CS9339); - naked long[] replies: a new "Shaping the signature" section states the conventions - ValueTask, flags and token last, RedisKey for keys, ReadOnlySpan<T> in, ReadOnlyLease<T> out, empty input answered without a round trip - with a sample compiled against the library; - no warning that CommandRetryReadOnly opts a command into client-side caching, and that a read whose answer drifts without a write should add NoClientCache. Groups.md's lease bullet presented leases as an occasional extra form; it now describes them as the shape of every variable-length reply.
- QueryAsync and the multi-item IncrByAsync return ReadOnlyLease<long> rather than long[], as the SE.Redis groups do for multi-value replies (Lists.PositionsAsync, Hashes.GetTimeToLiveAsync). The inbuilt handler covers it; the legacy proxies copy out with lease.ToArray(). - An empty item span short-circuits to ReadOnlyLease<long>.Empty with no round trip, matching MGET. MergeAsync with no sources still throws, now ArgumentException: it is malformed, not out of range. - NRedisStack.Downlevel.RedisStackGroups: the accessor as a method, for consumers below C# 14 who cannot see extension properties. Separate namespace because both in scope on C# 14 is CS9339. Mirrors StackExchange.Redis.Downlevel.RespGroups. - cancellationToken docs use the standard sentence (what happens to a command already written). - MergeAsync remarks say why it stays write-accumulating.
… not arrive CI failed AWriteFromAnotherClientIsAnnouncedForAModuleKey on the Redis 8.10 jobs only (net9.0 and net10.0, not net8.0; nothing on 7.2-8.8), and it does not reproduce locally against the same 8.10.0 image, filtered or as the full suite. On failure the test now reports whether the polled reads were cache hits or reached the server, the remote INCRBY's returned count, CLIENT TRACKINGINFO on the cached connection, and the server's tracking_* INFO fields, so the next run can tell a dropped push from a stale server answer.
…e test, document the caveat The CI failure was the Windows 6.2.6 job, not the 8.10 jobs (those failed on the pre-existing Search cluster flake). Redis Stack 6.2.6's RedisBloom does not signal CMS.INCRBY as a key modification, so a tracking client is never told and a client-side cached CMS.QUERY/INFO stays stale until TimeToLive. 7.2 and later announce correctly, as the Windows 7.2/7.4 and all Linux jobs show. The remote-invalidation test is gated to >= 7.2.0; the group's docs carry the caveat, since it is user-visible for anyone enabling ClientCache against a 6.2 server.
…ith [Resp]
The six command names become [Resp("CMS.X")] partial RespCommand
properties, generated by the package's analyzer alongside the WEIGHTS
fragment: one declaration style for commands and tokens, and the names
are checked at build time. Unknown names (all of these) are framed once at
startup; a known name would stay deferred so the command map could still
rename it. No behaviour change.
…k load PopulateTimeoutIndex writes 100k documents fire-and-forget, then pinged the database once to order itself after them. A ping is ordered after the writes on its own connection only; on a cluster the writes are spread over every primary, so the other nodes could still be draining while FT.INFO was read. Under StackExchange.Redis 3.x fire-and-forget writes went to the sockets inline, which hid the race; v4 queues them, and the 8.10 cluster jobs started failing with num_docs a few thousand short (94081, 96017, 98730 of 100000; a different on-timeout test each run). Ping each connected primary instead, and give AssertIndexSize a 50ms pause between its ten FT.INFO re-reads - indexing is asynchronous on the server, so an immediate re-read sees nothing new.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, for review of the shape rather than for merge: this is the first module (the smallest, Count-Min Sketch) migrated to the StackExchange.Redis v4 command-group surface. Built against
4.0.86-alpha.What changes for callers
Nothing existing breaks:
db.CMS(),ICmsCommands,ICmsCommandsAsync,Pipeline.CmsandTransaction.Cmskeep their signatures and behaviour. The new surface sits beside them. This section is what a caller sees when they move.Reaching the commands
readonly structholding only the context. Getting it allocates nothing; take it at the call site rather than storing it.IDatabase,IBatch,ITransactionandRespDatabaseContext, so a prefixed, blocking or cache-adjusted context reaches the same commands.db.CountMinSketchis a C# 14 extension property. On an older language version (the .NET 8 SDK defaults to C# 12), addusing NRedisStack.Downlevel;and writedb.CountMinSketch()instead. Do not import that namespace on C# 14, where both in scope is CS9339.Method shape
Every command is asynchronous and follows one pattern:
ValueTask, notTask. A result already known (a client-side cache hit) completes without allocating. Await it once; do not store it to await twice.db.AsTask(...)gives aTaskthat carries the database'sAsyncStateand marks faults observed; prefer it over.AsTask()if you need aTask.db.Context.Blocking().CountMinSketch.InfoAsync(key).GetAwaiter().GetResult(). Every send through that context completes on the calling thread; no thread-pool hop. A batch or transaction cannot block (nothing is sent until it executes).CommandFlagson every call. Fire-and-forget, replica preference,NoClientCache, and retry categories all work as on the core API. Each command already declares its own retry category; a category you pass wins.CancellationTokenon every call. It cancels the wait: a command not yet written is never sent; one already written still runs on the server and its reply is discarded.Return types
db.CMS())db.CountMinSketch)InitByDim,InitByProb,Mergebool(alwaystrue; errors threw)ValueTask- completes or throwsIncrBy(one item)longValueTask<long>IncrBy(many items),Querylong[]ValueTask<ReadOnlyLease<long>>- dispose itInfoCmsInformationValueTask<CmsInformation>A
ReadOnlyLease<T>is a pooled buffer, valid until disposed. From a client-side cache hit it is a read-only view over the shared entry; disposing it does not affect other readers.Arguments
RedisKey.MergeAsync(RedisKey destination, ReadOnlySpan<RedisKey> sources, ReadOnlySpan<long> weights = default, ...). The legacyMergetook these asRedisValue, so cluster routing and key prefixing never saw them; that is fixed in both surfaces (the legacy proxy converts). The legacynumKeysparameter is gone from the new surface and validated against the source count on the old one.QueryAsync(key, ReadOnlySpan<RedisValue>)andIncrByAsync(key, ReadOnlySpan<(RedisValue Item, long Increment)>). Arrays and collection expressions convert implicitly:QueryAsync(key, ["foo", "bar"]),IncrByAsync(key, [("foo", 5), ("bar", 15)]). The legacyTuple<RedisValue, long>[]form stays ondb.CMS().QueryAsync(key, [])andIncrByAsync(key, [])return an empty lease with no round trip (the key need not exist).MergeAsyncwith no sources, or with weights that do not match the sources one for one, throwsArgumentExceptionbefore sending.Client-side caching
QUERYandINFOare declared read-only, so withConfigurationOptions.ClientCacheset (RESP3) they are served from the local cache and invalidated by anyINCRBY,MERGEorDELon the key, including from other clients. Opt a call out withCommandFlags.NoClientCache, or a context withWithoutCache()/WithMaxCacheAge(...). Seedocs/ClientSideCaching.mdin StackExchange.Redis.Deprecated
CmsCommandBuilderand itsSerializedCommandoutput are[Obsolete]. They still work, but the new surface has no builder layer, and every module will follow.What the migration looks like
CmsCommandBuilder→SerializedCommand(object[])→db.Execute(...)cms.Context.SendAsync<T>($"{Query}{key}{items}", flags.WithRetryCategory(...))RedisResult→.ToLong()/.ToCmsInfo()long/long[];IRespHandler<CmsInformation>forINFOMergesent keys as values)RedisKeyholes: routed, prefixed, cache-trackeddb.CMS()→ class with sync + async twinsdb.CountMinSketch→readonly struct,ValueTaskextension methods onlydb.Context.Blocking(),Taskviadb.AsTask(...)Files worth reading in order:
src/NRedisStack/CountMinSketch/CountMinSketch.cs- the struct and the one generic accessor.src/NRedisStack/CountMinSketch/CountMinSketch.Methods.cs- the six commands. Four are one interpolated string;INCRBY(multi) andMERGEuseCompose;WEIGHTSis a[Resp]generated fragment (the generator ships in the SE.Redis package and runs here unchanged).CmsCommands.cs/CmsCommandsAsync.cs- the legacy proxies. One line each; the group is resolved once in the constructor.tests/.../CountMinSketchGroupTests.cs- new surface, RESP2+RESP3, standalone+cluster, prefixed context, batch.tests/.../CountMinSketchCachingTests.cs- client-side caching over a module command: read-only opt-in is cached, cache key covers value args, local and remote invalidation both reach a module key (RedisBloom does signal key modification).Decisions taken
CountMinSketch, notCms).ValueTask-only, no sync twins (results may complete synchronously from the client-side cache). Legacy API keepsTask<T>/ sync shapes untouched.InitByDim/InitByProb/Mergeon the new surface returnValueTask, not aboolthat could only betrue.CmsCommandBuilderis[Obsolete]. The builder surface has no v4 equivalent; expect this across all modules.db.Context.Multiplexer?.AddLibraryNameSuffix(...), but it is a suffix with no override or opt-out, where today's logic replaces the name and offers both; deciding that is separate.Behaviour changes in the legacy surface
Merge: destination/sources wereRedisValue, so cluster routing never saw a key. Now converted toRedisKey;numKeysis validated against the source count rather than passed through.Found along the way (fed back to SE.Redis; all address
Blocking().ValueTask→Taskbridge that carriesAsyncStateand marks faults observed →db.AsTask(...). - NeededRespDatabaseContextto be an `IRespKeyspaceTar prefixed context → done.RespHandlers.Inbuilt<T>. No non-nullable numericAppendFormattedoverloads (deliberate; only visible inComposeloops).Assert.Equaloverloads bind to collection expressions, soAssert.Equal([1,2], await ...)is CS4007. Hoist intoa typed local.
Test run
dotnet test -f net8.0withREDIS_VERSION=8.10.0againuster: CMS legacy, group, caching, pipeline, transactionand WithRetry tests all pass (109 passed, 6 skipped - the skips are pre-existingExampleTests/TestModulesPipeline). Library builds with 0 warnings on all five TFMs.Commits
cb8cb46Move to StackExchange.Redis 4.0.86-alpha3bae19cCountMinSketch: first module on the v4 command-group surfaceaf86fedCountMinSketch proxies: resolve the group oncNot in scope
Any other module. Next candidate by the same recipe: TopK