Valkey connector by bluayer · Pull Request #1743 · LMCache/LMCache

bluayer · 2025-10-05T08:19:43Z

What this PR does / why we need it:
This PR implements Valkey connector support for LMCache, addressing issue #1621.

Key implementations:

ValkeyConnectorAdapter: Handles valkey:// URL parsing and connection management (standalone and cluster mode)
ValkeyConnector: Using valkey-glide client with pipeline
- Standalone : For better network latency, use mget and pipeline
- Cluster : For better network latency, use mget, pipeline and hashslot
- The above features should also be implemented in the Redis Connector in the future.

Performance test example :
(Random 10keys, cluster mode with 3 shards.)

ValkeyClusterConnector PUT: 1.13ms | GET: 1.15ms | Total: 2.28ms
RedisClusterConnector: PUT: 6.97ms | GET: 2.00ms | Total: 8.96ms

Special notes for your reviewers:
Currently, it's only implemented for v1.

If applicable:

this PR contains user facing changes - docs added
this PR contains unit tests

gemini-code-assist · 2025-10-05T08:19:58Z

Summary of Changes

Hello @bluayer, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new Valkey connector to LMCache, significantly expanding its storage backend options. The integration provides robust support for both standalone and clustered Valkey environments, leveraging the valkey-glide client for optimized data retrieval and storage. This enhancement is designed to improve performance and flexibility for users seeking a high-performance, Redis-compatible caching solution.

Highlights

Valkey Connector Integration: Added comprehensive support for Valkey as a storage backend in LMCache, addressing issue Support Valkey Backend #1621.
Flexible Connection Modes: Introduced ValkeyConnectorAdapter to intelligently parse valkey:// URLs and manage connections for both standalone and cluster Valkey deployments.
Optimized Data Operations: Implemented ValkeyConnector and ValkeyClusterConnector utilizing the valkey-glide client for efficient mget and batched PUT operations, including hashslot management for cluster mode.
Performance Improvements: Initial performance tests indicate significant speedups for ValkeyCluster (e.g., 2.28ms total for PUT/GET) compared to RedisCluster (e.g., 8.96ms total).
Unit Test Coverage: New unit tests and a MockValkey fixture have been added to ensure the reliability and correctness of the Valkey connectors.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a Valkey connector for LMCache, supporting both standalone and cluster modes. The implementation is well-structured and includes performance optimizations like pipelining. My review focuses on improving code maintainability by reducing duplication, increasing the robustness of URL parsing, and enhancing the completeness of the new tests. Overall, this is a solid contribution, and the suggested changes aim to make the code even more robust and easier to maintain.

YaoJiayi · 2025-10-07T17:49:54Z

@maobaolong Would you like to take a look at this PR?

bluayer · 2025-10-13T13:21:08Z

Hi @maobaolong, This PR is quite large, and I know you're busy, but would you review it please?

maobaolong · 2025-10-15T02:17:03Z

@YaoJiayi Thanks for invite me to review this PR, and thanks also for your efforts to pick up the valkey connector. @bluayer

Will take a look this week. I'm sorry that this week we are busy for some urgency work.

maobaolong · 2025-10-15T02:18:17Z

@bluayer First of all, may i ask whether you use valkey enable RDMA? Or just use tcp mode?

bluayer · 2025-10-15T02:21:49Z

@maobaolong Thank you for your response.

Only use tcp in this PR. Because the main client used and developed by valkey org, valkey-glide, does not yet support RDMA. (Ref : valkey-io/valkey-glide#1963)

I plan to work on adding RDMA support after merging this connector.

maobaolong

@bluayer Left some comments, about the turn url parse to extra_config suggestion, let's wait the feedback from @sammshen .

sammshen · 2025-10-16T16:59:39Z

@bluayer, do you have any benchmarks on the read and write throughput (in terms of GB / s) of Valkey? (any benchmarking utility is fine)? :)

Also, would you mind updating the docs? https://docs.lmcache.ai/kv_cache/storage_backends/valkey.html

sammshen

LGTM on first pass! Please add documentation

bluayer · 2025-10-19T11:29:52Z

Hi @maobaolong, I've applied all of your feedback. Please check it out :) Also I added the document for valkey connector.
And due to dependencies on Valkey-GLIDE, the unit tests are failing. How can I resolve this? Please share your ideas.

And @sammshen, I don't have my own benchmark results but you can estimate throughput by this official blog post(https://valkey.io/blog/unlock-one-million-rps/). Valkey 7.2 is completely identical to Redis 7.2, so this high performance has been supported since Valkey 8 released last year.

sammshen · 2025-10-19T21:32:31Z

you can mock the behavior in the unit tests

maobaolong

@bluayer Left a comment within get method.

maobaolong

@bluayer Left a minor comment inline. Otherwise, LGMT.

sammshen · 2025-10-21T04:45:03Z

please merge from #1833 (comment) to fix integration tests

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

- Using same url prefix(valkey://) supporting standalone and cluster mode - Use query param for checking mode. - Standalone mode supports selecting database and cluster mode doesn't support it (but after Valkey 9, it can support database mode) Signed-off-by: Jungwoo Song <bluayer@gmail.com>

- Implement valkey connector based on redis connector but using other library, Valkey-GLIDE managed by Valkey organization. - Use mget and pipeline put at standalone and cluster mode - Use Hash slot for reducing network latency at cluster mode. - When tested with 10 keys, Valkey connector is 5 ~ 20% faster at standalone mode and up to 70%(avg 60%+) faster at cluster mode. Signed-off-by: Jungwoo Song <bluayer@gmail.com>

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

- Add try-except under assert statement for checking error - Add memory_obj.ref_count_down() for preventing memory leak Signed-off-by: Jungwoo Song <bluayer@gmail.com>

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

bluayer · 2025-10-21T05:43:17Z

@sammshen I pull dev branch and rebase this branch. Thank you.

sammshen · 2025-10-21T22:02:47Z

thanks @bluayer for the PR and @maobaolong for the great feedback!

meet-bhagdev · 2025-10-22T17:39:28Z

this is awesome! thank you @bluayer

* feat: Add dependency for valkey-glide Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add Valkey adapter - Using same url prefix(valkey://) supporting standalone and cluster mode - Use query param for checking mode. - Standalone mode supports selecting database and cluster mode doesn't support it (but after Valkey 9, it can support database mode) Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Valkey Connector - Implement valkey connector based on redis connector but using other library, Valkey-GLIDE managed by Valkey organization. - Use mget and pipeline put at standalone and cluster mode - Use Hash slot for reducing network latency at cluster mode. - When tested with 10 keys, Valkey connector is 5 ~ 20% faster at standalone mode and up to 70%(avg 60%+) faster at cluster mode. Signed-off-by: Jungwoo Song <bluayer@gmail.com> * test: add test for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: add specific version for supporting pipelinig Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: linting Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: add checking memory equality at test Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Use extra config for simple paramters Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: remove valkey-glide depedency from common.txt Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add try-except and prevent memory leak - Add try-except under assert statement for checking error - Add memory_obj.ref_count_down() for preventing memory leak Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: double check for config and extra_config Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: lint Signed-off-by: Jungwoo Song <bluayer@gmail.com> * docs: Add docs for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: Move import statement to the head of file and exception handling Signed-off-by: Jungwoo Song <bluayer@gmail.com> --------- Signed-off-by: Jungwoo Song <bluayer@gmail.com> Signed-off-by: TaoZQY <zhangtaolqy@mail.ustc.edu.cn>

* feat: Add dependency for valkey-glide Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add Valkey adapter - Using same url prefix(valkey://) supporting standalone and cluster mode - Use query param for checking mode. - Standalone mode supports selecting database and cluster mode doesn't support it (but after Valkey 9, it can support database mode) Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Valkey Connector - Implement valkey connector based on redis connector but using other library, Valkey-GLIDE managed by Valkey organization. - Use mget and pipeline put at standalone and cluster mode - Use Hash slot for reducing network latency at cluster mode. - When tested with 10 keys, Valkey connector is 5 ~ 20% faster at standalone mode and up to 70%(avg 60%+) faster at cluster mode. Signed-off-by: Jungwoo Song <bluayer@gmail.com> * test: add test for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: add specific version for supporting pipelinig Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: linting Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: add checking memory equality at test Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Use extra config for simple paramters Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: remove valkey-glide depedency from common.txt Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add try-except and prevent memory leak - Add try-except under assert statement for checking error - Add memory_obj.ref_count_down() for preventing memory leak Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: double check for config and extra_config Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: lint Signed-off-by: Jungwoo Song <bluayer@gmail.com> * docs: Add docs for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: Move import statement to the head of file and exception handling Signed-off-by: Jungwoo Song <bluayer@gmail.com> --------- Signed-off-by: Jungwoo Song <bluayer@gmail.com>

* feat: Add dependency for valkey-glide Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add Valkey adapter - Using same url prefix(valkey://) supporting standalone and cluster mode - Use query param for checking mode. - Standalone mode supports selecting database and cluster mode doesn't support it (but after Valkey 9, it can support database mode) Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Valkey Connector - Implement valkey connector based on redis connector but using other library, Valkey-GLIDE managed by Valkey organization. - Use mget and pipeline put at standalone and cluster mode - Use Hash slot for reducing network latency at cluster mode. - When tested with 10 keys, Valkey connector is 5 ~ 20% faster at standalone mode and up to 70%(avg 60%+) faster at cluster mode. Signed-off-by: Jungwoo Song <bluayer@gmail.com> * test: add test for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: add specific version for supporting pipelinig Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: linting Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: add checking memory equality at test Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Use extra config for simple paramters Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: remove valkey-glide depedency from common.txt Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add try-except and prevent memory leak - Add try-except under assert statement for checking error - Add memory_obj.ref_count_down() for preventing memory leak Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: double check for config and extra_config Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: lint Signed-off-by: Jungwoo Song <bluayer@gmail.com> * docs: Add docs for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: Move import statement to the head of file and exception handling Signed-off-by: Jungwoo Song <bluayer@gmail.com> --------- Signed-off-by: Jungwoo Song <bluayer@gmail.com> Signed-off-by: Lee Yao Yang <leey0227@e.ntu.edu.sg>

* feat: Add dependency for valkey-glide Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add Valkey adapter - Using same url prefix(valkey://) supporting standalone and cluster mode - Use query param for checking mode. - Standalone mode supports selecting database and cluster mode doesn't support it (but after Valkey 9, it can support database mode) Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Valkey Connector - Implement valkey connector based on redis connector but using other library, Valkey-GLIDE managed by Valkey organization. - Use mget and pipeline put at standalone and cluster mode - Use Hash slot for reducing network latency at cluster mode. - When tested with 10 keys, Valkey connector is 5 ~ 20% faster at standalone mode and up to 70%(avg 60%+) faster at cluster mode. Signed-off-by: Jungwoo Song <bluayer@gmail.com> * test: add test for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: add specific version for supporting pipelinig Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: linting Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: add checking memory equality at test Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Use extra config for simple paramters Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: remove valkey-glide depedency from common.txt Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: Add try-except and prevent memory leak - Add try-except under assert statement for checking error - Add memory_obj.ref_count_down() for preventing memory leak Signed-off-by: Jungwoo Song <bluayer@gmail.com> * feat: double check for config and extra_config Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: lint Signed-off-by: Jungwoo Song <bluayer@gmail.com> * docs: Add docs for valkey connector Signed-off-by: Jungwoo Song <bluayer@gmail.com> * fix: Move import statement to the head of file and exception handling Signed-off-by: Jungwoo Song <bluayer@gmail.com> --------- Signed-off-by: Jungwoo Song <bluayer@gmail.com>

gemini-code-assist Bot reviewed Oct 5, 2025

View reviewed changes

bluayer force-pushed the feature/valkey-connector branch from 5b6a6a3 to 99ee726 Compare October 6, 2025 12:32

YaoJiayi requested a review from maobaolong October 7, 2025 17:50

maobaolong requested changes Oct 15, 2025

View reviewed changes

Comment thread requirements/common.txt Outdated

Comment thread lmcache/v1/storage_backend/connector/valkey_adapter.py Outdated

Comment thread lmcache/v1/storage_backend/connector/valkey_connector.py

Comment thread lmcache/v1/storage_backend/connector/valkey_connector.py Outdated

sammshen approved these changes Oct 16, 2025

View reviewed changes

maobaolong reviewed Oct 20, 2025

View reviewed changes

Comment thread lmcache/v1/storage_backend/connector/valkey_connector.py

maobaolong approved these changes Oct 20, 2025

View reviewed changes

Comment thread lmcache/v1/storage_backend/connector/valkey_connector.py Outdated

Comment thread lmcache/v1/storage_backend/connector/valkey_connector.py Outdated

bluayer added 13 commits October 21, 2025 14:12

feat: Add dependency for valkey-glide

9ecb19e

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

test: add test for valkey connector

ff45261

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

feat: add specific version for supporting pipelinig

4df2ba2

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

fix: linting

9716fdd

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

fix: add checking memory equality at test

2ce0fa1

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

feat: Use extra config for simple paramters

c77e1db

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

fix: remove valkey-glide depedency from common.txt

c2f6c9c

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

feat: Add try-except and prevent memory leak

1ba7141

- Add try-except under assert statement for checking error - Add memory_obj.ref_count_down() for preventing memory leak Signed-off-by: Jungwoo Song <bluayer@gmail.com>

feat: double check for config and extra_config

5cae09f

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

fix: lint

936fd08

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

docs: Add docs for valkey connector

5a05a00

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

fix: Move import statement to the head of file and exception handling

afcdee2

Signed-off-by: Jungwoo Song <bluayer@gmail.com>

bluayer force-pushed the feature/valkey-connector branch from bee0f18 to afcdee2 Compare October 21, 2025 05:15

sammshen merged commit a5596c1 into LMCache:dev Oct 21, 2025
15 checks passed

bluayer deleted the feature/valkey-connector branch October 22, 2025 05:52

bluayer mentioned this pull request Dec 23, 2025

Support Valkey Backend #1621

Closed

zhengfeihe mentioned this pull request Jul 14, 2026

[Feat]Add Valkey L2 adapter (standalone + cluster) for MP mode #3505

Merged

2 tasks done

jbrinkman mentioned this pull request Aug 13, 2026

[Proposal] Add LMCache + Valkey cookbook (Python KV cache connector) valkey-io/valkey-developer-resources#93

Closed

1 of 3 tasks

Conversation

bluayer commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot commented Oct 5, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

YaoJiayi commented Oct 7, 2025

Uh oh!

bluayer commented Oct 13, 2025

Uh oh!

maobaolong commented Oct 15, 2025

Uh oh!

maobaolong commented Oct 15, 2025

Uh oh!

bluayer commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maobaolong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sammshen commented Oct 16, 2025

Uh oh!

sammshen left a comment

Choose a reason for hiding this comment

Uh oh!

bluayer commented Oct 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sammshen commented Oct 19, 2025

Uh oh!

maobaolong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

maobaolong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sammshen commented Oct 21, 2025

Uh oh!

bluayer commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

sammshen commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

meet-bhagdev commented Oct 22, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bluayer commented Oct 5, 2025 •

edited

Loading

bluayer commented Oct 15, 2025 •

edited

Loading

bluayer commented Oct 19, 2025 •

edited

Loading

bluayer commented Oct 21, 2025 •

edited

Loading

sammshen commented Oct 21, 2025 •

edited

Loading