Skip to content

fix(mcp): improve cache hits when using MCPs - #1271

Merged
caarlos0 merged 1 commit into
mainfrom
fix-cache
Oct 21, 2025
Merged

fix(mcp): improve cache hits when using MCPs#1271
caarlos0 merged 1 commit into
mainfrom
fix-cache

Conversation

@caarlos0

@caarlos0 caarlos0 commented Oct 20, 2025

Copy link
Copy Markdown
Contributor

This ensures that tools always have the same order, so caching works better.

In practice this means faster responses to subsequent messages in a conversation.

Closes #1249

@caarlos0
fix(mcp): improve cache hits when using MCPs
abdf962 
This ensures that tools always have the same order, so caching works
better.

In practice this means faster responses to subsequent messages in a
conversation.

Closes #1249

Signed-off-by: Carlos Alexandro Becker <caarlos0@users.noreply.github.com>
@caarlos0 caarlos0 self-assigned this Oct 20, 2025
@caarlos0
caarlos0 requested a review from a team as a code owner October 20, 2025 20:41
@caarlos0
caarlos0 requested review from aymanbagabas and raphamorim and removed request for a team October 20, 2025 20:41
@caarlos0 caarlos0 mentioned this pull request Oct 20, 2025
Closed
@Buhrietoe

Buhrietoe commented Oct 20, 2025

Copy link
Copy Markdown

Confirmed this fixes my issue in #1249
Only added 13 prompt tokens on secondary completion.

llama.cpp logs:

slot update_slots: id 0 | task 163 | n_past = 18064, memory_seq_rm [18064, end)
slot update_slots: id 0 | task 163 | prompt processing progress, n_past = 18077, n_tokens = 13, progress = 1.000000
slot update_slots: id 0 | task 163 | prompt done, n_past = 18077, n_tokens = 13
slot print_timing: id 0 | task 163 |
prompt eval time = 239.29 ms / 13 tokens ( 18.41 ms per token, 54.33 tokens per second)
eval time = 527.60 ms / 22 tokens ( 23.98 ms per token, 41.70 tokens per second)
total time = 766.90 ms / 35 tokens
slot release: id 0 | task 163 | stop processing: n_past = 18098, truncated = 0
srv update_slots: all slots are idle

@caarlos0
caarlos0 merged commit 4519e19 into main Oct 21, 2025
18 checks passed
@caarlos0
caarlos0 deleted the fix-cache branch October 21, 2025 00:27
meowgorithm pushed a commit that referenced this pull request May 13, 2026
@caarlos0
fix(mcp): improve cache hits when using MCPs (#1271)
ab6b8a3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aymanbagabas aymanbagabas Awaiting requested review from aymanbagabas aymanbagabas was automatically assigned from charmbracelet/everyone
@raphamorim raphamorim Awaiting requested review from raphamorim raphamorim was automatically assigned from charmbracelet/everyone

Assignees

@caarlos0 caarlos0

Labels

area: mcp

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Prompt Re-Processing - Performance regression starting with v0.11.0

3 participants

@caarlos0 @Buhrietoe @charmcli