Skip to content

fix(llm): make ChatContext.truncate() honor a non-positive max_items - #7367

Merged
longcw merged 2 commits into
livekit:mainfrom
sxh313:fix/chat-context-truncate-zero
Sep 30, 2026
Merged

longcw merged 2 commits into
livekit:mainfrom
sxh313:fix/chat-context-truncate-zero

Conversation

@sxh313

@sxh313 sxh313 commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

What

ChatContext.truncate(max_items=...) used self._items[-max_items:]. In Python -0 is 0, and items[0:] is the whole list -- so truncating to a budget of zero items returned every item, and a negative budget silently trimmed from the front of the conversation instead of keeping nothing.

The guard now treats any non-positive budget as "no conversational items", which still leaves the preserved instruction message, exactly as the docstring describes.

before:  truncate(max_items=0)  -> ['system', 'user', 'assistant', 'user']   (nothing dropped)
after:   truncate(max_items=0)  -> ['system']
before:  truncate(max_items=-2) -> front-trimmed slice
after:   truncate(max_items=-2) -> ['developer']

Why it matters

truncate is the documented way to bound a context (llm/chat_context.py:571). A budget normally comes from a token/message calculation, and the interesting case is the one where it reaches 0: the caller asks for no history and gets all of it, so the request goes out at full size instead of failing or shrinking -- the opposite of what the bound was for. #4843 / #4845 earlier fixed which roles count as an instruction and the duplicate insert, but the slice itself was untouched, so this path is still live.

Valid (positive) budgets are unaffected: len(self._items) <= max_items still short-circuits, and items[-max_items:] is unchanged for max_items > 0.

Tests

Three regression cases in tests/test_chat_ctx.py, next to the existing truncate tests. Differential, with only chat_context.py reverted:

FAILED tests/test_chat_ctx.py::test_truncate_zero_max_items_keeps_only_instruction
FAILED tests/test_chat_ctx.py::test_truncate_zero_max_items_without_instruction
FAILED tests/test_chat_ctx.py::test_truncate_negative_max_items_keeps_only_instruction
3 failed, 6 passed, 63 deselected

with the fix: 71 passed, 1 skipped   (whole tests/test_chat_ctx.py)

ruff check and ruff format --check are clean on both files.

AI disclosure

This fix was produced with an AI coding assistant (Claude), reviewed and verified by me locally as described above.

@sxh313
sxh313 requested a review from a team as a code owner September 20, 2026 16:50
@CLAassistant

CLAassistant commented Sep 20, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@sxh313
sxh313 force-pushed the fix/chat-context-truncate-zero branch from 642ffb8 to b5acab8 Compare September 20, 2026 16:56
Comment thread livekit-agents/livekit/agents/llm/chat_context.py Outdated
@sxh313
sxh313 force-pushed the fix/chat-context-truncate-zero branch 2 times, most recently from dfa5c16 to 70710f2 Compare September 25, 2026 12:53
`items[-max_items:]` is the whole list when max_items is 0, so truncating to
zero silently kept the entire conversation; a negative value trimmed the front.
Review feedback: a negative budget has no sensible meaning, so rejecting it
is better than guessing an empty one. Zero still keeps only the preserved
instruction; the guard sits ahead of the under-limit early return so the
contract holds for every context size.
@sxh313
sxh313 force-pushed the fix/chat-context-truncate-zero branch from 70710f2 to e7b8d68 Compare September 25, 2026 15:30
@longcw
longcw merged commit 157ed1d into livekit:main Sep 30, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants