Skip to content

fix(core): keep tool call pairs during token truncation - #8025

Open
林SO (Linxiushen) wants to merge 1 commit into
microsoft:mainfrom
Linxiushen:fix/token-context-tool-pairs
Open

fix(core): keep tool call pairs during token truncation#8025
林SO (Linxiushen) wants to merge 1 commit into
microsoft:mainfrom
Linxiushen:fix/token-context-tool-pairs

Conversation

@Linxiushen

Copy link
Copy Markdown

Why are these changes needed?

TokenLimitedChatCompletionContext removes messages from the middle of the history one at a time. When the midpoint lands on either side of an assistant tool call and its execution result, this can leave an invalid history containing an orphaned result or call.

This change treats adjacent assistant tool-call and execution-result messages as an indivisible pair during token truncation. It applies the same behavior to both explicit token_limit counting and model-client remaining_tokens counting.

Regression coverage exercises both possible midpoint positions across both token-limit modes.

Related issue number

Closes #7955

Checks

  • I've included any doc changes needed for https://microsoft.github.io/autogen/. No documentation changes are needed for this internal bug fix.
  • I've added tests (if relevant) corresponding to the changes introduced in this PR.
  • I've made sure all auto checks have passed.

Local checks:

  • pytest tests -q (230 passed)
  • ruff format --check src tests
  • ruff check src tests
  • mypy --config-file ../../pyproject.toml --exclude src/autogen_core/application/protos --exclude tests/protos src tests
  • pyright --pythonpath ../../.venv/Scripts/python.exe

@Linxiushen

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping an adjacent tool-call and its result together during truncation is the right call, since splitting the pair leaves a dangling result the model cannot interpret. The helper chooses a middle message and expands in either direction, which is reasonable. One edge: when the middle message is a tool call with no following result, or a result with no preceding call, it falls back to popping a single message — consider a comment spelling out that a lone message may legitimately be removed in those cases so it reads as intended rather than accidental.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TokenLimitedChatCompletionContext can orphan a FunctionExecutionResultMessage mid-list when truncating

2 participants