Skip to content

fix(server): hoist system messages to front for templates requiring system-first - #487

Open
rafaeldrincon wants to merge 2 commits into
FlashML-org:mainfrom
rafaeldrincon:fix/hoist-system-messages
Open

rafaeldrincon wants to merge 2 commits into
FlashML-org:mainfrom
rafaeldrincon:fix/hoist-system-messages

Conversation

@rafaeldrincon

Copy link
Copy Markdown

Problem

Some chat templates (e.g. Qwen3.6) validate message order and raise:

System message must be at the beginning.

when the system message is not at index 0. This causes a 400 Bad Request for callers that send system messages in non-first positions (e.g. OpenAI-compatible API gateways that reorder messages, or multi-turn conversations where a system message appears after a user message).

Fix

render_messages() in generation.py now hoists system messages to the front of the list before returning. This is a no-op when:

  • System message is already at index 0
  • No system message is present

Verification

Tested with Qwen3.6-35B-A3B (NVFP4):

  • Before: POST /v1/chat/completions with [user, system, user]400 "System message must be at the beginning"
  • After: same request → 200 OK, correct response
  • Also verified with tools present (system + tools array) → 200 OK

Tests

Three unit tests added to tests/server/test_mm_input.py:

  • test_render_messages_hoists_system_to_front — system not first → hoisted
  • test_render_messages_preserves_system_first — system already first → no-op
  • test_render_messages_no_system_unchanged — no system → order preserved

Impact

  • Minimal: 6 lines of logic, no new dependencies, no API change
  • Backward compatible: only reorders when system messages exist and aren't first
  • Multiple system messages: all grouped at front (preserves relative order among system msgs)

…ystem-first

Some chat templates (e.g. Qwen3.6) raise 'System message must be at the
beginning' when the system message is not at index 0. render_messages()
now reorders system messages to the front before passing to the template.

This is a no-op when system is already first or absent.
Qwen3.6 chat template raises 'System message must be at the beginning'
when a second system message appears after loop.first. Merging all
system messages into a single one at index 0 fixes this.

Tested with: system+user+system, user+system+user (hoisted+merged).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant