Skip to content

serve: expose guarded prompt ids from apply-template - #603

Merged
juliankang4 merged 1 commit into
midagedev:mainfrom
Burntgogi:feat/issue591-guarded-prompt-ids
Oct 21, 2026
Merged

juliankang4 merged 1 commit into
midagedev:mainfrom
Burntgogi:feat/issue591-guarded-prompt-ids

Conversation

@Burntgogi

Copy link
Copy Markdown
Contributor

Closes #591.

Contract and implementation

  • Opt in with POST /apply-template and "return_prompt_ids": true: the existing prompt is accompanied by prompt_ids.
  • Absent/false/null preserves the old text-only response and does not tokenize. Wrong types return a named boolean 400.
  • One render and one shared private Rendered::ids owner for this route and chat; uses existing guarded/marked tokenization, not a re-tokenization of visible text.
  • Media placeholders remain unexpanded in both fields; these are pre-media-expansion ids, not expanded multimodal engine input. Documented in docs/serving.md.
  • Based on main 938167b, including serve: DeepSeek's official V4.1 prompt encoding #595's shared request preparation. serve: authorize request heads and bound network image decoding #590 remains owned by the other PC/session; no changes copied or pushed there.

Verification

  • FAIL-first real-BPE regression: old implementation omits prompt_ids. After the change the response keeps template <|im_start|>/<|im_end|> as special ids while a user's USER_DEFINED <think> is the seven ordinary byte ids.
  • Built the bloomery-serve subprocess prerequisite, then ran the exact gate-serve cargo selection through tools/gate.sh: 522 tests passed (338 library + 184 integration, no ignored/skipped tests).
  • Actual loopback bloomery-serve HTTP listener, explicitly MockEngine/MockTokenizer: default/false/null shape, opt-in ids, /detokenize roundtrip including Korean text, count equality against /chat/completions/input_tokens, named wrong-type 400, orderly /shutdown: all observed. Real-BPE isolation is covered by the regression above, not claimed for the mock vocabulary; no GPU inference run.
  • cargo fmt --all --check, standard cargo clippy -p bloomery-serve --all-targets, check-comments.sh, check-arch.sh: passed. Clippy reports one pre-existing test-only too_many_arguments warning in unchanged genloop.rs:2510; an extra strict -D warnings invocation therefore failed and was not hidden or relaxed in source.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CLEAN: The opt-in response reuses chat's guarded tokenization and preserves the default response.

Checked

Duplicates: 2 new functions checked; no duplicate owner. Rendered::ids owns the extracted encoding logic. Dryer reported 0 candidate pairs.
Complexity: crapper flagged no functions. The route adds an optional response field; the new method moves the existing two-arm match.
Silent failure: 1 default site followed at api.rs:2436. Only absent/null becomes false; wrong types propagate a named 400.
Tests: static mutation check — replacing guarded encoding at api.rs:2460 with ordinary encoding fails the new exact-id assertion; unconditional ids fail its default-shape assertions. Accepting a string flag at api.rs:2436 fails the changed HTTP test's status assertion.
Gates: no pins, thresholds or cases removed or relaxed. The PR body records FAIL-first for the new regression.
Performance: chat retains one encoding per prompt; the route encodes ids only when requested. No decode-step work changes.
Escape catalog: no C1–C4 patterns matched.

Notes

Static review only; tests were not executed.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@Burntgogi

Copy link
Copy Markdown
Contributor Author

Read-only coordination/integration check complete at the other session's published head d3c938f: T0 is SUCCESS, review APPROVED, mergeable. I did not edit or push this branch.

In a new detached temporary worktree, #603 (ebcb327, based on main including #595) merged with #590 without conflicts in api.rs or docs/serving.md. The real-BPE guarded-ID regression passed on the combined tree. An actual combined MockEngine HTTP listener with a smoke-only API key returned 401 to an unauthorized request head without waiting for its declared body, then served authorized opt-in prompt IDs with a detokenization roundtrip and orderly shutdown. The temporary merge/worktree was removed after verification. No GPU inference or merger of either PR is claimed. This account has READ permission; landing remains with a maintainer/the owning session.

@juliankang4
juliankang4 merged commit e9ec9c7 into midagedev:main Oct 21, 2026
6 checks passed
@juliankang4

Copy link
Copy Markdown
Collaborator

@Burntgogi Thanks for the contribution. The review is done, so I'm merging this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation serve The HTTP server: crates/serve

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serve: show the prompt ids a chat request runs (the guarded ids differ from /tokenize of /apply-template)

2 participants