Repository navigation
serve: expose guarded prompt ids from apply-template - #603
Conversation
There was a problem hiding this comment.
Verdict: CLEAN: The opt-in response reuses chat's guarded tokenization and preserves the default response.
Checked
Duplicates: 2 new functions checked; no duplicate owner. Rendered::ids owns the extracted encoding logic. Dryer reported 0 candidate pairs.
Complexity: crapper flagged no functions. The route adds an optional response field; the new method moves the existing two-arm match.
Silent failure: 1 default site followed at api.rs:2436. Only absent/null becomes false; wrong types propagate a named 400.
Tests: static mutation check — replacing guarded encoding at api.rs:2460 with ordinary encoding fails the new exact-id assertion; unconditional ids fail its default-shape assertions. Accepting a string flag at api.rs:2436 fails the changed HTTP test's status assertion.
Gates: no pins, thresholds or cases removed or relaxed. The PR body records FAIL-first for the new regression.
Performance: chat retains one encoding per prompt; the route encodes ids only when requested. No decode-step work changes.
Escape catalog: no C1–C4 patterns matched.
Notes
Static review only; tests were not executed.
🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)
|
Read-only coordination/integration check complete at the other session's published head d3c938f: T0 is SUCCESS, review APPROVED, mergeable. I did not edit or push this branch. In a new detached temporary worktree, #603 (ebcb327, based on main including #595) merged with #590 without conflicts in api.rs or docs/serving.md. The real-BPE guarded-ID regression passed on the combined tree. An actual combined MockEngine HTTP listener with a smoke-only API key returned 401 to an unauthorized request head without waiting for its declared body, then served authorized opt-in prompt IDs with a detokenization roundtrip and orderly shutdown. The temporary merge/worktree was removed after verification. No GPU inference or merger of either PR is claimed. This account has READ permission; landing remains with a maintainer/the owning session. |
|
@Burntgogi Thanks for the contribution. The review is done, so I'm merging this. |
Closes #591.
Contract and implementation
POST /apply-templateand"return_prompt_ids": true: the existingpromptis accompanied byprompt_ids.Rendered::idsowner for this route and chat; uses existing guarded/marked tokenization, not a re-tokenization of visible text.docs/serving.md.Verification
prompt_ids. After the change the response keeps template<|im_start|>/<|im_end|>as special ids while a user's USER_DEFINED<think>is the seven ordinary byte ids.bloomery-servesubprocess prerequisite, then ran the exactgate-servecargo selection throughtools/gate.sh: 522 tests passed (338 library + 184 integration, no ignored/skipped tests).bloomery-serveHTTP listener, explicitly MockEngine/MockTokenizer: default/false/null shape, opt-in ids,/detokenizeroundtrip including Korean text, count equality against/chat/completions/input_tokens, named wrong-type 400, orderly/shutdown: all observed. Real-BPE isolation is covered by the regression above, not claimed for the mock vocabulary; no GPU inference run.cargo fmt --all --check, standardcargo clippy -p bloomery-serve --all-targets,check-comments.sh,check-arch.sh: passed. Clippy reports one pre-existing test-onlytoo_many_argumentswarning in unchangedgenloop.rs:2510; an extra strict-D warningsinvocation therefore failed and was not hidden or relaxed in source.