From 4e258f325b7089a3437394fe03a4702159ab6c59 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Thu, 1 Oct 2026 13:26:43 -0400 Subject: [PATCH 01/53] design/84: a local OpenAI-compatible backend behind the same loop (proposed) Notebook only, no code. Records the 2026-10-01 llama.cpp/Qwen spike against 9e9271a, decisions D1-D8, blocks 84a/84b, gate 84, and two operator questions. Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 152 +++++++++++++++++++ 1 file changed, 152 insertions(+) create mode 100644 design/84-local-openai-compatible-backend.md diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md new file mode 100644 index 00000000..acbed990 --- /dev/null +++ b/design/84-local-openai-compatible-backend.md @@ -0,0 +1,152 @@ +# A local model behind the same loop + +Status: **PROPOSED**, 2026-10-01. Nothing implemented. The evidence below is a +three-request spike, not a session. + +## Problem + +Microclaw can only think through the Anthropic API. An operator wants to run it +against a local Qwen served by llama.cpp on another machine — no per-call bill +(design/82: $102 over five sessions), and no dependence on an external service. +Microclaw must not anchor on that machine, that model or that server: the +target is "any OpenAI-compatible `/v1/chat/completions` endpoint". + +The seam is one function. `agent._stream_one_round` is the only SDK call site; +it returns a `Message` whose `content` blocks (`text`, `tool_use`) and +`stop_reason` drive `run_agent_iter`. Everything downstream — the history +JSONL, `ConversationStore`, `export_session_script`, the viewer — reads the +Anthropic-shaped history, not the SDK. + +## Evidence (spike, 2026-10-01) + +Server: llama.cpp `b11260-e904318a2`, `Qwen3.8-27B-UD-Q4_K_XL.gguf`, +`-c 65536 -np 1 --jinja`, `n_ctx_train 262144`, `modalities.vision: false`, +chat template supports tools and ``. Request: `SYSTEM_PROMPT` alone (no +knowledge base, no rig interview) plus all 83 `TOOLS` translated to +`{"type":"function","function":{name, description, parameters}}`, non-streamed. +Main at `9e9271a`. No tool was executed; round 3 answered with a **synthetic** +payload that said so in a `note` field. + +| request | prompt tokens | new / cached | prefill | wall | result | +|---|---|---|---|---|---| +| cold "stage position?" | 33,592 | 33,592 / 0 | 37.3 s | 44.0 s | `get_xy_position` + `get_z_position` | +| warm "50 ms, DAPI" | 33,600 | 26 / 33,574 | 0.17 s | 4.6 s | `set_exposure {"ms":50}` + `get_available_channels` | +| tool result → reply | 33,688 | 114 / 33,574 | 0.27 s | 11.8 s | `end_turn` prose | + +Prefill ~900 tok/s, generation ~30 tok/s. Every tool name called exists and +every argument matched its schema. The same three requests on the older +`128c478` tree (81 tools, 32,499 tokens) gave the same shape. + +**One behavioural observation, n=2.** On `128c478` the reply relayed the +payload's "synthetic" note; on `9e9271a` it reported the synthetic numbers as +"Current stage position" with no caveat. That is the *say only what a tool told +you* rule, and it is what the gate scores — not a finding about the model. + +What the spike did **not** measure: streaming, a session near the context +limit, compaction, the knowledge-base and skill-discovery blocks, images, and +any prompt-level rule over a real session. + +## Decisions + +**D1 — One backend, translated at the seam.** The canonical history stays +Anthropic-shaped. A second branch of `_stream_one_round` translates the request +out and the reply back, and returns a real `anthropic.types.Message` built with +`Message.model_validate` — so `run_agent_iter` (which reads `block.type`, +`block.id`, `response.stop_reason` by attribute), the history JSONL and the +exporter cannot tell which backend produced it. Checked: a translated +`tool_use` reply validates and round-trips `id`, `stop_reason` and +`cache_read_input_tokens`. No provider registry, no abstract base class: two +branches of one function. + +**D2 — Translation.** + +| Anthropic (history) | chat completions (wire) | +|---|---| +| `system` blocks | one `system` message, blocks joined; `cache_control` dropped | +| `tools[i]` | `{"type":"function","function":{name, description, parameters: input_schema}}`; `cache_control` dropped | +| assistant `text` + `tool_use` blocks | one assistant message: `content` + `tool_calls[{id, function:{name, arguments: json}}]` | +| user `tool_result` blocks | one `tool` message per block, `tool_call_id`, text content; `is_error` prefixed into the text | +| `image` block in a tool result | text-only model: replaced by `{"image_omitted": "this model does not accept images"}` — never dropped silently. Vision model: a following `user` message with `image_url` data URIs | +| `finish_reason` `stop` / `tool_calls` / `length` | `stop_reason` `end_turn` / `tool_use` / `max_tokens` | +| `delta.content` | `text_delta` event | +| `delta.reasoning_content` | not streamed as prose, not stored in history (the template strips it on replay); its length goes to `usage_sink` | +| `usage.prompt_tokens`, `prompt_tokens_details.cached_tokens` | `input_tokens`, `cache_read_input_tokens` in the existing `usage_sink` record | + +Tool-call arguments arrive as JSON *text*; an unparsable one becomes a +`tool_result` error naming the parse failure, so the model sees and corrects it. + +**D3 — Selection is one setting.** `--llm-base-url` / `MICROCLAW_LLM_BASE_URL` +set means the local backend; unset means Anthropic, unchanged. The model id +defaults to the server's single `/v1/models` entry, so the operator names +nothing else; the existing model picker lists that endpoint's models. The key, +when the server has one, goes through `credentials.py` under its own name — +never the Anthropic one — and the startup checks (`webserve.py`'s "No Anthropic +API key", `__main__`) accept a configured base URL instead. The URL is +operator configuration, never a constant in `microclaw/`. + +**D4 — Vision is read, not assumed.** If the server answers llama.cpp's +`/props`, `modalities.vision` decides. Otherwise text-only, the safe direction: +an omitted-image note is honest, an image sent to a model that cannot read it +is a 400 or a hallucination. + +**D5 — Errors map onto the existing two.** Connection refused/reset, 503 and +429 → the transient retry path, message naming the base URL. 404 on the model +→ `_BadModel`. A 400 whose body reports the context exceeded → an error that +states the server's `n_ctx` and the request's token count, not a generic +failure. `TRANSIENT_API_ERROR_MESSAGE` stops saying "Anthropic" on this path. + +**D6 — The context budget comes from the server.** `ConversationStore`'s +120k/90k watermarks are Anthropic-sized; at `n_ctx 65536` a session would +overflow long before it compacted. Read `n_ctx` (llama.cpp `/props` or +`/v1/models` `meta.n_ctx`; otherwise a stated default) and set +`high = n_ctx − fixed_prefix − max_output`, low at 75 % of high. At the spike's +numbers that is 65,536 − 33,592 − 8,192 ≈ **23.7k tokens of history**, which is +small (see Q1). + +**D7 — Timing contract.** Prefix reuse is what makes this usable: 37 s cold +against 0.2 s warm per round. Anything that rewrites the prompt's front pays the +cold cost again — compaction, a changed knowledge base or skill-discovery block, +another client on a one-slot server. The local branch must keep the prefix +byte-stable between rounds (same system string, same tool order, same +serialisation of earlier turns), and `usage_sink` records `prompt_n`, +`cache_n` and `prompt_ms` so a cold hit is visible per round rather than +inferred. Not attributed yet: how often a real session goes cold. + +**D8 — Export is untouched.** `export_session_script` reads recorded tool calls +and results; the history shape does not change, so no emitter changes. A test +asserts a session driven through the local branch exports the same script as the +same calls driven through the Anthropic one. + +## Blocks + +**84a — the backend.** D1–D5 and D8. Streaming chat-completions client over +`httpx` (already a direct dependency; no new package), translation +both ways, selection, credentials, startup checks, `usage_sink` mapping. +Tests: a fake server written from llama.cpp's actual SSE shape (`tool_calls` +deltas split across chunks, `reasoning_content` deltas, the final `timings` +object), not from our caller. Watched-fail evidence on each translation row. + +**84b — the budget and the prefix.** D6 and D7: watermarks from `n_ctx`, a +prefix-stability test (two consecutive rounds serialise an identical leading +byte range), compaction measured as a cold hit. + +**Gate 84 — one driven session against a real llama.cpp server**, from a +machine that can reach it. Score from the history and usage JSONL: per-round +`prompt_ms` and `cache_n` (how many cold rounds, and which events caused each), +whether compaction fired before overflow, and the reporting rules (numbers only +from tool results; `unknown` relayed as `unknown`; illumination state at +hand-off). Image tools exercised once to see the omitted-image note reach the +model. + +## Open questions for the operator + +- **Q1 — context size.** 23.7k of history at `-c 65536` means frequent + compaction, each a ~37 s cold round. The model trained at 262,144; does the + workstation have memory for `-c 131072` (≈ 89k of history)? Quantised KV + (`-fa on --cache-type-k q8_0 --cache-type-v q8_0`) roughly halves the cost. +- **Q2 — gate machine.** Which machine runs gate 84: the demo machine (can it + reach the workstation on the JHU network?), M5 or M2? + +## Run ledger + +Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. No block assigned. From 2af5a0817901204a83cc4f9ad4a4eccbb96a9f4f Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Thu, 1 Oct 2026 17:49:21 -0400 Subject: [PATCH 02/53] design/84: record the 128k/q8 KV spike; settle Q1, reframe Q2 Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 32 +++++++++++++++----- 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md index acbed990..636719a9 100644 --- a/design/84-local-openai-compatible-backend.md +++ b/design/84-local-openai-compatible-backend.md @@ -42,6 +42,13 @@ payload's "synthetic" note; on `9e9271a` it reported the synthetic numbers as "Current stage position" with no caveat. That is the *say only what a tool told you* rule, and it is what the gate scores — not a finding about the model. +**At `-c 131072 -fa on --cache-type-k q8_0 --cache-type-v q8_0`** (operator +restart, 2026-10-01; ~3 GB VRAM still free, KV size not printed): the same +three calls with the same arguments, cold prefill 37.9 s, warm 0.19 / 0.29 s, +generation **26.4 tok/s against 30.0** — 12 % slower, not attributed between +the quantised KV, flash attention and the larger context. The reply relayed the +synthetic note again, so that rule held in **2 of 3** runs. + What the spike did **not** measure: streaming, a session near the context limit, compaction, the knowledge-base and skill-discovery blocks, images, and any prompt-level rule over a real session. @@ -100,8 +107,15 @@ failure. `TRANSIENT_API_ERROR_MESSAGE` stops saying "Anthropic" on this path. overflow long before it compacted. Read `n_ctx` (llama.cpp `/props` or `/v1/models` `meta.n_ctx`; otherwise a stated default) and set `high = n_ctx − fixed_prefix − max_output`, low at 75 % of high. At the spike's -numbers that is 65,536 − 33,592 − 8,192 ≈ **23.7k tokens of history**, which is -small (see Q1). +spike's numbers: at `n_ctx 65536` that is ≈ 23.7k tokens of history; at the +operator's chosen `131072`, **≈ 89.3k**, low water ≈ 67k. + +A larger window is not free, and the cost moves rather than vanishing: +compaction becomes rarer, but the round after it re-prefills the whole +low-water prompt — ≈ 100k tokens, **≈ 110 s at the measured 900 tok/s**, and +prefill rate falls with length, so likely more. 84b must measure that round +rather than this estimate, and the operator should see it coming: a progress +event saying the context is being rebuilt, not a silent two-minute pause. **D7 — Timing contract.** Prefix reuse is what makes this usable: 37 s cold against 0.2 s warm per round. Anything that rewrites the prompt's front pays the @@ -140,12 +154,14 @@ model. ## Open questions for the operator -- **Q1 — context size.** 23.7k of history at `-c 65536` means frequent - compaction, each a ~37 s cold round. The model trained at 262,144; does the - workstation have memory for `-c 131072` (≈ 89k of history)? Quantised KV - (`-fa on --cache-type-k q8_0 --cache-type-v q8_0`) roughly halves the cost. -- **Q2 — gate machine.** Which machine runs gate 84: the demo machine (can it - reach the workstation on the JHU network?), M5 or M2? +- **Q1 — context size. Settled, 2026-10-01:** `-c 131072` with q8_0 KV and + flash attention, ~3 GB VRAM spare. Not pushed to 262,144: a cold round after + compaction would then cost minutes (D6). +- **Q2 — gate machine. Open.** The earlier gates ran at another site, which + cannot be assumed to reach `IIC-LLS-Wkstn`. Candidates: Micro-Manager's demo + config on a machine at this site (the workstation itself is the shortest + path); a llama.cpp server at the gate site; or a tunnel, which adds latency to + every timing the gate measures. ## Run ledger From 1c6ab64336272f533e10ee444bfd6d39c8a9a549 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Thu, 1 Oct 2026 17:52:29 -0400 Subject: [PATCH 03/53] design/84: settle the gate machine; open 84a's ledger row Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md index 636719a9..6464aaea 100644 --- a/design/84-local-openai-compatible-backend.md +++ b/design/84-local-openai-compatible-backend.md @@ -157,12 +157,17 @@ model. - **Q1 — context size. Settled, 2026-10-01:** `-c 131072` with q8_0 KV and flash attention, ~3 GB VRAM spare. Not pushed to 262,144: a cold round after compaction would then cost minutes (D6). -- **Q2 — gate machine. Open.** The earlier gates ran at another site, which - cannot be assumed to reach `IIC-LLS-Wkstn`. Candidates: Micro-Manager's demo - config on a machine at this site (the workstation itself is the shortest - path); a llama.cpp server at the gate site; or a tunnel, which adds latency to - every timing the gate measures. +- **Q2 — gate machine. Settled, 2026-10-01:** the llama.cpp workstation + itself, with Micro-Manager's demo config and Microclaw installed beside the + model (operator to install). No network path between the gate's Microclaw and + its server, so the timings are a local deployment's, not a tunnel's. The + earlier gate machines are at another site and are not assumed to reach it. ## Run ledger -Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. No block assigned. +Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. + +| block | branch | start | state | +|---|---|---|---| +| 84a | `design84/84a-backend` | `Qwen-local-model` tip at assignment | assigned 2026-10-01 | +| 84b | — | — | not started | From e1fadc4fa190c4879198d02bec0bda331b0bafd3 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Thu, 1 Oct 2026 17:54:58 -0400 Subject: [PATCH 04/53] CLAUDE.md: the yonce push key is per-machine; ask which remote elsewhere Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 79bd5afa..97dbede1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -657,7 +657,10 @@ because a block looks small. Standing rules that support the above: one worktree per concurrent agent, never `git add -A`, never `pip install -e .` while another tree is live. Push uses a -non-default SSH key (`GIT_SSH_COMMAND="ssh -i ~/.ssh/yonce"`). Rig-facing +non-default SSH key (`GIT_SSH_COMMAND="ssh -i ~/.ssh/yonce"`) to `origin` — on +the machine that has that key. Elsewhere, check `git remote -v` and ask which +remote a block's branches go to (design/84's go to the operator's fork, `ian`, +with the default key). Rig-facing commands must be PowerShell/cmd-safe. Rig facts belong in gate docs, design notes, and rig profiles — never in `microclaw/`. From afa31ef9a0cb717af97cb4e1fb365b5ac4321c3b Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 12:45:29 -0400 Subject: [PATCH 05/53] design/84: record 84a's review Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 22 +++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md index 6464aaea..5164ddb1 100644 --- a/design/84-local-openai-compatible-backend.md +++ b/design/84-local-openai-compatible-backend.md @@ -169,5 +169,25 @@ Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. | block | branch | start | state | |---|---|---|---| -| 84a | `design84/84a-backend` | `Qwen-local-model` tip at assignment | assigned 2026-10-01 | +| 84a | `design84/84a-backend` | `55ccc95` (fixtures) on `4b2e9cf` | reviewed 2026-10-02, gate not run | | 84b | — | — | not started | + +**84a review, 2026-10-02.** Baseline at `55ccc95`: 5512 passed, 108 skipped, +1 failed (`test_built_wheel_contains_the_source_tree_skill_catalog`, passes +alone; ran during a network drop). Three runner turns: `f6917b3` (incomplete +by its own report), `ec16e85` (round 1: eleven defects F1–F11, among them the +compaction checkpoint silently dropped and `--llm-base-url` lost before +`serve`), `64065e6` (round 2: G1–G7; turn killed by the coordinator's 2-hour +background limit during its final check, committed as found, then reviewed). +Coordinator full suite on `64065e6`: **5532 passed, 108 skipped**. Mutation +check, each D2 row and error mapping broken in turn: 11/14 caught on +`ec16e85`, **19/19** on `64065e6`. Live against the workstation through +`_stream_one_round` with `_system_blocks()` (33,889 tokens): 124 streamed +deltas, first at 8.1 s, `end_turn`; second round `get_z_position {}` with +`cache_n 33853`, `prompt_ms 174`. Known nits left: one-line `if`s and a +redundant `kind` branch in `webserve.py`'s key route. + +**Runner lesson.** A revision turn outlived Claude Code's 2-hour background cap +when the network dropped mid-turn. Launch long runner turns under `nohup` +(CLAUDE.md step 2) rather than as session background tasks. + From 6bece9bbad44d94c546ac5b9301de0628b78d66d Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 12:54:35 -0400 Subject: [PATCH 06/53] design/70: R160, estimate_tokens counts thumbnail base64 as text Co-Authored-By: Claude Opus 5.5 --- design/70-carried-forward-register.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/design/70-carried-forward-register.md b/design/70-carried-forward-register.md index 2af3711b..89ab48a3 100644 --- a/design/70-carried-forward-register.md +++ b/design/70-carried-forward-register.md @@ -3403,3 +3403,14 @@ the model" as the actual blast radius. - **Importance** — MEDIUM. Recommended before 89b, not required: 89b's own M2 gate compares cadence with feedback on and off, so this would only find a problem earlier and more cheaply (operator decision, 2026-10-19). - **Effort** — SMALL - **Provenance** — `design/89` 89a, 2026-10-19. + + +### R160 — `estimate_tokens` counts a thumbnail's base64 bytes as text + +**One 512 px PNG thumbnail of a noisy field estimates at 121,748 tokens — above the Anthropic path's 120k high-water on its own.** + +- **What happened** — measured 2026-10-02 (design/84, coordinator): `make_thumbnail` of a 2048² uint16 noise frame is 287,144 base64 bytes, and `estimate_tokens` over one `tool_result` holding it returns 121,748; the same call on a smooth ramp is 6,040 bytes and 2,636. `estimate_tokens` divides the JSON of the whole history by 2.36, image data included. Real camera frames are noise-dominated, so their thumbnails are the large case. +- **Why it matters** — a single `snap_and_analyze` that keeps its thumbnail can push `ConversationStore` past high-water and fold every older turn into a checkpoint on the next round, rewriting the cached prefix (design/82's cost lever) for an image the API bills at a few hundred tokens. Not observed in a recorded session yet: no artifact has been checked for a thumbnail-triggered compaction. +- **What a fix would look like** — estimate an image block at its billed size (a per-image constant from the API's image-token rule) rather than its byte length; check an archived design/82 history for a compaction that followed a thumbnail. design/84 84b fixes the local text-only path only, where the image is never sent. +- **Provenance** — design/84 84b planning, 2026-10-02. + From 55845b277af84c54f80b7c4c52c4fb19f2db3c8c Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 15:50:21 -0400 Subject: [PATCH 07/53] design/84: record 84b's review; gate 84 pushed Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 28 +++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md index 5164ddb1..c39eb964 100644 --- a/design/84-local-openai-compatible-backend.md +++ b/design/84-local-openai-compatible-backend.md @@ -170,7 +170,7 @@ Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. | block | branch | start | state | |---|---|---|---| | 84a | `design84/84a-backend` | `55ccc95` (fixtures) on `4b2e9cf` | reviewed 2026-10-02, gate not run | -| 84b | — | — | not started | +| 84b | `design84/84b-budget` (on 84a) | `3b0f1c7` | reviewed 2026-10-02; gate 84 pushed, not run | **84a review, 2026-10-02.** Baseline at `55ccc95`: 5512 passed, 108 skipped, 1 failed (`test_built_wheel_contains_the_source_tree_skill_catalog`, passes @@ -191,3 +191,29 @@ redundant `kind` branch in `webserve.py`'s key route. when the network dropped mid-turn. Launch long runner turns under `nohup` (CLAUDE.md step 2) rather than as session background tasks. +**84b review, 2026-10-02.** Four runner turns: `246a72f` (budget from the +server), `1dcb5b1` (H1: fixed prompt counted through `/apply-template` + +`/tokenize` = **33,864**, equal to the server's own `prompt_tokens`; the +untemplated count was 30,118), `befe1a7` (K1: the browser locked Send without a +key on a keyless local server), `0ecbbcc` (L1–L4: with `--api-key` every +llama.cpp endpoint but `/health` 401s, the store was built before the key was +loaded, and failed probes were cached — the budget clamped to **2**). +Coordinator fixes: `55af3bb` (a stored key was no longer registered with the +history audit's redaction after that reorder), `2780191` (`keyRequired` +declared with the page state; an absent `key_required` reads as required), +`557200f` (a stale round-1 test). Live on the workstation: budget 84,920 / +63,690 with the key; without it a stated provisional 4,096 / 3,072 and every +401 named, re-read exactly when the key is entered. Prefix reuse measured over +a 5-round tool loop: each round reused all but 4–56 tokens of the previous. +Full suite on `a8c4f81`: **5547 passed, 108 skipped, 1 failed** — +`test_built_wheel_contains_the_source_tree_skill_catalog`, which passes alone +and failed the same way at 84a's baseline (2 of 6 full runs this session). + +**D6 correction.** After compaction the system+tools prefix stays cached, so +the slow round re-processes the history only: ≈ low-water (63,690) at +~900 tok/s, about 70 s, not the whole prompt. Gate 84 measures it. + +**For later, not a block yet.** The model's chat template renders "Reasoning +effort is set to xhigh" by default; a lower per-request effort may cut reply +latency substantially. Measure against gate 84's baseline before changing it. + From 57d1647573e462ecb9234d5c76d6e2cb12dc34bb Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Mon, 5 Oct 2026 17:35:53 -0400 Subject: [PATCH 08/53] design/84: score gate 84 round 1; open 84c (compaction only when it helps) Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 58 +++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md index c39eb964..c237b391 100644 --- a/design/84-local-openai-compatible-backend.md +++ b/design/84-local-openai-compatible-backend.md @@ -170,7 +170,8 @@ Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. | block | branch | start | state | |---|---|---|---| | 84a | `design84/84a-backend` | `55ccc95` (fixtures) on `4b2e9cf` | reviewed 2026-10-02, gate not run | -| 84b | `design84/84b-budget` (on 84a) | `3b0f1c7` | reviewed 2026-10-02; gate 84 pushed, not run | +| 84b | `design84/84b-budget` (on 84a) | `3b0f1c7` | gate 84 run 2026-10-05: behaviour passes; compaction defect → 84c | +| 84c | `design84/84c-compaction` (on 84b) | `3e778b5` | assigned 2026-10-05 | **84a review, 2026-10-02.** Baseline at `55ccc95`: 5512 passed, 108 skipped, 1 failed (`test_built_wheel_contains_the_source_tree_skill_catalog`, passes @@ -217,3 +218,58 @@ the slow round re-processes the history only: ≈ low-water (63,690) at effort is set to xhigh" by default; a lower per-request effort may cut reply latency substantially. Measure against gate 84's baseline before changing it. +## Gate 84, round 1 (workstation, 2026-10-05) — scored from artifacts + +Session `20261005_153316_502746`: 14 model calls, all local; 8 tool calls, all +registered, none unparsed; the no-key turn gave the 401 error naming +`MICROCLAW_LLM_API_KEY` (O2) as a toast, Send stayed enabled with no banner +(O1); entering the key re-read the budget exactly (O3b: 33,864 / 84,920 / +63,690); the illumination answer relayed `unknown` from `get_system_state` +(O5). Seven same-turn rounds reused all but ≤ 64 tokens of the previous prompt. +The thumbnail moved the estimate by 519 tokens. Peak prompt + output reserve +120,205 of 131,072. + +**F1 — a compaction that cannot get under high-water is committed anyway, and +repeats.** One `read_hook_log(limit=200)` returned 140,218 characters (~59k +tokens). `ConversationStore` could fold only the small early turns (the +two-recent-turns floor), committed a checkpoint at **87,067 > 84,920**, and +compacted again on each of the next two calls (86,773, then 67,021). Each +commit rewrites the history head, so each paid a full history re-prefill: +**134.0 s, 133.3 s, 98.7 s — 366 of the session's 443 s of prompt +processing**, the second for ~300 tokens of saving. The operator saw a +"rebuilding" toast and minutes of silence, with nothing saying it had not +helped (O4: "several minutes"). + +**F2 — the gate's budget limb could not fail on F1.** It excluded compaction +records, so it passed with a post-compaction estimate above high-water. + +**Measured, not graded.** Prefill slows with position: 826 tok/s at 34k, 671 at +54k, 583 at 112k. Generation was 464 s of the session (more than prompt +processing), with 28,209 characters of hidden reasoning. The history estimate +ran 1.12–1.63× the server's own history count (safe direction). + +## 84c — compaction only when it helps, and say so + +**D9 — commit a compaction only if it reaches high-water or below.** Otherwise +send the current view unchanged: the prefix (and on the Anthropic path the +cache) survives, and the real limit is the server's, which the request still +fits (F1's peak: 120,205 of 131,072). Both backends; the Anthropic path gains +the same saving on cache rewrites (design/82). + +**D10 — the operator sees what a compaction does and costs.** The `compaction` +event carries the estimate before and after, and an expected rebuild time from +this session's own measured prefill rate (not a constant); a declined +compaction emits an information event saying the history is above the +threshold and recent turns cannot be folded yet. Information, never a +confirmation (CLAUDE.md). + +**D11 — the context window is operator configuration, measured, not coded.** +84b already reads `n_ctx`. A larger `-c` makes compaction rarer but each rebuild +longer, and slower per token (above): at `-c 262144` a low-water near 160k would +re-prefill ~125k tokens at well under 583 tok/s — several minutes per rebuild. +Gate 84c measures one larger window rather than assuming it helps. + +**Gate 84c.** Re-run gate 84 on the workstation with the fixed scorer (F2: grade +prompt + reserve ≤ `n_ctx` and report every request sent above high-water; a +mutant for each), expecting one compaction-cost round where round 1 had three. + From 712086dd5ba8c9e0402167e23ae817b2be6e9dec Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Mon, 5 Oct 2026 21:30:59 -0400 Subject: [PATCH 09/53] design/84: record 84c's review; gate 84 round 2 pushed Co-Authored-By: Claude Opus 5.5 --- design/84-local-openai-compatible-backend.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/design/84-local-openai-compatible-backend.md b/design/84-local-openai-compatible-backend.md index c237b391..61ef5102 100644 --- a/design/84-local-openai-compatible-backend.md +++ b/design/84-local-openai-compatible-backend.md @@ -171,7 +171,7 @@ Notebook drafted on `Qwen-local-model` from `main` `9e9271a`. |---|---|---|---| | 84a | `design84/84a-backend` | `55ccc95` (fixtures) on `4b2e9cf` | reviewed 2026-10-02, gate not run | | 84b | `design84/84b-budget` (on 84a) | `3b0f1c7` | gate 84 run 2026-10-05: behaviour passes; compaction defect → 84c | -| 84c | `design84/84c-compaction` (on 84b) | `3e778b5` | assigned 2026-10-05 | +| 84c | `design84/84c-compaction` (on 84b) | `3e778b5` | reviewed 2026-10-05; gate 84 round 2 pushed, not run | **84a review, 2026-10-02.** Baseline at `55ccc95`: 5512 passed, 108 skipped, 1 failed (`test_built_wheel_contains_the_source_tree_skill_catalog`, passes @@ -273,3 +273,17 @@ Gate 84c measures one larger window rather than assuming it helps. prompt + reserve ≤ `n_ctx` and report every request sent above high-water; a mutant for each), expecting one compaction-cost round where round 1 had three. +**84c review, 2026-10-05.** Three runner turns: `fa61636` (D9/D10; three +existing tests weakened until they no longer tested their purpose — 84a's F1 +checkpoint-on-the-wire test among them), `ce3d2fc` (N1–N4: purposes restored at +watermarks where D9 commits; the gate shape pinned — old store 1, 2, 3 commits; +rebuild estimate text-only; declined notice once per stretch), `4c99b81` (six +more fixtures outside the targeted files resized; the design/82 R63 probe now +says it replays under the current policy, so a pre-84c session's partition is +not historical proof). Coordinator: `f26df3e` (gate scorer F2 — fit and D9 +graded separately; re-scored on round 1's real artifacts it fails D9 at records +10 and 11), `c650837` (round-2 runbook). Replay of round 1's real history +through the final store at the gate's watermarks: old 3 commits at calls 10, 11, +12; new: declined once at 10, sent as is at 11, **one** commit at 12. Full suite +on `4c99b81`: **5551 passed, 108 skipped**. + From 0380ac106eed0ac634c51a132697261c8982ed42 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Thu, 1 Oct 2026 18:32:04 -0400 Subject: [PATCH 10/53] design/84 84a: recorded llama.cpp responses for the fake server Recorded by the coordinator from a real server; the runner sandbox has no network. Co-Authored-By: Claude Opus 5.5 --- tests/fixtures/llamacpp/README.md | 10 + tests/fixtures/llamacpp/error_bad_key.http | 4 + .../llamacpp/error_context_exceeded.http | 4 + tests/fixtures/llamacpp/get_props.http | 4 + tests/fixtures/llamacpp/get_v1_models.http | 4 + .../llamacpp/stream_length_cutoff.http | 91 +++ .../fixtures/llamacpp/stream_text_reply.http | 593 ++++++++++++++++++ .../fixtures/llamacpp/stream_tool_calls.http | 105 ++++ 8 files changed, 815 insertions(+) create mode 100644 tests/fixtures/llamacpp/README.md create mode 100644 tests/fixtures/llamacpp/error_bad_key.http create mode 100644 tests/fixtures/llamacpp/error_context_exceeded.http create mode 100644 tests/fixtures/llamacpp/get_props.http create mode 100644 tests/fixtures/llamacpp/get_v1_models.http create mode 100644 tests/fixtures/llamacpp/stream_length_cutoff.http create mode 100644 tests/fixtures/llamacpp/stream_text_reply.http create mode 100644 tests/fixtures/llamacpp/stream_tool_calls.http diff --git a/tests/fixtures/llamacpp/README.md b/tests/fixtures/llamacpp/README.md new file mode 100644 index 00000000..bf53c1b1 --- /dev/null +++ b/tests/fixtures/llamacpp/README.md @@ -0,0 +1,10 @@ +Raw HTTP responses recorded on 2026-10-01 from a real llama.cpp server +(build b11260-e904318a2, Qwen3.8-27B-UD-Q4_K_XL.gguf, `-c 131072 -np 1 --jinja +-fa on --cache-type-k q8_0 --cache-type-v q8_0`, text-only) for design/84. + +Each `.http` file: `HTTP `, a JSON dict of response headers, a blank +line, then the exact body bytes. Chat requests carried Microclaw's +SYSTEM_PROMPT and all TOOLS, streamed with `stream_options.include_usage`, +except `error_bad_key` (wrong key) and `error_context_exceeded` (a 140k-token +user message, no tools). `get_props` / `get_v1_models` are the GET endpoints. +Replay these; do not hand-write SSE. diff --git a/tests/fixtures/llamacpp/error_bad_key.http b/tests/fixtures/llamacpp/error_bad_key.http new file mode 100644 index 00000000..40171704 --- /dev/null +++ b/tests/fixtures/llamacpp/error_bad_key.http @@ -0,0 +1,4 @@ +HTTP 401 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "Content-Type": "application/json; charset=utf-8", "Content-Length": "80", "Connection": "close"} + +{"error":{"message":"Invalid API Key","type":"authentication_error","code":401}} \ No newline at end of file diff --git a/tests/fixtures/llamacpp/error_context_exceeded.http b/tests/fixtures/llamacpp/error_context_exceeded.http new file mode 100644 index 00000000..2e15fe80 --- /dev/null +++ b/tests/fixtures/llamacpp/error_context_exceeded.http @@ -0,0 +1,4 @@ +HTTP 400 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "Content-Type": "application/json; charset=utf-8", "Content-Length": "203", "Connection": "close"} + +{"error":{"code":400,"message":"request (140052 tokens) exceeds the available context size (131072 tokens), try increasing it","type":"exceed_context_size_error","n_prompt_tokens":140052,"n_ctx":131072}} \ No newline at end of file diff --git a/tests/fixtures/llamacpp/get_props.http b/tests/fixtures/llamacpp/get_props.http new file mode 100644 index 00000000..81ce5781 --- /dev/null +++ b/tests/fixtures/llamacpp/get_props.http @@ -0,0 +1,4 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "Content-Type": "application/json; charset=utf-8", "Content-Length": "12106", "Keep-Alive": "timeout=5, max=100"} + +{"default_generation_settings":{"params":{"seed":4294967295,"temperature":1.0,"dynatemp_range":0.0,"dynatemp_exponent":1.0,"top_k":20,"top_p":0.949999988079071,"min_p":0.05000000074505806,"top_n_sigma":-1.0,"xtc_probability":0.0,"xtc_threshold":0.10000000149011612,"typical_p":1.0,"repeat_last_n":64,"repeat_penalty":1.0,"presence_penalty":0.0,"frequency_penalty":0.0,"dry_multiplier":0.0,"dry_base":1.75,"dry_allowed_length":2,"dry_penalty_last_n":64,"mirostat":0,"mirostat_tau":5.0,"mirostat_eta":0.10000000149011612,"adaptive_target":-1.0,"adaptive_decay":0.8999999761581421,"max_tokens":-1,"n_predict":-1,"n_keep":0,"n_discard":0,"ignore_eos":false,"stream":false,"n_probs":0,"min_keep":0,"chat_format":"Content-only","reasoning_format":"none","reasoning_in_content":false,"generation_prompt":"","samplers":["penalties","dry","top_n_sigma","top_k","typ_p","top_p","min_p","xtc","temperature"],"speculative.types":"none","timings_per_token":false,"post_sampling_probs":false,"backend_sampling":false,"lora":[]},"n_ctx":131072},"total_slots":1,"model_alias":"Qwen3.8-27B-UD-Q4_K_XL.gguf","model_ftype":"Q4_K - Medium","model_path":"Qwen3.8-27B-UD-Q4_K_XL.gguf","modalities":{"vision":false,"video":false,"audio":false},"media_marker":"<__media_5y23Ges0DNrB1g55JBPeVW817GpW9uJV__>","endpoint_slots":true,"endpoint_props":false,"endpoint_metrics":false,"ui":true,"ui_settings":{},"chat_template":"{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- macro render_content(content, do_vision_count, is_system_content=false) %}\n {%- if content is string %}\n {{- content }}\n {%- elif content is iterable and content is not mapping %}\n {%- for item in content %}\n {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain images.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set image_count.value = image_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Picture ' ~ image_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|image_pad|><|vision_end|>' }}\n {%- elif 'video' in item or item.type == 'video' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain videos.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set video_count.value = video_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Video ' ~ video_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|video_pad|><|vision_end|>' }}\n {%- elif 'text' in item %}\n {{- item.text }}\n {%- else %}\n {{- raise_exception('Unexpected item type in content.') }}\n {%- endif %}\n {%- endfor %}\n {%- elif content is none or content is undefined %}\n {{- '' }}\n {%- else %}\n {{- raise_exception('Unexpected content type.') }}\n {%- endif %}\n{%- endmacro %}\n{%- if not messages %}\n {{- raise_exception('No messages provided.') }}\n{%- endif %}\n{%- set sysns = namespace(count=0, text='') %}\n{%- for message in messages %}\n {%- if sysns.count == loop.index0 and (message.role == 'system' or message.role == 'developer') %}\n {%- set sys_content = render_content(message.content, false, true)|trim %}\n {%- if sys_content %}\n {%- set sysns.text = sysns.text + ('\\n' if sysns.text else '') + sys_content %}\n {%- endif %}\n {%- set sysns.count = sysns.count + 1 %}\n {%- endif %}\n{%- endfor %}\n{%- set num_sys = sysns.count %}\n{%- set merged_system = sysns.text %}\n{%- set reasoning_instructions = '' %}\n{%- if enable_thinking is undefined or enable_thinking is true %}\n {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}\n {%- if resolved_reasoning_effort == 'high' %}\n {%- set resolved_reasoning_effort = 'xhigh' %}\n {%- endif %}\n {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}\n {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}\n {%- endif %}\n {%- if resolved_reasoning_effort == 'xhigh' %}\n {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}\n {%- elif resolved_reasoning_effort == 'low' %}\n {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}\n {%- endif %}\n{%- endif %}\n{%- if tools and tools is iterable and tools is not mapping %}\n {{- '<|im_start|>system\\n' }}\n {%- if reasoning_instructions %}\n {{- reasoning_instructions + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou have access to the following functions:\\n\\n\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n\" }}\n {{- '\\n\\nIf you choose to call a function ONLY reply in the following format with NO suffix:\\n\\n\\n\\n\\nvalue_1\\n\\n\\nThis is the value for the second parameter\\nthat can span\\nmultiple lines\\n\\n\\n\\n\\n\\nReminder:\\n- Function calls MUST follow the specified format: an inner block must be nested within XML tags\\n- Required parameters MUST be specified\\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\\n' }}\n {%- if merged_system %}\n {{- '\\n\\n' + merged_system }}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n{%- else %}\n {%- if merged_system %}\n {{- '<|im_start|>system\\n' + (reasoning_instructions + '\\n\\n' if reasoning_instructions else '') + merged_system + '<|im_end|>\\n' }}\n {%- elif reasoning_instructions %}\n {{- '<|im_start|>system\\n' + reasoning_instructions + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" %}\n {%- set content = render_content(message.content, false)|trim %}\n {%- if not(content.startswith('') and content.endswith('')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if loop.index0 >= num_sys %}\n {%- set content = render_content(message.content, true)|trim %}\n {%- if message.role == \"system\" or message.role == \"developer\" %}\n {{- raise_exception('System message must be at the beginning.') }}\n {%- elif message.role == \"user\" %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- endif %}\n {%- set reasoning_content = reasoning_content|trim %}\n {%- if preserve_thinking is undefined or preserve_thinking is true or loop.index0 > ns.last_query_index %}\n {{- '<|im_start|>' + message.role + '\\n\\n' + reasoning_content + '\\n\\n\\n' + content }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {%- if tool_call.name is not defined or tool_call.name is none %}\n {{- raise_exception('Tool call is missing a function name.') }}\n {%- endif %}\n {%- if loop.first %}\n {%- if content|trim %}\n {{- '\\n\\n\\n\\n' }}\n {%- else %}\n {{- '\\n\\n' }}\n {%- endif %}\n {%- else %}\n {{- '\\n\\n\\n' }}\n {%- endif %}\n {%- if tool_call.arguments is mapping %}\n {%- for args_name, args_value in tool_call.arguments|items %}\n {{- '\\n' }}\n {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}\n {{- args_value }}\n {{- '\\n\\n' }}\n {%- endfor %}\n {%- elif tool_call.arguments is string %}\n {%- if tool_call.arguments|trim %}\n {{- raise_exception('Tool call arguments for function \"' + (tool_call.name | string) + '\" were passed as a JSON string. Parse them into an object before calling apply_chat_template.') }}\n {%- endif %}\n {%- elif tool_call.arguments is defined and tool_call.arguments is not none %}\n {{- raise_exception('Tool call arguments for function \"' + (tool_call.name | string) + '\" must be an object/mapping or a JSON string.') }}\n {%- endif %}\n {{- '\\n' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.previtem and loop.previtem.role != \"tool\" %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n\\n' }}\n {{- content }}\n {{- '\\n' }}\n {%- if not loop.last and loop.nextitem.role != \"tool\" %}\n {{- '<|im_end|>\\n' }}\n {%- elif loop.last %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- else %}\n {{- raise_exception('Unexpected message role.') }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n {%- if enable_thinking is defined and enable_thinking is false %}\n {{- '\\n\\n\\n\\n' }}\n {%- else %}\n {{- '\\n' }}\n {%- endif %}\n{%- endif %}\n{#- Unsloth fixes - developer role, merged system messages, tool calling #}","chat_template_caps":{"supports_object_arguments":true,"supports_parallel_tool_calls":true,"supports_preserve_reasoning":true,"supports_reasoning_effort":true,"supports_string_content":true,"supports_system_role":true,"supports_tool_calls":true,"supports_tools":true,"supports_typed_content":true},"bos_token":"<|endoftext|>","eos_token":"<|im_end|>","build_info":"b11260-e904318a2","is_sleeping":false,"cors_proxy_enabled":false} \ No newline at end of file diff --git a/tests/fixtures/llamacpp/get_v1_models.http b/tests/fixtures/llamacpp/get_v1_models.http new file mode 100644 index 00000000..3f978af3 --- /dev/null +++ b/tests/fixtures/llamacpp/get_v1_models.http @@ -0,0 +1,4 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "Content-Type": "application/json; charset=utf-8", "Content-Length": "667", "Keep-Alive": "timeout=5, max=100"} + +{"models":[{"name":"Qwen3.8-27B-UD-Q4_K_XL.gguf","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","modified_at":"","size":"","digest":"","type":"model","description":"","tags":[""],"capabilities":["completion"],"parameters":"","details":{"parent_model":"","format":"gguf","family":"","families":[""],"parameter_size":"","quantization_level":""}}],"object":"list","data":[{"id":"Qwen3.8-27B-UD-Q4_K_XL.gguf","aliases":["Qwen3.8-27B-UD-Q4_K_XL.gguf"],"tags":[],"object":"model","created":1790893916,"owned_by":"llamacpp","meta":{"vocab_type":2,"n_vocab":248320,"n_ctx":131072,"n_ctx_train":262144,"n_embd":5120,"n_params":27320697856,"size":17548181504,"ftype":"Q4_K - Medium"}}]} \ No newline at end of file diff --git a/tests/fixtures/llamacpp/stream_length_cutoff.http b/tests/fixtures/llamacpp/stream_length_cutoff.http new file mode 100644 index 00000000..cb0ae718 --- /dev/null +++ b/tests/fixtures/llamacpp/stream_length_cutoff.http @@ -0,0 +1,91 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "X-Accel-Buffering": "no", "Content-Type": "text/event-stream", "Transfer-Encoding": "chunked", "Connection": "close"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"role":"assistant","content":null}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"The"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" user"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" asking"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" me"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" to"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" explain"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" detail"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" how"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Ten"}}],"created":1790893765,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"eng"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"rad"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" focus"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" metric"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" works"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" This"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" knowledge"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" question"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" about"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" image"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" processing"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" and"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" no"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" tool"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" call"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" needed"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" I"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"'ll"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" write"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" thorough"}}],"created":1790893766,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" explanation"}}],"created":1790893767,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":"length","index":0,"delta":{}}],"created":1790893767,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[],"created":1790893767,"id":"chatcmpl-n9AMwrBpHl8x5wVvhfLgcbA7yCQsgLBl","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk","usage":{"completion_tokens":40,"prompt_tokens":33596,"total_tokens":33636,"prompt_tokens_details":{"cached_tokens":33574}},"timings":{"cache_n":33574,"prompt_n":22,"prompt_ms":182.684,"prompt_per_token_ms":8.303818181818182,"prompt_per_second":120.42652886952332,"predicted_n":40,"predicted_ms":1462.951,"predicted_per_token_ms":37.5115641025641,"predicted_per_second":26.658445839949525}} + +data: [DONE] + diff --git a/tests/fixtures/llamacpp/stream_text_reply.http b/tests/fixtures/llamacpp/stream_text_reply.http new file mode 100644 index 00000000..4eab4bdc --- /dev/null +++ b/tests/fixtures/llamacpp/stream_text_reply.http @@ -0,0 +1,593 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "X-Accel-Buffering": "no", "Content-Type": "text/event-stream", "Transfer-Encoding": "chunked", "Connection": "close"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"role":"assistant","content":null}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"The"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" user"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" asking"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" simple"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" factual"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" question"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":":"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" \""}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"In"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" one"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" sentence"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" what"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" N"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"DT"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"?\""}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"\n\n"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"ND"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"T"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" ="}}],"created":1790893754,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" ND"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"2"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"'s"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" t"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"?"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Actually"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" N"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"DT"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" refers"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" to"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" N"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"DT"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" format"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" used"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" by"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" p"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"yc"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"ro"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-manager"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" for"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" saving"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" acquisition"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" datasets"}}],"created":1790893755,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" It"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"'s"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" multi"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-dimensional"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" ("}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"n"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-dimensional"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":")"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" TIFF"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" format"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" —"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" variant"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" of"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" TIFF"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" format"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" that"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" can"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" store"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" metadata"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" and"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" multiple"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" axes"}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" ("}}],"created":1790893756,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"time"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" channel"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Z"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" position"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":")"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" structured"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" way"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" written"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" by"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" p"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"yc"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"ro"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-manager"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" It"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" can"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" be"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" opened"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" directly"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Fiji"}}],"created":1790893757,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"/Image"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"J"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" and"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" other"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" programs"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" that"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" support"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" TIFF"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"s"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"\n\n"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"Following"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" guidelines"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":":"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" \""}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"Ac"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"quisition"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" outputs"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" are"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" p"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"yc"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"ro"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-manager"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" N"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"DT"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893758,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" datasets"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" N"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"DT"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" can"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" be"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" opened"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" directly"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Fiji"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"/Image"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"J"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" or"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" any"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" other"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" program"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" that"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" supports"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" TIFF"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"s"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":".\""}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"\n\n"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"So"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" one"}}],"created":1790893759,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" sentence"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":":"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" N"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"DT"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"iff"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" n"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-dimensional"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" TIFF"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" format"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" that"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" p"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"yc"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"ro"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-manager"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" uses"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" to"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" save"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" acquisition"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" datasets"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" and"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" it"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" stores"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" all"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893760,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" axes"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" and"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" metadata"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" of"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" multi"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"-dimensional"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" acquisition"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" form"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" that"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" can"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" be"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" opened"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" directly"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Fiji"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"/Image"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"J"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" or"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" any"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" other"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" program"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" that"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" supports"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" TIFF"}}],"created":1790893761,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"s"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"\n\n"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"This"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" knowledge"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" question"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" so"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" no"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" tool"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" call"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" is"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" needed"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Keep"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" it"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" to"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" one"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" sentence"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"\n"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"ND"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"T"}}],"created":1790893762,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"iff"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" is"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" the"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" n"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"-dimensional"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" TIFF"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" format"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" p"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"yc"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"ro"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"-manager"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" uses"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" to"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" save"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" acquisition"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" datasets"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":","}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" encoding"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" the"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" full"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" multi"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"-dimensional"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" structure"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" ("}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"time"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":","}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" channel"}}],"created":1790893763,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":","}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" Z"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":","}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" position"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":")"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" and"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" its"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" metadata"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" in"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" a"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" way"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" that"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" opens"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" directly"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" in"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" Fiji"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"/Image"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"J"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" and"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" any"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" other"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" TIFF"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"-cap"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"able"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":" program"}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"."}}],"created":1790893764,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":"stop","index":0,"delta":{}}],"created":1790893765,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[],"created":1790893765,"id":"chatcmpl-qXEvURvvF36kDnKUqQs1kowW8lYVtta8","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk","usage":{"completion_tokens":294,"prompt_tokens":33594,"total_tokens":33888,"prompt_tokens_details":{"cached_tokens":33574}},"timings":{"cache_n":33574,"prompt_n":20,"prompt_ms":174.428,"prompt_per_token_ms":8.7214,"prompt_per_second":114.66049028825648,"predicted_n":294,"predicted_ms":10914.734,"predicted_per_token_ms":37.251651877133106,"predicted_per_second":26.84444714823101}} + +data: [DONE] + diff --git a/tests/fixtures/llamacpp/stream_tool_calls.http b/tests/fixtures/llamacpp/stream_tool_calls.http new file mode 100644 index 00000000..32f0d264 --- /dev/null +++ b/tests/fixtures/llamacpp/stream_tool_calls.http @@ -0,0 +1,105 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "X-Accel-Buffering": "no", "Content-Type": "text/event-stream", "Transfer-Encoding": "chunked", "Connection": "close"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"role":"assistant","content":null}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"The"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" user"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" wants"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" me"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" to"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" set"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" exposure"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" to"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" "}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"5"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"0"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" ms"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" and"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" tell"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" them"}}],"created":1790893750,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" the"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" current"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" Z"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" position"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" These"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" are"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" two"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" independent"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" tool"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" calls"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":","}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" so"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" I"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" can"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" run"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" them"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" together"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" in"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" a"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" single"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":" block"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"."}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"reasoning_content":"\n"}}],"created":1790893751,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"id":"ipQX5BPZX3xG7RuJKJYFu18DrcORALAD","type":"function","function":{"name":"set_exposure","arguments":"{"}}]}}],"created":1790893752,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"ms\":"}}]}}],"created":1790893752,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"5"}}]}}],"created":1790893752,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"0"}}]}}],"created":1790893752,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]}}],"created":1790893752,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":1,"id":"AnLs73TL6WYtAeYSb0KB1R5MVeLjb1Ju","type":"function","function":{"name":"get_z_position","arguments":"{"}}]}}],"created":1790893753,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":1,"function":{"arguments":"}"}}]}}],"created":1790893753,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[{"finish_reason":"tool_calls","index":0,"delta":{}}],"created":1790893753,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk"} + +data: {"choices":[],"created":1790893753,"id":"chatcmpl-khDKH3cse6v0a2uH7Svv7uR5rA0mevzn","model":"Qwen3.8-27B-UD-Q4_K_XL.gguf","system_fingerprint":"b11260-e904318a2","object":"chat.completion.chunk","usage":{"completion_tokens":85,"prompt_tokens":33600,"total_tokens":33685,"prompt_tokens_details":{"cached_tokens":33574}},"timings":{"cache_n":33574,"prompt_n":26,"prompt_ms":426.34,"prompt_per_token_ms":16.397692307692306,"prompt_per_second":60.984191021250645,"predicted_n":85,"predicted_ms":3118.368,"predicted_per_token_ms":37.12342857142857,"predicted_per_second":26.937167133577564}} + +data: [DONE] + From f625e289b662609f8166cea45c2276444a34d161 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 09:32:30 -0400 Subject: [PATCH 11/53] Add local OpenAI-compatible backend path --- microclaw/__main__.py | 14 +++- microclaw/agent.py | 129 +++++++++++++++++++++++++++++++++++- microclaw/credentials.py | 52 ++++++++++++++- microclaw/webserve.py | 18 +++-- tests/test_local_backend.py | 64 ++++++++++++++++++ 5 files changed, 265 insertions(+), 12 deletions(-) create mode 100644 tests/test_local_backend.py diff --git a/microclaw/__main__.py b/microclaw/__main__.py index 348c96ae..df2c3a72 100644 --- a/microclaw/__main__.py +++ b/microclaw/__main__.py @@ -137,8 +137,14 @@ def run_session(args): parsed_safety = load_safety_config_or_exit(args.safety_config) guard = SafetyGuard(parsed_safety.constraints) - key, source = credentials.load_api_key() - if key is None: + from microclaw.agent import llm_base_url + if llm_base_url(): + key, source = credentials.load_llm_api_key() + from microclaw.agent import set_llm_api_key + if key: set_llm_api_key(key) + else: + key, source = credentials.load_api_key() + if key is None and not llm_base_url(): sys.exit( "No Anthropic API key found. Looked in the ANTHROPIC_API_KEY " "environment variable, the system keyring, and the Microclaw credential file." @@ -465,6 +471,7 @@ def main(): help="Skip this launch's managed-install update check.", ) parser.add_argument("--port", type=int, default=4827) + parser.add_argument("--llm-base-url", default=None, help="OpenAI-compatible local server base URL (also MICROCLAW_LLM_BASE_URL).") parser.add_argument( "--setup-write-security-config", action="store_true", help="Permit one confirmed write of setup security bounds (serve on loopback only).", @@ -511,6 +518,7 @@ def main(): ), ) sv.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1).") + sv.add_argument("--llm-base-url", default=None, help="OpenAI-compatible local server base URL (also MICROCLAW_LLM_BASE_URL).") sv.add_argument("--web-port", type=int, default=8000, help="HTTP port (default: 8000).") sv.add_argument( "--no-browser", @@ -591,6 +599,8 @@ def main(): ) args = parser.parse_args() + if args.llm_base_url: + os.environ["MICROCLAW_LLM_BASE_URL"] = args.llm_base_url if args.setup_write_security_config and args.command != "serve": parser.error("--setup-write-security-config is valid only with serve") diff --git a/microclaw/agent.py b/microclaw/agent.py index 9c9e12f6..03d6514c 100644 --- a/microclaw/agent.py +++ b/microclaw/agent.py @@ -6,6 +6,7 @@ from datetime import datetime, timezone import os import time +import httpx import anthropic @@ -31,6 +32,27 @@ _client: anthropic.Anthropic | None = None _known_models: list[str] | None = None +LLM_BASE_URL_ENV = "MICROCLAW_LLM_BASE_URL" +LLM_API_KEY_ENV = "MICROCLAW_LLM_API_KEY" +_local_models: list[str] | None = None +_local_vision: dict[str, bool] = {} +_local_http = httpx.Client(timeout=120) + + +def llm_base_url() -> str | None: + return os.environ.get(LLM_BASE_URL_ENV, "").rstrip("/") or None + + +def set_llm_api_key(key: str) -> None: + os.environ[LLM_API_KEY_ENV] = key + global _local_models + _local_models = None + + +def _local_get(path: str): + url = llm_base_url() + headers = {"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} if os.environ.get(LLM_API_KEY_ENV) else {} + return _local_http.get(url + path, headers=headers, timeout=30) MAX_OUTPUT_TOKENS = 8192 @@ -60,7 +82,7 @@ def set_api_key(key: str) -> None: def resolve_model(model: str | None = None) -> str: """Pick the model: explicit arg > MICROCLAW_MODEL env > DEFAULT_MODEL.""" - return model or os.environ.get(MODEL_ENV) or DEFAULT_MODEL + return model or os.environ.get(MODEL_ENV) or (known_models()[0] if llm_base_url() and known_models() else DEFAULT_MODEL) def known_models() -> list[str]: @@ -70,7 +92,14 @@ def known_models() -> list[str]: populated must still be usable, so callers keep the field free-text. An unreachable API is not an error here — it just means no suggestions. """ - global _known_models + global _known_models, _local_models + if llm_base_url(): + if _local_models is None: + try: + _local_models = [m["id"] for m in _local_get("/v1/models").json().get("data", [])] + except Exception: + _local_models = [] + return _local_models if _known_models is None: try: _known_models = [m.id for m in _get_client().models.list(limit=100).data] @@ -721,6 +750,9 @@ def _stream_one_round(messages, system_blocks, model, tool_schemas, `_TransientAPIError` once the transient-failure backoff is spent. """ retry_after = 0.0 + if llm_base_url(): + return (yield from _stream_local_round(messages, system_blocks, model, tool_schemas, + context_provider, iteration, usage_sink)) for attempt, delay in enumerate([0, *_RETRY_DELAYS]): delay = max(delay, retry_after) if delay: @@ -802,6 +834,91 @@ def _stream_one_round(messages, system_blocks, model, tool_schemas, raise RuntimeError("unexpected loop exit") # pragma: no cover +def _stream_local_round(messages, system_blocks, model, tool_schemas, + context_provider, iteration, usage_sink): + base = llm_base_url() + headers = {"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} if os.environ.get(LLM_API_KEY_ENV) else {} + try: + if base not in _local_vision: + try: + _local_vision[base] = bool(_local_get("/props").json().get("modalities", {}).get("vision")) + except Exception: + _local_vision[base] = False + model_messages = context_provider(messages) if context_provider else messages + wire = [{"role": "system", "content": "\n\n".join( + b.get("text", "") for b in system_blocks if b.get("type") == "text") }] + for msg in model_messages: + role = msg["role"] + content = msg["content"] + if isinstance(content, str): + wire.append({"role": role, "content": content}); continue + if role == "assistant": + texts, calls = [], [] + for block in content: + typ = getattr(block, "type", None) or block.get("type") + get = (lambda k: getattr(block, k)) if not isinstance(block, dict) else block.get + if typ == "text": texts.append(get("text")) + elif typ == "tool_use": calls.append({"id": get("id"), "type": "function", "function": {"name": get("name"), "arguments": json.dumps(get("input"), separators=(",", ":"))}}) + entry = {"role": "assistant", "content": "".join(texts) or None} + if calls: entry["tool_calls"] = calls + wire.append(entry) + else: + for block in content: + if block.get("type") != "tool_result": continue + parts = block.get("content", "") + text = "\n".join(p.get("text", "") for p in parts if p.get("type") == "text") if isinstance(parts, list) else str(parts) + if isinstance(parts, list) and any(p.get("type") == "image" for p in parts): + if _local_vision[base]: + for p in parts: + if p.get("type") == "image": + src = p["source"]; wire.append({"role": "user", "content": [{"type": "image_url", "image_url": {"url": f"data:{src['media_type']};base64,{src['data']}"}}]}) + else: text += ('\n' if text else '') + '{"image_omitted": "this model does not accept images"}' + if block.get("is_error"): text = "Error: " + text + wire.append({"role": "tool", "tool_call_id": block["tool_use_id"], "content": text}) + payload = {"model": model, "messages": wire, "max_tokens": MAX_OUTPUT_TOKENS, "stream": True, "stream_options": {"include_usage": True}} + if tool_schemas: + payload["tools"] = [{"type": "function", "function": {"name": t["name"], "description": t.get("description", ""), "parameters": t["input_schema"]}} for t in tool_schemas] + response = _local_http.post(base + "/v1/chat/completions", json=payload, headers=headers, timeout=120) + if response.status_code >= 400: + try: err = response.json().get("error", {}) + except Exception: err = {} + if response.status_code == 404: raise _BadModel(f"Local server {base} does not know model {model!r}.") + if response.status_code == 400 and err.get("n_ctx"): + raise RuntimeError(f"Local server {base} context limit is {err['n_ctx']} tokens; request uses {err.get('n_prompt_tokens', 'an unknown number of')} tokens.") + if response.status_code in (429, 503): raise _TransientAPIError(f"Local server {base} is still unavailable after retrying.") + raise RuntimeError(f"Local server {base} returned HTTP {response.status_code}: {err.get('message', response.text)}") + text, calls, finish, usage, timings = [], {}, None, {}, {} + for line in response.text.splitlines(): + if not line.startswith("data: "): continue + data = line[6:] + if data == "[DONE]": continue + chunk = json.loads(data) + if not chunk.get("choices"): + usage = chunk.get("usage", {}); timings = chunk.get("timings", {}); continue + choice = chunk["choices"][0]; delta = choice.get("delta", {}) + if delta.get("content"): + text.append(delta["content"]); yield {"type": "text_delta", "text": delta["content"]} + for call in delta.get("tool_calls", []): + item = calls.setdefault(call["index"], {"id": "", "name": "", "arguments": ""}) + item["id"] += call.get("id", "") + fn = call.get("function", {}); item["name"] += fn.get("name", ""); item["arguments"] += fn.get("arguments", "") + if choice.get("finish_reason"): finish = choice["finish_reason"] + blocks = ([{"type": "text", "text": "".join(text)}] if text else []) + for item in calls.values(): + try: inp = json.loads(item["arguments"] or "{}") + except json.JSONDecodeError as exc: inp = {"_microclaw_parse_error": f"Invalid tool arguments JSON: {exc}", "raw": item["arguments"]} + blocks.append({"type": "tool_use", "id": item["id"], "name": item["name"], "input": inp}) + stop = {"stop": "end_turn", "tool_calls": "tool_use", "length": "max_tokens"}.get(finish, "end_turn") + result = anthropic.types.Message.model_validate({"id": "local", "type": "message", "role": "assistant", "model": model, "content": blocks, "stop_reason": stop, "stop_sequence": None, "usage": {"input_tokens": usage.get("prompt_tokens", 0), "output_tokens": usage.get("completion_tokens", 0), "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)}}) + if usage_sink: + record = {"timestamp": datetime.now(timezone.utc).isoformat(), "model": model, "iteration": iteration, "stop_reason": stop, "input_tokens": usage.get("prompt_tokens"), "output_tokens": usage.get("completion_tokens"), "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens")} + record.update({k: timings[k] for k in ("prompt_n", "cache_n", "prompt_ms", "predicted_n", "predicted_ms") if k in timings}) + usage_sink(record) + return result + except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException) as exc: + raise _TransientAPIError(f"Local server {base} is unavailable after retrying: {exc}") from exc + + def run_agent_iter( user_message: str, ctrl: MicroscopeController, @@ -933,6 +1050,14 @@ def append(message: dict) -> None: confirmation_start = ( len(confirmation_records) if confirmation_records is not None else 0 ) + if isinstance(block.input, dict) and "_microclaw_parse_error" in block.input: + result_json = json.dumps({"error": block.input["_microclaw_parse_error"]}) + result_block = {"type": "tool_result", "tool_use_id": block.id, + "is_error": True, "content": result_json} + tool_results.append(result_block) + yield {"type": "tool_result", "tool_use_id": block.id, + "content": result_json, "is_error": True} + continue result_json = execute_tool( block.name, block.input, ctrl, guard, tool_registry, setup_mode=setup_mode, diff --git a/microclaw/credentials.py b/microclaw/credentials.py index 5f110520..3f001cb9 100644 --- a/microclaw/credentials.py +++ b/microclaw/credentials.py @@ -30,6 +30,9 @@ _KEYRING_USERNAME = "anthropic-api-key" _TOML_KEY = "anthropic_api_key" +LLM_ENV_VAR = "MICROCLAW_LLM_API_KEY" +_LLM_KEYRING_USERNAME = "local-llm-api-key" +_LLM_TOML_KEY = "llm_api_key" def config_path() -> Path: @@ -53,6 +56,10 @@ def _keyring(): def _read_config_key() -> str | None: + return _read_toml_key(_TOML_KEY) + + +def _read_toml_key(name: str) -> str | None: path = config_path() try: text = path.read_text(encoding="utf-8") @@ -60,7 +67,7 @@ def _read_config_key() -> str | None: return None import tomllib try: - return tomllib.loads(text).get(_TOML_KEY) or None + return tomllib.loads(text).get(name) or None except Exception: return None @@ -111,9 +118,48 @@ def store_api_key(key: str) -> tuple[str, str | None]: path = config_path() path.parent.mkdir(parents=True, exist_ok=True) # json.dumps gives us a correctly-escaped TOML basic string. - path.write_text(f"{_TOML_KEY} = {json.dumps(key)}\n", encoding="utf-8") + _write_config_key(_TOML_KEY, key) + if os.name != "nt": + os.chmod(path, stat.S_IRUSR | stat.S_IWUSR) + return "file", str(path) + + +def _write_config_key(name: str, key: str) -> None: + path = config_path() + import tomllib + try: + values = tomllib.loads(path.read_text(encoding="utf-8")) + except Exception: + values = {} + values[name] = key + path.write_text("\n".join(f"{k} = {json.dumps(v)}" for k, v in values.items()) + "\n", encoding="utf-8") + + +def load_llm_api_key() -> tuple[str | None, str | None]: + env = os.environ.get(LLM_ENV_VAR) + if env: + return env, "env" + kr = _keyring() + if kr is not None: + try: + key = kr.get_password(_KEYRING_SERVICE, _LLM_KEYRING_USERNAME) + if key: return key, "keyring" + except Exception: pass + key = _read_toml_key(_LLM_TOML_KEY) + return (key, "file") if key else (None, None) + + +def store_llm_api_key(key: str) -> tuple[str, str | None]: + kr = _keyring() + if kr is not None: + try: + kr.set_password(_KEYRING_SERVICE, _LLM_KEYRING_USERNAME, key) + return "keyring", None + except Exception: pass + path = config_path() + path.parent.mkdir(parents=True, exist_ok=True) + _write_config_key(_LLM_TOML_KEY, key) if os.name != "nt": - # Meaningful on POSIX; a no-op worth skipping on Windows (see module docstring). os.chmod(path, stat.S_IRUSR | stat.S_IWUSR) return "file", str(path) diff --git a/microclaw/webserve.py b/microclaw/webserve.py index f55fb829..9c4811a5 100644 --- a/microclaw/webserve.py +++ b/microclaw/webserve.py @@ -61,6 +61,8 @@ resolve_model, run_agent_iter, set_api_key, + llm_base_url, + set_llm_api_key, ) from microclaw.assets import icon_bytes, load_page from microclaw.config import ConfigValidationResult, ParsedSafetyConfig @@ -383,13 +385,16 @@ def _initialize(self, args): # env > keyring > file; a key found in a store is pushed into the # environment now so the first turn doesn't have to look for it. - key, source = credentials.load_api_key() + key, source = (credentials.load_llm_api_key() if llm_base_url() + else credentials.load_api_key()) if key: _add_audit_secret(self, key) - set_api_key(key) - print(f"Anthropic API key: {credentials.mask(key)} (from {source})") + if llm_base_url(): set_llm_api_key(key) + else: set_api_key(key) + print(f"LLM API key: {credentials.mask(key)} (from {source})") else: - print("No Anthropic API key found — set one from the browser.") + if llm_base_url(): print("No local LLM API key set.") + else: print("No Anthropic API key found — set one from the browser.") def _audit_confirmation( @@ -1161,7 +1166,8 @@ async def post_prompt(p: Prompt, request: Request): msg = p.message.strip() if not msg: raise HTTPException(400, "Empty message.") - if credentials.load_api_key()[0] is None: + if ((credentials.load_llm_api_key()[0] if llm_base_url() else credentials.load_api_key()[0]) is None + and not llm_base_url()): raise HTTPException(400, "No Anthropic API key is set.") with update_job_lock: refuse_extension_install() @@ -1574,6 +1580,8 @@ def wait(): def serve(args): + if getattr(args, "llm_base_url", None): + os.environ["MICROCLAW_LLM_BASE_URL"] = args.llm_base_url """Entry point for the `serve` subcommand.""" if getattr(args, "setup_write_security_config", False): if args.host not in LOCAL_HOSTS: diff --git a/tests/test_local_backend.py b/tests/test_local_backend.py new file mode 100644 index 00000000..599a965b --- /dev/null +++ b/tests/test_local_backend.py @@ -0,0 +1,64 @@ +import json +from pathlib import Path + +import httpx + +from microclaw import agent + +FIXTURES = Path(__file__).parent / "fixtures" / "llamacpp" + + +def replay(name): + head, body = (FIXTURES / f"{name}.http").read_bytes().split(b"\n\n", 1) + status, headers = head.split(b"\n", 1) + code = int(status.split()[1]) + return httpx.Response(code, headers=json.loads(headers), content=body) + + +def install(monkeypatch, name): + def handler(request): + if request.url.path.endswith("/props"): + return replay("get_props") + if request.url.path.endswith("/models"): + return replay("get_v1_models") + return replay(name) + client = httpx.Client(transport=httpx.MockTransport(handler)) + monkeypatch.setattr(agent, "_local_http", client) + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://recorded-server") + monkeypatch.delenv("MICROCLAW_LLM_API_KEY", raising=False) + agent._local_vision.clear() + agent._local_models = None + + +def test_recorded_stream_tool_translation_and_usage(monkeypatch): + install(monkeypatch, "stream_tool_calls") + seen = [] + gen = agent._stream_one_round([], [{"type": "text", "text": "system"}], "model", [ + {"name": "a", "description": "A", "input_schema": {"type": "object"}} + ], usage_sink=seen.append) + events = [] + while True: + try: events.append(next(gen)) + except StopIteration as stop: + msg = stop.value; break + assert msg.stop_reason == "tool_use" + assert len([b for b in msg.content if b.type == "tool_use"]) == 2 + assert not any("The" == e.get("text") for e in events) + assert seen[0]["prompt_n"] == 26 and seen[0]["cache_n"] == 33574 + + +def test_recorded_text_and_length_finish_reasons(monkeypatch): + for fixture, expected in (("stream_text_reply", "end_turn"), ("stream_length_cutoff", "max_tokens")): + install(monkeypatch, fixture) + gen = agent._stream_one_round([], [], "model", []) + while True: + try: next(gen) + except StopIteration as stop: + assert stop.value.stop_reason == expected + break + + +def test_props_and_models_are_recorded(monkeypatch): + install(monkeypatch, "stream_text_reply") + assert agent.known_models() == ["Qwen3.8-27B-UD-Q4_K_XL.gguf"] + assert agent.resolve_model() == "Qwen3.8-27B-UD-Q4_K_XL.gguf" From 83fa42993a11a6da0d122112e2ab7d3580770d82 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 10:26:25 -0400 Subject: [PATCH 12/53] Fix local backend streaming and error handling --- microclaw/__main__.py | 14 +- microclaw/agent.py | 103 ++++++++++--- microclaw/conversation.py | 2 +- microclaw/tools.py | 7 +- microclaw/webserve.py | 18 ++- tests/test_conversation.py | 5 + tests/test_local_backend.py | 290 +++++++++++++++++++++++++++++++++++- tests/test_webserve.py | 13 ++ 8 files changed, 410 insertions(+), 42 deletions(-) diff --git a/microclaw/__main__.py b/microclaw/__main__.py index df2c3a72..0e18ea61 100644 --- a/microclaw/__main__.py +++ b/microclaw/__main__.py @@ -454,7 +454,14 @@ def _positive_seconds(value: str) -> float: def main(): - parser = argparse.ArgumentParser(description="Microclaw: AI agent for Micro-Manager") + llm_url_option = argparse.ArgumentParser(add_help=False) + llm_url_option.add_argument( + "--llm-base-url", default=argparse.SUPPRESS, + help="OpenAI-compatible local server base URL (also MICROCLAW_LLM_BASE_URL).", + ) + parser = argparse.ArgumentParser( + description="Microclaw: AI agent for Micro-Manager", parents=[llm_url_option] + ) # Agent-session options stay on the top-level parser so `microclaw --port ...` # (no subcommand) keeps launching a session, as before. parser.add_argument( @@ -471,7 +478,6 @@ def main(): help="Skip this launch's managed-install update check.", ) parser.add_argument("--port", type=int, default=4827) - parser.add_argument("--llm-base-url", default=None, help="OpenAI-compatible local server base URL (also MICROCLAW_LLM_BASE_URL).") parser.add_argument( "--setup-write-security-config", action="store_true", help="Permit one confirmed write of setup security bounds (serve on loopback only).", @@ -516,9 +522,9 @@ def main(): "Drive Microclaw from a browser. Binds 127.0.0.1 only — this " "endpoint moves real hardware." ), + parents=[llm_url_option], ) sv.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1).") - sv.add_argument("--llm-base-url", default=None, help="OpenAI-compatible local server base URL (also MICROCLAW_LLM_BASE_URL).") sv.add_argument("--web-port", type=int, default=8000, help="HTTP port (default: 8000).") sv.add_argument( "--no-browser", @@ -599,7 +605,7 @@ def main(): ) args = parser.parse_args() - if args.llm_base_url: + if getattr(args, "llm_base_url", None): os.environ["MICROCLAW_LLM_BASE_URL"] = args.llm_base_url if args.setup_write_security_config and args.command != "serve": diff --git a/microclaw/agent.py b/microclaw/agent.py index 03d6514c..3e7c2612 100644 --- a/microclaw/agent.py +++ b/microclaw/agent.py @@ -36,7 +36,14 @@ LLM_API_KEY_ENV = "MICROCLAW_LLM_API_KEY" _local_models: list[str] | None = None _local_vision: dict[str, bool] = {} -_local_http = httpx.Client(timeout=120) +_local_http: httpx.Client | None = None + + +def _get_local_http() -> httpx.Client: + global _local_http + if _local_http is None: + _local_http = httpx.Client() + return _local_http def llm_base_url() -> str | None: @@ -52,7 +59,7 @@ def set_llm_api_key(key: str) -> None: def _local_get(path: str): url = llm_base_url() headers = {"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} if os.environ.get(LLM_API_KEY_ENV) else {} - return _local_http.get(url + path, headers=headers, timeout=30) + return _get_local_http().get(url + path, headers=headers, timeout=30) MAX_OUTPUT_TOKENS = 8192 @@ -648,6 +655,16 @@ class _BadModel(Exception): """The API does not know this model id. Internal to this module.""" +class _LocalAPIError(Exception): + """A non-retryable local server response suitable for a user error event.""" + + +class _LocalTransient(Exception): + def __init__(self, message: str, retry_after: float = 0): + super().__init__(message) + self.retry_after = retry_after + + def _cancelled(cancel) -> bool: return cancel is not None and cancel.is_set() @@ -750,15 +767,15 @@ def _stream_one_round(messages, system_blocks, model, tool_schemas, `_TransientAPIError` once the transient-failure backoff is spent. """ retry_after = 0.0 - if llm_base_url(): - return (yield from _stream_local_round(messages, system_blocks, model, tool_schemas, - context_provider, iteration, usage_sink)) for attempt, delay in enumerate([0, *_RETRY_DELAYS]): delay = max(delay, retry_after) if delay: yield {"type": "retry", "delay": delay, "attempt": attempt} time.sleep(delay) try: + if llm_base_url(): + return (yield from _stream_local_round(messages, system_blocks, model, tool_schemas, + context_provider, iteration, usage_sink)) model_messages = ( context_provider(messages) if context_provider is not None else messages ) @@ -813,6 +830,23 @@ def _stream_one_round(messages, system_blocks, model, tool_schemas, # A free-text model picker (v4c) can hold an id the API rejects. # Without this it escapes run_agent_iter as a 500 on the SSE stream. raise _BadModel(str(e)) from None + except _LocalTransient as e: + if attempt == len(_RETRY_DELAYS): + raise _TransientAPIError(str(e)) from e + retry_after = e.retry_after + if retry_after > MAX_RETRY_AFTER_SECONDS: + raise _TransientAPIError( + f"Local server {llm_base_url()} requested a {retry_after:g} second retry delay, " + "which is too long to hold this turn open. Please try again later." + ) from e + except _LocalAPIError: + raise + except httpx.TransportError as e: + if attempt == len(_RETRY_DELAYS): + raise _TransientAPIError( + f"Local server {llm_base_url()} is still unavailable after retrying." + ) from e + retry_after = 0.0 except (anthropic.RateLimitError, anthropic.InternalServerError, anthropic._exceptions.OverloadedError, anthropic.APIConnectionError) as e: @@ -855,40 +889,58 @@ def _stream_local_round(messages, system_blocks, model, tool_schemas, if role == "assistant": texts, calls = [], [] for block in content: - typ = getattr(block, "type", None) or block.get("type") + typ = getattr(block, "type", None) or (block.get("type") if isinstance(block, dict) else None) get = (lambda k: getattr(block, k)) if not isinstance(block, dict) else block.get if typ == "text": texts.append(get("text")) elif typ == "tool_use": calls.append({"id": get("id"), "type": "function", "function": {"name": get("name"), "arguments": json.dumps(get("input"), separators=(",", ":"))}}) entry = {"role": "assistant", "content": "".join(texts) or None} if calls: entry["tool_calls"] = calls wire.append(entry) - else: + elif role == "user": + images = [] for block in content: - if block.get("type") != "tool_result": continue + if block.get("type") == "text": + wire.append({"role": "user", "content": block.get("text", "")}) + continue + if block.get("type") != "tool_result": + raise ValueError(f"Cannot translate user content block {block.get('type')!r} to the local chat API") parts = block.get("content", "") text = "\n".join(p.get("text", "") for p in parts if p.get("type") == "text") if isinstance(parts, list) else str(parts) if isinstance(parts, list) and any(p.get("type") == "image" for p in parts): if _local_vision[base]: for p in parts: if p.get("type") == "image": - src = p["source"]; wire.append({"role": "user", "content": [{"type": "image_url", "image_url": {"url": f"data:{src['media_type']};base64,{src['data']}"}}]}) + src = p["source"]; images.append({"type": "image_url", "image_url": {"url": f"data:{src['media_type']};base64,{src['data']}"}}) else: text += ('\n' if text else '') + '{"image_omitted": "this model does not accept images"}' if block.get("is_error"): text = "Error: " + text wire.append({"role": "tool", "tool_call_id": block["tool_use_id"], "content": text}) + if images: + wire.append({"role": "user", "content": images}) + else: + raise ValueError(f"Cannot translate message role {role!r} to the local chat API") payload = {"model": model, "messages": wire, "max_tokens": MAX_OUTPUT_TOKENS, "stream": True, "stream_options": {"include_usage": True}} if tool_schemas: payload["tools"] = [{"type": "function", "function": {"name": t["name"], "description": t.get("description", ""), "parameters": t["input_schema"]}} for t in tool_schemas] - response = _local_http.post(base + "/v1/chat/completions", json=payload, headers=headers, timeout=120) - if response.status_code >= 400: + text, calls, finish, usage, timings = [], {}, None, {}, {} + reason_length = 0 + timeout = httpx.Timeout(connect=5.0, read=None, write=30.0, pool=5.0) + with _get_local_http().stream("POST", base + "/v1/chat/completions", json=payload, + headers=headers, timeout=timeout) as response: + if response.status_code >= 400: + response.read() try: err = response.json().get("error", {}) except Exception: err = {} if response.status_code == 404: raise _BadModel(f"Local server {base} does not know model {model!r}.") + if response.status_code in (429, 503): + try: retry_after = float(response.headers.get("retry-after", 0)) + except ValueError: retry_after = 0 + raise _LocalTransient(f"Local server {base} is still unavailable after retrying.", retry_after) if response.status_code == 400 and err.get("n_ctx"): - raise RuntimeError(f"Local server {base} context limit is {err['n_ctx']} tokens; request uses {err.get('n_prompt_tokens', 'an unknown number of')} tokens.") - if response.status_code in (429, 503): raise _TransientAPIError(f"Local server {base} is still unavailable after retrying.") - raise RuntimeError(f"Local server {base} returned HTTP {response.status_code}: {err.get('message', response.text)}") - text, calls, finish, usage, timings = [], {}, None, {}, {} - for line in response.text.splitlines(): + raise _LocalAPIError(f"Local server {base} context limit is {err['n_ctx']} tokens; request uses {err.get('n_prompt_tokens', 'an unknown number of')} tokens.") + if response.status_code == 401: + raise _LocalAPIError(f"Local server {base} rejected its API key. Set {LLM_API_KEY_ENV} or enter the local key in the browser.") + raise _LocalAPIError(f"Local server {base} returned HTTP {response.status_code}: {err.get('message', response.read().decode(errors='replace'))}") + for line in response.iter_lines(): if not line.startswith("data: "): continue data = line[6:] if data == "[DONE]": continue @@ -896,6 +948,7 @@ def _stream_local_round(messages, system_blocks, model, tool_schemas, if not chunk.get("choices"): usage = chunk.get("usage", {}); timings = chunk.get("timings", {}); continue choice = chunk["choices"][0]; delta = choice.get("delta", {}) + reason_length += len(delta.get("reasoning_content") or "") if delta.get("content"): text.append(delta["content"]); yield {"type": "text_delta", "text": delta["content"]} for call in delta.get("tool_calls", []): @@ -905,18 +958,25 @@ def _stream_local_round(messages, system_blocks, model, tool_schemas, if choice.get("finish_reason"): finish = choice["finish_reason"] blocks = ([{"type": "text", "text": "".join(text)}] if text else []) for item in calls.values(): + if not item["id"]: + import uuid + item["id"] = "call_" + uuid.uuid4().hex try: inp = json.loads(item["arguments"] or "{}") except json.JSONDecodeError as exc: inp = {"_microclaw_parse_error": f"Invalid tool arguments JSON: {exc}", "raw": item["arguments"]} blocks.append({"type": "tool_use", "id": item["id"], "name": item["name"], "input": inp}) - stop = {"stop": "end_turn", "tool_calls": "tool_use", "length": "max_tokens"}.get(finish, "end_turn") + stop = {"stop": "end_turn", "tool_calls": "tool_use", "length": "max_tokens"}.get(finish) result = anthropic.types.Message.model_validate({"id": "local", "type": "message", "role": "assistant", "model": model, "content": blocks, "stop_reason": stop, "stop_sequence": None, "usage": {"input_tokens": usage.get("prompt_tokens", 0), "output_tokens": usage.get("completion_tokens", 0), "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)}}) if usage_sink: record = {"timestamp": datetime.now(timezone.utc).isoformat(), "model": model, "iteration": iteration, "stop_reason": stop, "input_tokens": usage.get("prompt_tokens"), "output_tokens": usage.get("completion_tokens"), "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens")} record.update({k: timings[k] for k in ("prompt_n", "cache_n", "prompt_ms", "predicted_n", "predicted_ms") if k in timings}) - usage_sink(record) + record["reasoning_characters"] = reason_length + try: + usage_sink(record) + except Exception as exc: + print(f"[microclaw] Could not record usage: {exc}", file=sys.stderr) return result - except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException) as exc: - raise _TransientAPIError(f"Local server {base} is unavailable after retrying: {exc}") from exc + except httpx.TransportError: + raise def run_agent_iter( @@ -995,6 +1055,9 @@ def append(message: dict) -> None: yield {"type": "error", "message": f"The API rejected the model '{model}': {e}"} return + if isinstance(e, _LocalAPIError): + yield {"type": "error", "message": str(e)} + return raise append({"role": "assistant", "content": response.content}) diff --git a/microclaw/conversation.py b/microclaw/conversation.py index b9142290..2a12d0e5 100644 --- a/microclaw/conversation.py +++ b/microclaw/conversation.py @@ -33,7 +33,7 @@ _API_KEY_RE = re.compile(r"\bsk-ant-[A-Za-z0-9_-]{12,}\b") _BEARER_RE = re.compile(r"(?i)(bearer\s+)[A-Za-z0-9._~+/=-]{12,}") _SECRET_KEYS = { - "api_key", "anthropic_api_key", "authorization", "bearer_token", + "api_key", "anthropic_api_key", "llm_api_key", "authorization", "bearer_token", "remote_token", "pairing_code", "session_token", } diff --git a/microclaw/tools.py b/microclaw/tools.py index 02ec86b3..da680ac4 100644 --- a/microclaw/tools.py +++ b/microclaw/tools.py @@ -1049,7 +1049,12 @@ def _recorded_tool_calls(records: list[dict]) -> list[tuple[str, RecordedParams] name = block.get("name") if isinstance(block, dict) else block.name params = block.get("input", {}) if isinstance(block, dict) else block.input block_id = block.get("id") if isinstance(block, dict) else block.id - recorded_params = RecordedParams(dict(params), results.get(str(block_id))) + recorded_result = results.get(str(block_id)) + if isinstance(params, dict) and "_microclaw_parse_error" in params: + # Balance the invalid tool_use in history, then report it as a + # skipped failed call. Never pass its raw arguments to an emitter. + recorded_result = {"error": params["_microclaw_parse_error"]} + recorded_params = RecordedParams(dict(params), recorded_result) recorded_params["_tool_use_id"] = str(block_id) calls.append((str(name), recorded_params)) return calls diff --git a/microclaw/webserve.py b/microclaw/webserve.py index 9c4811a5..70a60260 100644 --- a/microclaw/webserve.py +++ b/microclaw/webserve.py @@ -1166,8 +1166,7 @@ async def post_prompt(p: Prompt, request: Request): msg = p.message.strip() if not msg: raise HTTPException(400, "Empty message.") - if ((credentials.load_llm_api_key()[0] if llm_base_url() else credentials.load_api_key()[0]) is None - and not llm_base_url()): + if not llm_base_url() and credentials.load_api_key()[0] is None: raise HTTPException(400, "No Anthropic API key is set.") with update_job_lock: refuse_extension_install() @@ -1496,7 +1495,8 @@ async def get_artifact(path: str): @app.get("/api/key") async def get_key(): - key, source = credentials.load_api_key() + key, source = (credentials.load_llm_api_key() if llm_base_url() + else credentials.load_api_key()) return JSONResponse( { "has_key": key is not None, @@ -1517,15 +1517,19 @@ async def post_key(k: Key): raise HTTPException(400, "Empty key.") # Overwrites whatever was set, so a key can be swapped mid-session. _add_audit_secret(session, key) - set_api_key(key) + local_path = bool(llm_base_url()) + store = credentials.store_llm_api_key if local_path else credentials.store_api_key + load_stored = credentials.load_llm_api_key if local_path else credentials.load_stored_key + if local_path: set_llm_api_key(key) + else: set_api_key(key) stored_in = stored_at = None stale_store = False if k.persist: - stored_in, stored_at = credentials.store_api_key(key) + stored_in, stored_at = store(key) else: # A key held for this process only doesn't displace an older one in # the credential store — that one comes back on the next start. - stored, _ = credentials.load_stored_key() + stored, _ = load_stored() stale_store = stored is not None and stored != key # Never echo the key: a suffix is enough to confirm which one is set. return JSONResponse( @@ -1580,8 +1584,6 @@ def wait(): def serve(args): - if getattr(args, "llm_base_url", None): - os.environ["MICROCLAW_LLM_BASE_URL"] = args.llm_base_url """Entry point for the `serve` subcommand.""" if getattr(args, "setup_write_security_config", False): if args.host not in LOCAL_HOSTS: diff --git a/tests/test_conversation.py b/tests/test_conversation.py index a02392fe..15954c0f 100644 --- a/tests/test_conversation.py +++ b/tests/test_conversation.py @@ -75,6 +75,11 @@ def test_audit_redacts_credentials_but_keeps_images_and_scientific_values(tmp_pa assert "IMAGEBYTES" in raw +def test_llm_api_key_field_is_redacted(): + from microclaw.conversation import redact_credentials + assert redact_credentials({"llm_api_key": "local-secret-value"})["llm_api_key"] == "[REDACTED]" + + def test_atomic_write_interruption_leaves_previous_file_intact(tmp_path, monkeypatch): path = tmp_path / "history.json" path.write_text("previous", encoding="utf-8") diff --git a/tests/test_local_backend.py b/tests/test_local_backend.py index 599a965b..168ce654 100644 --- a/tests/test_local_backend.py +++ b/tests/test_local_backend.py @@ -1,9 +1,11 @@ import json from pathlib import Path +from types import SimpleNamespace import httpx -from microclaw import agent +import anthropic +from microclaw import agent, conversation, tools FIXTURES = Path(__file__).parent / "fixtures" / "llamacpp" @@ -15,23 +17,51 @@ def replay(name): return httpx.Response(code, headers=json.loads(headers), content=body) -def install(monkeypatch, name): +def install(monkeypatch, name, *, extra=None, stream_chunks=False, vision=False): + class Captured(list): + timeouts = [] + captured = Captured() + remaining = list(name if isinstance(name, list) else [name]) def handler(request): + if request.url.path.endswith("chat/completions"): + captured.append(json.loads(request.content)) + captured.timeouts.append(request.extensions.get("timeout")) + selected = remaining.pop(0) + if isinstance(selected, httpx.Response): return selected + response = replay(selected) + if extra: response = extra(response) + if stream_chunks: + body = response.content + class Chunks(httpx.SyncByteStream): + exhausted = False + def __iter__(self): + for offset in range(0, len(body), 2048): + yield body[offset:offset + 2048] + self.exhausted = True + stream = Chunks() + captured.append(stream) + return httpx.Response(response.status_code, headers=response.headers, stream=stream) + return response if request.url.path.endswith("/props"): - return replay("get_props") + prop = replay("get_props") + if vision: + prop = httpx.Response(prop.status_code, headers=prop.headers, + content=prop.content.replace(b'"vision":false', b'"vision":true')) + return prop if request.url.path.endswith("/models"): return replay("get_v1_models") - return replay(name) + return replay("stream_text_reply") client = httpx.Client(transport=httpx.MockTransport(handler)) monkeypatch.setattr(agent, "_local_http", client) monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://recorded-server") monkeypatch.delenv("MICROCLAW_LLM_API_KEY", raising=False) agent._local_vision.clear() agent._local_models = None + return captured def test_recorded_stream_tool_translation_and_usage(monkeypatch): - install(monkeypatch, "stream_tool_calls") + captured = install(monkeypatch, "stream_tool_calls") seen = [] gen = agent._stream_one_round([], [{"type": "text", "text": "system"}], "model", [ {"name": "a", "description": "A", "input_schema": {"type": "object"}} @@ -42,19 +72,33 @@ def test_recorded_stream_tool_translation_and_usage(monkeypatch): except StopIteration as stop: msg = stop.value; break assert msg.stop_reason == "tool_use" - assert len([b for b in msg.content if b.type == "tool_use"]) == 2 - assert not any("The" == e.get("text") for e in events) + tools = [b for b in msg.content if b.type == "tool_use"] + assert [(b.name, b.id, b.input) for b in tools] == [ + ("set_exposure", "ipQX5BPZX3xG7RuJKJYFu18DrcORALAD", {"ms": 50}), + ("get_z_position", "AnLs73TL6WYtAeYSb0KB1R5MVeLjb1Ju", {}), + ] + reasoning = [e["text"] for e in events if e.get("type") == "text_delta"] + assert "".join(reasoning) == "" + assert not any("reasoning" in str(b.model_dump()) for b in msg.content) + assert captured[0]["messages"][0] == {"role": "system", "content": "system"} assert seen[0]["prompt_n"] == 26 and seen[0]["cache_n"] == 33574 + assert seen[0]["reasoning_characters"] > 0 def test_recorded_text_and_length_finish_reasons(monkeypatch): for fixture, expected in (("stream_text_reply", "end_turn"), ("stream_length_cutoff", "max_tokens")): install(monkeypatch, fixture) gen = agent._stream_one_round([], [], "model", []) + deltas = [] while True: - try: next(gen) + try: + event = next(gen) + if event["type"] == "text_delta": deltas.append(event["text"]) except StopIteration as stop: assert stop.value.stop_reason == expected + if fixture == "stream_text_reply": + content = "".join(b.text for b in stop.value.content if b.type == "text") + assert content and "".join(deltas) == content break @@ -62,3 +106,233 @@ def test_props_and_models_are_recorded(monkeypatch): install(monkeypatch, "stream_text_reply") assert agent.known_models() == ["Qwen3.8-27B-UD-Q4_K_XL.gguf"] assert agent.resolve_model() == "Qwen3.8-27B-UD-Q4_K_XL.gguf" + + +def test_request_translation_includes_checkpoint_and_all_d2_rows(monkeypatch): + captured = install(monkeypatch, "stream_text_reply") + store = conversation.ConversationStore(conversation.AuditLog(None, enabled=False), + high_water_tokens=300, low_water_tokens=100) + history = [] + for index in range(8): + history.extend([ + {"role": "user", "content": f"remember checkpoint {index} " + "x" * 300}, + {"role": "assistant", "content": [{"type": "text", "text": "reply " + "y" * 100}]}, + ]) + messages = store.model_messages(history) + assert store.compaction_count == 1 and messages[0]["content"][0]["text"].startswith("Conversation checkpoint:") + messages.extend([{ "role": "assistant", "content": [ + {"type": "text", "text": "working"}, + {"type": "tool_use", "id": "a", "name": "alpha", "input": {"x": 1}}, + {"type": "tool_use", "id": "b", "name": "beta", "input": {}}, + ]}, {"role": "user", "content": [ + {"type": "tool_result", "tool_use_id": "a", "is_error": True, "content": "bad"}, + {"type": "tool_result", "tool_use_id": "b", "content": [ + {"type": "text", "text": "ok"}, + {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "AA=="}}, + ]}, + ]}]) + schemas = [{"name": "alpha", "description": "A", "input_schema": {"type": "object"}, "cache_control": {"type": "ephemeral"}}] + gen = agent._stream_one_round(messages, [{"type": "text", "text": "sys1", "cache_control": {"type": "ephemeral"}}, {"type": "text", "text": "sys2"}], "m", schemas) + for _ in gen: pass + body = captured[0] + assert body["messages"][0] == {"role": "system", "content": "sys1\n\nsys2"} + assert body["messages"][1]["content"].startswith("Conversation checkpoint:") + assistant = next(m for m in body["messages"] if m.get("tool_calls")) + assert assistant == {"role": "assistant", "content": "working", "tool_calls": [ + {"id": "a", "type": "function", "function": {"name": "alpha", "arguments": '{"x":1}'}}, + {"id": "b", "type": "function", "function": {"name": "beta", "arguments": "{}"}}, + ]} + tools_wire = [m for m in body["messages"] if m["role"] == "tool"] + assert tools_wire[-2]["content"] == "Error: bad" + assert tools_wire[-1]["content"].endswith('{"image_omitted": "this model does not accept images"}') + assert body["tools"][0]["function"]["parameters"] == {"type": "object"} + assert "cache_control" not in str(body) + + +def test_local_stream_yields_before_stream_exhaustion(monkeypatch): + captured = install(monkeypatch, "stream_text_reply", stream_chunks=True) + gen = agent._stream_one_round([], [], "m", []) + event = next(gen) + assert event["type"] == "text_delta" + assert captured[1].exhausted is False + assert captured.timeouts[0]["connect"] == 5.0 + assert captured.timeouts[0]["read"] is None + gen.close() + + +def test_retry_after_503_then_tool_calls(monkeypatch): + response = httpx.Response(503, headers={"retry-after": "1"}, json={"error": {"message": "busy"}}) + install(monkeypatch, [response, "stream_tool_calls"]) + monkeypatch.setattr(agent.time, "sleep", lambda _: None) + gen = agent._stream_one_round([], [], "m", []) + assert next(gen) == {"type": "retry", "delay": 5, "attempt": 1} + while True: + try: next(gen) + except StopIteration as stop: + assert stop.value.stop_reason == "tool_use" + break + + +def test_429_retry_after_over_cap_returns_local_error(monkeypatch): + response = httpx.Response(429, headers={"retry-after": "61"}, json={"error": {"message": "busy"}}) + install(monkeypatch, response) + monkeypatch.setattr(agent.time, "sleep", lambda _: None) + from microclaw.agent import run_agent_iter + events = list(run_agent_iter("hello", None, None, [], model="m", tool_schemas=[])) + assert events[-1]["type"] == "error" + assert "http://recorded-server" in events[-1]["message"] + + +def test_vision_images_follow_all_tool_messages(monkeypatch): + # /props is minimally edited from the recorded text-only response: vision false → true. + captured = install(monkeypatch, "stream_text_reply", vision=True) + messages = [ + {"role": "assistant", "content": [ + {"type": "tool_use", "id": "a", "name": "alpha", "input": {}}, + {"type": "tool_use", "id": "b", "name": "beta", "input": {}}, + ]}, + {"role": "user", "content": [ + {"type": "tool_result", "tool_use_id": "a", "content": "ok"}, + {"type": "tool_result", "tool_use_id": "b", "content": [ + {"type": "text", "text": "image result"}, + {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "AA=="}}, + ]}, + ]}, + ] + for _ in agent._stream_one_round(messages, [], "m", []): pass + wire = captured[0]["messages"] + assert [m["role"] for m in wire[-3:]] == ["tool", "tool", "user"] + assert wire[-1]["content"] == [{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}}] + + +def test_empty_ids_are_generated_and_missing_finish_is_not_end_turn(monkeypatch): + body = replay("stream_tool_calls").content + body = body.replace(b"ipQX5BPZX3xG7RuJKJYFu18DrcORALAD", b"") + body = body.replace(b"AnLs73TL6WYtAeYSb0KB1R5MVeLjb1Ju", b"") + body = b"\n".join(line for line in body.split(b"\n") if b'"finish_reason":"tool_calls"' not in line) + edited = httpx.Response(200, headers={"content-type": "text/event-stream"}, content=body) + install(monkeypatch, edited) + gen = agent._stream_one_round([], [], "m", []) + while True: + try: next(gen) + except StopIteration as stop: + msg = stop.value + break + ids = [b.id for b in msg.content if b.type == "tool_use"] + assert len(ids) == 2 and len(set(ids)) == 2 and all(ids) + assert msg.stop_reason is None + install(monkeypatch, edited) + from microclaw.agent import run_agent_iter + events = list(run_agent_iter("hello", None, None, [], model="m", tool_schemas=[])) + assert events[-1]["type"] == "error" and "Unexpected stop reason: None" in events[-1]["message"] + + +def test_local_http_errors_become_agent_error_events(monkeypatch): + from microclaw.agent import run_agent_iter + for name, excerpt in (("error_context_exceeded", "131072"), ("error_bad_key", "MICROCLAW_LLM_API_KEY")): + install(monkeypatch, replay(name)) + history = [] + events = list(run_agent_iter("hello", None, None, history, model="m", tool_schemas=[])) + assert events[-1]["type"] == "error" and excerpt in events[-1]["message"] + if name == "error_context_exceeded": + assert "140052" in events[-1]["message"] + assert history == [] + + +def test_serve_accepts_llm_url_before_or_after_subcommand(monkeypatch): + from microclaw import __main__ as cli, webserve + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "") + seen = [] + monkeypatch.setattr(webserve, "serve", lambda args: seen.append(args.llm_base_url)) + monkeypatch.setattr("microclaw.shortcut.pause_on_exit", lambda: None) + for argv in ( + ["microclaw", "--llm-base-url", "http://one", "serve"], + ["microclaw", "serve", "--llm-base-url", "http://two"], + ): + monkeypatch.setattr("sys.argv", argv) + cli.main() + assert seen == ["http://one", "http://two"] + + +def test_local_http_client_is_lazy(monkeypatch): + monkeypatch.setattr(agent, "_local_http", None) + assert agent._local_http is None + client = agent._get_local_http() + assert isinstance(client, httpx.Client) + client.close() + monkeypatch.setattr(agent, "_local_http", None) + + +def test_export_is_identical_for_anthropic_and_local_replies(monkeypatch, tmp_path): + install(monkeypatch, "stream_tool_calls") + local_gen = agent._stream_one_round([], [], "m", []) + while True: + try: next(local_gen) + except StopIteration as stop: + local_message = stop.value + break + response = anthropic.types.Message.model_validate({ + "id": "fake", "type": "message", "role": "assistant", "model": "m", + "content": [ + {"type": "tool_use", "id": "ipQX5BPZX3xG7RuJKJYFu18DrcORALAD", "name": "set_exposure", "input": {"ms": 50}}, + {"type": "tool_use", "id": "AnLs73TL6WYtAeYSb0KB1R5MVeLjb1Ju", "name": "get_z_position", "input": {}}, + ], "stop_reason": "tool_use", "stop_sequence": None, + "usage": {"input_tokens": 1, "output_tokens": 1}, + }) + class FakeStream: + def __enter__(self): return self + def __exit__(self, *exc): return False + def __iter__(self): return iter(()) + def get_final_message(self): return response + fake_client = SimpleNamespace(messages=SimpleNamespace(stream=lambda **kw: FakeStream())) + monkeypatch.setattr(agent, "_local_vision", {}) + monkeypatch.delenv("MICROCLAW_LLM_BASE_URL", raising=False) + monkeypatch.setattr(agent, "_get_client", lambda: fake_client) + anth_gen = agent._stream_one_round([], [], "m", []) + while True: + try: next(anth_gen) + except StopIteration as stop: + anthropic_message = stop.value + break + + def history(message): + blocks = [b.model_dump() for b in message.content] + results = [{"type": "tool_result", "tool_use_id": b["id"], + "content": json.dumps({"ok": True})} for b in blocks] + return [{"role": "assistant", "content": blocks}, {"role": "user", "content": results}] + + class Guard: + def __init__(self, root): + from types import SimpleNamespace + self.root = Path(root) + self._c = SimpleNamespace(stage=SimpleNamespace(x_min=-1000, x_max=1000, + y_min=-1000, y_max=1000, z_min=-1000, z_max=1000), + camera=SimpleNamespace(max_exposure_ms=None)) + @property + def analysis_min_snr(self): return None + def resolve_in_workspace(self, path): return str(self.root / path) + + outputs = [] + for index, message in enumerate((anthropic_message, local_message)): + path = f"script{index}.py" + tools.export_session_script(None, Guard(tmp_path), path, history(message)) + outputs.append((tmp_path / path).read_text()) + assert outputs[0] == outputs[1] + + bad_body = replay("stream_tool_calls").content.replace(b'"arguments":"0"', b'"arguments":"?"') + bad = httpx.Response(200, headers={"content-type": "text/event-stream"}, content=bad_body) + install(monkeypatch, bad) + bad_gen = agent._stream_one_round([], [], "m", []) + while True: + try: next(bad_gen) + except StopIteration as stop: + malformed = stop.value + break + bad_history = history(malformed) + failed = next(b for b in malformed.content if b.type == "tool_use" and "_microclaw_parse_error" in b.input) + bad_history[1]["content"] = [{"type": "tool_result", "tool_use_id": failed.id, + "is_error": True, "content": json.dumps({"error": failed.input["_microclaw_parse_error"]})}] + tools.export_session_script(None, Guard(tmp_path), "bad.py", bad_history) + bad_script = (tmp_path / "bad.py").read_text() + assert "core.set_exposure" not in bad_script + assert "SKIPPED: set_exposure" in bad_script diff --git a/tests/test_webserve.py b/tests/test_webserve.py index 38c88192..198ccb25 100644 --- a/tests/test_webserve.py +++ b/tests/test_webserve.py @@ -1568,6 +1568,19 @@ def test_post_key_sets_and_persists(client, monkeypatch): assert "sk-ant-xyzXYZW" not in json.dumps(body) +def test_post_key_uses_local_credential_store_when_local_backend_selected(client, monkeypatch): + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://local-server") + state = {} + monkeypatch.setattr(webserve, "set_llm_api_key", lambda key: state.update(live=key)) + monkeypatch.setattr(webserve, "set_api_key", lambda key: pytest.fail("set Anthropic key")) + monkeypatch.setattr(credentials, "store_llm_api_key", lambda key: ("file", "/tmp/local-config")) + monkeypatch.setattr(credentials, "store_api_key", lambda key: pytest.fail("stored Anthropic key")) + body = client.post("/api/key", json={"key": "local-secret-xyz", "persist": True}).json() + assert state["live"] == "local-secret-xyz" + assert body["stored_in"] == "file" + assert body["suffix"] == "…-xyz" + + def test_post_key_can_replace_a_key_already_set(client, monkeypatch): """The chip in the header reopens the banner; the endpoint overwrites.""" stored = {} From c9af47c6d13a54e3b6c5e7e4a86111821033ac5a Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 12:36:23 -0400 Subject: [PATCH 13/53] design/84 84a: round-2 review fixes (UNREVIEWED; runner turn killed at the end) Codex revision-002 addressed G1-G7 and was killed by the coordinator's 2-hour background limit during its final verification step, after its targeted run (665 passed, 2 sandbox socket failures). Committed by the coordinator as found; not yet reviewed. Co-Authored-By: Claude Opus 5.5 --- microclaw/__main__.py | 2 +- microclaw/agent.py | 366 +++++++++++++++++++++++++----------- microclaw/credentials.py | 67 +++---- microclaw/tools.py | 7 +- microclaw/webserve.py | 17 +- tests/test_credentials.py | 17 ++ tests/test_local_backend.py | 78 ++++++-- tests/test_webserve.py | 20 +- 8 files changed, 394 insertions(+), 180 deletions(-) diff --git a/microclaw/__main__.py b/microclaw/__main__.py index 0e18ea61..0e9437a3 100644 --- a/microclaw/__main__.py +++ b/microclaw/__main__.py @@ -139,7 +139,7 @@ def run_session(args): from microclaw.agent import llm_base_url if llm_base_url(): - key, source = credentials.load_llm_api_key() + key, source = credentials.load_api_key("llm") from microclaw.agent import set_llm_api_key if key: set_llm_api_key(key) else: diff --git a/microclaw/agent.py b/microclaw/agent.py index 3e7c2612..3eae6043 100644 --- a/microclaw/agent.py +++ b/microclaw/agent.py @@ -6,6 +6,7 @@ from datetime import datetime, timezone import os import time +import uuid import httpx import anthropic @@ -868,115 +869,268 @@ def _stream_one_round(messages, system_blocks, model, tool_schemas, raise RuntimeError("unexpected loop exit") # pragma: no cover +def _local_request_body(model_messages, system_blocks, tool_schemas, model, vision): + """Translate canonical history into one OpenAI-compatible request body.""" + wire = [{ + "role": "system", + "content": "\n\n".join( + block.get("text", "") + for block in system_blocks + if block.get("type") == "text" + ), + }] + for message in model_messages: + role = message["role"] + content = message["content"] + if isinstance(content, str): + wire.append({"role": role, "content": content}) + continue + if role == "assistant": + texts = [] + calls = [] + for block in content: + if isinstance(block, dict): + kind = block.get("type") + get = block.get + else: + kind = getattr(block, "type", None) + get = lambda key: getattr(block, key) + if kind == "text": + texts.append(get("text")) + elif kind == "tool_use": + calls.append({ + "id": get("id"), + "type": "function", + "function": { + "name": get("name"), + "arguments": json.dumps(get("input"), separators=(",", ":")), + }, + }) + else: + raise ValueError(f"Cannot translate assistant content block {kind!r}") + entry = {"role": "assistant", "content": "".join(texts) or None} + if calls: + entry["tool_calls"] = calls + wire.append(entry) + elif role == "user": + images = [] + for block in content: + kind = block.get("type") + if kind == "text": + wire.append({"role": "user", "content": block.get("text", "")}) + continue + if kind != "tool_result": + raise ValueError(f"Cannot translate user content block {kind!r} to the local chat API") + parts = block.get("content", "") + if isinstance(parts, list): + text = "\n".join( + part.get("text", "") + for part in parts + if part.get("type") == "text" + ) + for part in parts: + if part.get("type") != "image": + continue + if vision: + source = part["source"] + images.append({ + "type": "image_url", + "image_url": { + "url": f"data:{source['media_type']};base64,{source['data']}" + }, + }) + if not vision and any(part.get("type") == "image" for part in parts): + omitted = '{"image_omitted": "this model does not accept images"}' + text += ("\n" if text else "") + omitted + else: + text = str(parts) + if block.get("is_error"): + text = "Error: " + text + wire.append({ + "role": "tool", + "tool_call_id": block["tool_use_id"], + "content": text, + }) + if images: + wire.append({"role": "user", "content": images}) + else: + raise ValueError(f"Cannot translate message role {role!r} to the local chat API") + + payload = { + "model": model, + "messages": wire, + "max_tokens": MAX_OUTPUT_TOKENS, + "stream": True, + "stream_options": {"include_usage": True}, + } + if tool_schemas: + payload["tools"] = [ + { + "type": "function", + "function": { + "name": tool["name"], + "description": tool.get("description", ""), + "parameters": tool["input_schema"], + }, + } + for tool in tool_schemas + ] + return payload + + +def _local_error(response, base): + response.read() + try: + error = response.json().get("error", {}) + except Exception: + error = {} + if response.status_code == 404: + raise _BadModel(f"Local server {base} does not know the selected model.") + if response.status_code in (429, 503): + try: + retry_after = float(response.headers.get("retry-after", 0)) + except ValueError: + retry_after = 0 + raise _LocalTransient( + f"Local server {base} is still unavailable after retrying.", retry_after + ) + if response.status_code == 400 and error.get("n_ctx"): + raise _LocalAPIError( + f"Local server {base} context limit is {error['n_ctx']} tokens; " + f"request uses {error.get('n_prompt_tokens', 'an unknown number of')} tokens." + ) + if response.status_code == 401: + raise _LocalAPIError( + f"Local server {base} rejected its API key. Set {LLM_API_KEY_ENV} " + "or enter the local key in the browser." + ) + detail = error.get("message", response.content.decode(errors="replace")) + raise _LocalAPIError(f"Local server {base} returned HTTP {response.status_code}: {detail}") + + +def _read_local_stream(response, base, model, iteration, usage_sink): + """Read chat-completion SSE and return the Anthropic-shaped Message.""" + text = [] + calls = {} + finish = None + usage = {} + timings = {} + reasoning_length = 0 + for line in response.iter_lines(): + if not line.startswith("data: "): + continue + data = line[6:] + if data == "[DONE]": + continue + chunk = json.loads(data) + if "error" in chunk: + message = chunk["error"].get("message", str(chunk["error"])) + raise _LocalAPIError(f"Local server {base} returned a stream error: {message}") + if not chunk.get("choices"): + usage = chunk.get("usage", {}) + timings = chunk.get("timings", {}) + continue + choice = chunk["choices"][0] + delta = choice.get("delta", {}) + reasoning_length += len(delta.get("reasoning_content") or "") + if delta.get("content"): + text.append(delta["content"]) + yield {"type": "text_delta", "text": delta["content"]} + for call in delta.get("tool_calls", []): + item = calls.setdefault(call["index"], {"id": "", "name": "", "arguments": ""}) + item["id"] += call.get("id", "") + function = call.get("function", {}) + item["name"] += function.get("name", "") + item["arguments"] += function.get("arguments", "") + if choice.get("finish_reason"): + finish = choice["finish_reason"] + + blocks = [{"type": "text", "text": "".join(text)}] if text else [] + for item in calls.values(): + if not item["id"]: + item["id"] = "call_" + uuid.uuid4().hex + try: + tool_input = json.loads(item["arguments"] or "{}") + except json.JSONDecodeError as exc: + tool_input = { + "_microclaw_parse_error": f"Invalid tool arguments JSON: {exc}", + "raw": item["arguments"], + } + blocks.append({ + "type": "tool_use", + "id": item["id"], + "name": item["name"], + "input": tool_input, + }) + + stop_reason = { + "stop": "end_turn", + "tool_calls": "tool_use", + "length": "max_tokens", + }.get(finish) + response = anthropic.types.Message.model_validate({ + "id": "local", + "type": "message", + "role": "assistant", + "model": model, + "content": blocks, + "stop_reason": stop_reason, + "stop_sequence": None, + "usage": { + "input_tokens": usage.get("prompt_tokens", 0), + "output_tokens": usage.get("completion_tokens", 0), + "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0), + }, + }) + if usage_sink is not None: + record = { + "timestamp": datetime.now(timezone.utc).isoformat(), + "model": model, + "iteration": iteration, + "stop_reason": stop_reason, + "input_tokens": usage.get("prompt_tokens"), + "output_tokens": usage.get("completion_tokens"), + "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens"), + "reasoning_characters": reasoning_length, + } + record.update({ + field: timings[field] + for field in ("prompt_n", "cache_n", "prompt_ms", "predicted_n", "predicted_ms") + if field in timings + }) + try: + usage_sink(record) + except Exception as exc: + print(f"[microclaw] Could not record usage: {exc}", file=sys.stderr) + return response + + def _stream_local_round(messages, system_blocks, model, tool_schemas, context_provider, iteration, usage_sink): base = llm_base_url() - headers = {"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} if os.environ.get(LLM_API_KEY_ENV) else {} - try: - if base not in _local_vision: - try: - _local_vision[base] = bool(_local_get("/props").json().get("modalities", {}).get("vision")) - except Exception: - _local_vision[base] = False - model_messages = context_provider(messages) if context_provider else messages - wire = [{"role": "system", "content": "\n\n".join( - b.get("text", "") for b in system_blocks if b.get("type") == "text") }] - for msg in model_messages: - role = msg["role"] - content = msg["content"] - if isinstance(content, str): - wire.append({"role": role, "content": content}); continue - if role == "assistant": - texts, calls = [], [] - for block in content: - typ = getattr(block, "type", None) or (block.get("type") if isinstance(block, dict) else None) - get = (lambda k: getattr(block, k)) if not isinstance(block, dict) else block.get - if typ == "text": texts.append(get("text")) - elif typ == "tool_use": calls.append({"id": get("id"), "type": "function", "function": {"name": get("name"), "arguments": json.dumps(get("input"), separators=(",", ":"))}}) - entry = {"role": "assistant", "content": "".join(texts) or None} - if calls: entry["tool_calls"] = calls - wire.append(entry) - elif role == "user": - images = [] - for block in content: - if block.get("type") == "text": - wire.append({"role": "user", "content": block.get("text", "")}) - continue - if block.get("type") != "tool_result": - raise ValueError(f"Cannot translate user content block {block.get('type')!r} to the local chat API") - parts = block.get("content", "") - text = "\n".join(p.get("text", "") for p in parts if p.get("type") == "text") if isinstance(parts, list) else str(parts) - if isinstance(parts, list) and any(p.get("type") == "image" for p in parts): - if _local_vision[base]: - for p in parts: - if p.get("type") == "image": - src = p["source"]; images.append({"type": "image_url", "image_url": {"url": f"data:{src['media_type']};base64,{src['data']}"}}) - else: text += ('\n' if text else '') + '{"image_omitted": "this model does not accept images"}' - if block.get("is_error"): text = "Error: " + text - wire.append({"role": "tool", "tool_call_id": block["tool_use_id"], "content": text}) - if images: - wire.append({"role": "user", "content": images}) - else: - raise ValueError(f"Cannot translate message role {role!r} to the local chat API") - payload = {"model": model, "messages": wire, "max_tokens": MAX_OUTPUT_TOKENS, "stream": True, "stream_options": {"include_usage": True}} - if tool_schemas: - payload["tools"] = [{"type": "function", "function": {"name": t["name"], "description": t.get("description", ""), "parameters": t["input_schema"]}} for t in tool_schemas] - text, calls, finish, usage, timings = [], {}, None, {}, {} - reason_length = 0 - timeout = httpx.Timeout(connect=5.0, read=None, write=30.0, pool=5.0) - with _get_local_http().stream("POST", base + "/v1/chat/completions", json=payload, - headers=headers, timeout=timeout) as response: - if response.status_code >= 400: - response.read() - try: err = response.json().get("error", {}) - except Exception: err = {} - if response.status_code == 404: raise _BadModel(f"Local server {base} does not know model {model!r}.") - if response.status_code in (429, 503): - try: retry_after = float(response.headers.get("retry-after", 0)) - except ValueError: retry_after = 0 - raise _LocalTransient(f"Local server {base} is still unavailable after retrying.", retry_after) - if response.status_code == 400 and err.get("n_ctx"): - raise _LocalAPIError(f"Local server {base} context limit is {err['n_ctx']} tokens; request uses {err.get('n_prompt_tokens', 'an unknown number of')} tokens.") - if response.status_code == 401: - raise _LocalAPIError(f"Local server {base} rejected its API key. Set {LLM_API_KEY_ENV} or enter the local key in the browser.") - raise _LocalAPIError(f"Local server {base} returned HTTP {response.status_code}: {err.get('message', response.read().decode(errors='replace'))}") - for line in response.iter_lines(): - if not line.startswith("data: "): continue - data = line[6:] - if data == "[DONE]": continue - chunk = json.loads(data) - if not chunk.get("choices"): - usage = chunk.get("usage", {}); timings = chunk.get("timings", {}); continue - choice = chunk["choices"][0]; delta = choice.get("delta", {}) - reason_length += len(delta.get("reasoning_content") or "") - if delta.get("content"): - text.append(delta["content"]); yield {"type": "text_delta", "text": delta["content"]} - for call in delta.get("tool_calls", []): - item = calls.setdefault(call["index"], {"id": "", "name": "", "arguments": ""}) - item["id"] += call.get("id", "") - fn = call.get("function", {}); item["name"] += fn.get("name", ""); item["arguments"] += fn.get("arguments", "") - if choice.get("finish_reason"): finish = choice["finish_reason"] - blocks = ([{"type": "text", "text": "".join(text)}] if text else []) - for item in calls.values(): - if not item["id"]: - import uuid - item["id"] = "call_" + uuid.uuid4().hex - try: inp = json.loads(item["arguments"] or "{}") - except json.JSONDecodeError as exc: inp = {"_microclaw_parse_error": f"Invalid tool arguments JSON: {exc}", "raw": item["arguments"]} - blocks.append({"type": "tool_use", "id": item["id"], "name": item["name"], "input": inp}) - stop = {"stop": "end_turn", "tool_calls": "tool_use", "length": "max_tokens"}.get(finish) - result = anthropic.types.Message.model_validate({"id": "local", "type": "message", "role": "assistant", "model": model, "content": blocks, "stop_reason": stop, "stop_sequence": None, "usage": {"input_tokens": usage.get("prompt_tokens", 0), "output_tokens": usage.get("completion_tokens", 0), "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)}}) - if usage_sink: - record = {"timestamp": datetime.now(timezone.utc).isoformat(), "model": model, "iteration": iteration, "stop_reason": stop, "input_tokens": usage.get("prompt_tokens"), "output_tokens": usage.get("completion_tokens"), "cache_read_input_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens")} - record.update({k: timings[k] for k in ("prompt_n", "cache_n", "prompt_ms", "predicted_n", "predicted_ms") if k in timings}) - record["reasoning_characters"] = reason_length - try: - usage_sink(record) - except Exception as exc: - print(f"[microclaw] Could not record usage: {exc}", file=sys.stderr) - return result - except httpx.TransportError: - raise + if base not in _local_vision: + try: + _local_vision[base] = bool( + _local_get("/props").json().get("modalities", {}).get("vision") + ) + except Exception: + _local_vision[base] = False + model_messages = context_provider(messages) if context_provider else messages + payload = _local_request_body( + model_messages, system_blocks, tool_schemas, model, _local_vision[base] + ) + headers = {} + key = os.environ.get(LLM_API_KEY_ENV) + if key: + headers["Authorization"] = f"Bearer {key}" + timeout = httpx.Timeout(connect=5.0, read=None, write=30.0, pool=5.0) + with _get_local_http().stream( + "POST", base + "/v1/chat/completions", json=payload, + headers=headers, timeout=timeout, + ) as response: + if response.status_code >= 400: + _local_error(response, base) + return (yield from _read_local_stream(response, base, model, iteration, usage_sink)) def run_agent_iter( diff --git a/microclaw/credentials.py b/microclaw/credentials.py index 3f001cb9..2509080a 100644 --- a/microclaw/credentials.py +++ b/microclaw/credentials.py @@ -1,4 +1,4 @@ -"""Where the Anthropic API key lives when it isn't in the environment. +"""Where Anthropic and local OpenAI-compatible API keys live. `microclaw serve` can collect a key from the browser on first run, so it needs somewhere to put it. Resolution order is env var > OS credential store > a @@ -33,6 +33,10 @@ LLM_ENV_VAR = "MICROCLAW_LLM_API_KEY" _LLM_KEYRING_USERNAME = "local-llm-api-key" _LLM_TOML_KEY = "llm_api_key" +_KEY_CONFIG = { + "anthropic": (ENV_VAR, _KEYRING_USERNAME, _TOML_KEY), + "llm": (LLM_ENV_VAR, _LLM_KEYRING_USERNAME, _LLM_TOML_KEY), +} def config_path() -> Path: @@ -55,10 +59,6 @@ def _keyring(): return keyring -def _read_config_key() -> str | None: - return _read_toml_key(_TOML_KEY) - - def _read_toml_key(name: str) -> str | None: path = config_path() try: @@ -72,45 +72,55 @@ def _read_toml_key(name: str) -> str | None: return None -def load_stored_key() -> tuple[str | None, str | None]: +def load_stored_key(kind: str = "anthropic") -> tuple[str | None, str | None]: """The persisted key, ignoring the environment: (key, source) or (None, None). Separate from `load_api_key` because a key set for this process only still leaves whatever was persisted earlier in place, to be picked up on the next start. Callers that need to say so have to look past os.environ. """ + _, username, toml_key = _key_config(kind) kr = _keyring() if kr is not None: try: - key = kr.get_password(_KEYRING_SERVICE, _KEYRING_USERNAME) + key = kr.get_password(_KEYRING_SERVICE, username) except Exception: key = None if key: return key, "keyring" - key = _read_config_key() + key = _read_toml_key(toml_key) if key: return key, "file" return None, None -def load_api_key() -> tuple[str | None, str | None]: +def _key_config(kind: str) -> tuple[str, str, str]: + try: + return _KEY_CONFIG[kind] + except KeyError: + raise ValueError(f"Unknown credential kind: {kind!r}") from None + + +def load_api_key(kind: str = "anthropic") -> tuple[str | None, str | None]: """Return (key, source) where source is 'env', 'keyring', 'file' or None.""" - env = os.environ.get(ENV_VAR) + env_name, _, _ = _key_config(kind) + env = os.environ.get(env_name) if env: return env, "env" - return load_stored_key() + return load_stored_key(kind) -def store_api_key(key: str) -> tuple[str, str | None]: +def store_api_key(key: str, kind: str = "anthropic") -> tuple[str, str | None]: """Persist `key`, preferring the OS credential store. Returns (destination, path) where destination is 'keyring' or 'file'. """ + _, username, toml_key = _key_config(kind) kr = _keyring() if kr is not None: try: - kr.set_password(_KEYRING_SERVICE, _KEYRING_USERNAME, key) + kr.set_password(_KEYRING_SERVICE, username, key) return "keyring", None except Exception: pass # fall through to the file @@ -118,7 +128,7 @@ def store_api_key(key: str) -> tuple[str, str | None]: path = config_path() path.parent.mkdir(parents=True, exist_ok=True) # json.dumps gives us a correctly-escaped TOML basic string. - _write_config_key(_TOML_KEY, key) + _write_config_key(toml_key, key) if os.name != "nt": os.chmod(path, stat.S_IRUSR | stat.S_IWUSR) return "file", str(path) @@ -135,35 +145,6 @@ def _write_config_key(name: str, key: str) -> None: path.write_text("\n".join(f"{k} = {json.dumps(v)}" for k, v in values.items()) + "\n", encoding="utf-8") -def load_llm_api_key() -> tuple[str | None, str | None]: - env = os.environ.get(LLM_ENV_VAR) - if env: - return env, "env" - kr = _keyring() - if kr is not None: - try: - key = kr.get_password(_KEYRING_SERVICE, _LLM_KEYRING_USERNAME) - if key: return key, "keyring" - except Exception: pass - key = _read_toml_key(_LLM_TOML_KEY) - return (key, "file") if key else (None, None) - - -def store_llm_api_key(key: str) -> tuple[str, str | None]: - kr = _keyring() - if kr is not None: - try: - kr.set_password(_KEYRING_SERVICE, _LLM_KEYRING_USERNAME, key) - return "keyring", None - except Exception: pass - path = config_path() - path.parent.mkdir(parents=True, exist_ok=True) - _write_config_key(_LLM_TOML_KEY, key) - if os.name != "nt": - os.chmod(path, stat.S_IRUSR | stat.S_IWUSR) - return "file", str(path) - - def mask(key: str) -> str: """A suffix, enough to confirm *which* key is set. Never echo the key.""" return "…" + key[-4:] if len(key) > 4 else "…" diff --git a/microclaw/tools.py b/microclaw/tools.py index da680ac4..02ec86b3 100644 --- a/microclaw/tools.py +++ b/microclaw/tools.py @@ -1049,12 +1049,7 @@ def _recorded_tool_calls(records: list[dict]) -> list[tuple[str, RecordedParams] name = block.get("name") if isinstance(block, dict) else block.name params = block.get("input", {}) if isinstance(block, dict) else block.input block_id = block.get("id") if isinstance(block, dict) else block.id - recorded_result = results.get(str(block_id)) - if isinstance(params, dict) and "_microclaw_parse_error" in params: - # Balance the invalid tool_use in history, then report it as a - # skipped failed call. Never pass its raw arguments to an emitter. - recorded_result = {"error": params["_microclaw_parse_error"]} - recorded_params = RecordedParams(dict(params), recorded_result) + recorded_params = RecordedParams(dict(params), results.get(str(block_id))) recorded_params["_tool_use_id"] = str(block_id) calls.append((str(name), recorded_params)) return calls diff --git a/microclaw/webserve.py b/microclaw/webserve.py index 70a60260..67d6388b 100644 --- a/microclaw/webserve.py +++ b/microclaw/webserve.py @@ -385,7 +385,7 @@ def _initialize(self, args): # env > keyring > file; a key found in a store is pushed into the # environment now so the first turn doesn't have to look for it. - key, source = (credentials.load_llm_api_key() if llm_base_url() + key, source = (credentials.load_api_key("llm") if llm_base_url() else credentials.load_api_key()) if key: _add_audit_secret(self, key) @@ -1495,7 +1495,7 @@ async def get_artifact(path: str): @app.get("/api/key") async def get_key(): - key, source = (credentials.load_llm_api_key() if llm_base_url() + key, source = (credentials.load_api_key("llm") if llm_base_url() else credentials.load_api_key()) return JSONResponse( { @@ -1518,18 +1518,23 @@ async def post_key(k: Key): # Overwrites whatever was set, so a key can be swapped mid-session. _add_audit_secret(session, key) local_path = bool(llm_base_url()) - store = credentials.store_llm_api_key if local_path else credentials.store_api_key - load_stored = credentials.load_llm_api_key if local_path else credentials.load_stored_key + credential_kind = "llm" if local_path else "anthropic" if local_path: set_llm_api_key(key) else: set_api_key(key) stored_in = stored_at = None stale_store = False if k.persist: - stored_in, stored_at = store(key) + if local_path: + stored_in, stored_at = credentials.store_api_key(key, credential_kind) + else: + stored_in, stored_at = credentials.store_api_key(key) else: # A key held for this process only doesn't displace an older one in # the credential store — that one comes back on the next start. - stored, _ = load_stored() + if local_path: + stored, _ = credentials.load_stored_key(credential_kind) + else: + stored, _ = credentials.load_stored_key() stale_store = stored is not None and stored != key # Never echo the key: a suffix is enough to confirm which one is set. return JSONResponse( diff --git a/tests/test_credentials.py b/tests/test_credentials.py index daddfdb4..23ed8216 100644 --- a/tests/test_credentials.py +++ b/tests/test_credentials.py @@ -71,6 +71,23 @@ def set_password(self, *a): assert where == "file" +def test_local_key_uses_separate_environment_and_stored_value(monkeypatch, no_keyring, config_home): + credentials.store_api_key("anthropic-key") + credentials.store_api_key("local-key", "llm") + monkeypatch.setenv(credentials.LLM_ENV_VAR, "process-local-key") + assert credentials.load_api_key("llm") == ("process-local-key", "env") + assert credentials.load_stored_key("llm") == ("local-key", "file") + assert credentials.load_api_key() == ("anthropic-key", "file") + assert credentials.load_stored_key() == ("anthropic-key", "file") + + +def test_storing_either_key_preserves_both_toml_entries(no_keyring, config_home): + credentials.store_api_key("anthropic-key") + credentials.store_api_key("local-key", "llm") + assert credentials.load_api_key() == ("anthropic-key", "file") + assert credentials.load_api_key("llm") == ("local-key", "file") + + def test_mask_shows_only_a_suffix(): assert credentials.mask("sk-ant-api03-AA8f") == "…AA8f" assert credentials.mask("abc") == "…" diff --git a/tests/test_local_backend.py b/tests/test_local_backend.py index 168ce654..7e1e099c 100644 --- a/tests/test_local_backend.py +++ b/tests/test_local_backend.py @@ -26,7 +26,7 @@ def handler(request): if request.url.path.endswith("chat/completions"): captured.append(json.loads(request.content)) captured.timeouts.append(request.extensions.get("timeout")) - selected = remaining.pop(0) + selected = remaining.pop(0) if remaining else "stream_text_reply" if isinstance(selected, httpx.Response): return selected response = replay(selected) if extra: response = extra(response) @@ -72,6 +72,8 @@ def test_recorded_stream_tool_translation_and_usage(monkeypatch): except StopIteration as stop: msg = stop.value; break assert msg.stop_reason == "tool_use" + assert msg.usage.input_tokens == 33600 + assert msg.usage.cache_read_input_tokens == 33574 tools = [b for b in msg.content if b.type == "tool_use"] assert [(b.name, b.id, b.input) for b in tools] == [ ("set_exposure", "ipQX5BPZX3xG7RuJKJYFu18DrcORALAD", {"ms": 50}), @@ -143,8 +145,13 @@ def test_request_translation_includes_checkpoint_and_all_d2_rows(monkeypatch): {"id": "b", "type": "function", "function": {"name": "beta", "arguments": "{}"}}, ]} tools_wire = [m for m in body["messages"] if m["role"] == "tool"] - assert tools_wire[-2]["content"] == "Error: bad" - assert tools_wire[-1]["content"].endswith('{"image_omitted": "this model does not accept images"}') + calls_by_id = {call["id"]: call for call in assistant["tool_calls"]} + results_by_id = {message["tool_call_id"]: message for message in tools_wire} + assert set(calls_by_id) == set(results_by_id) == {"a", "b"} + assert results_by_id["a"]["content"] == "Error: bad" + assert results_by_id["b"]["content"].endswith( + '{"image_omitted": "this model does not accept images"}' + ) assert body["tools"][0]["function"]["parameters"] == {"type": "object"} assert "cache_control" not in str(body) @@ -239,6 +246,32 @@ def test_local_http_errors_become_agent_error_events(monkeypatch): assert history == [] +def test_midstream_error_chunk_becomes_agent_error_event(monkeypatch): + # Minimal derivative of the recorded tool-call stream: insert one error + # event immediately before the recorded final usage-only chunk. + body = replay("stream_tool_calls").content + usage_start = body.rfind(b'data: {"choices":[]') + assert usage_start >= 0 + body = ( + body[:usage_start] + + b'data: {"error":{"message":"stream interrupted"}}\n\n' + + body[usage_start:] + ) + response = httpx.Response( + 200, headers={"content-type": "text/event-stream"}, content=body + ) + install(monkeypatch, response) + from microclaw.agent import run_agent_iter + + history = [] + events = list(run_agent_iter("hello", None, None, history, model="m", tool_schemas=[])) + assert events[-1] == { + "type": "error", + "message": "Local server http://recorded-server returned a stream error: stream interrupted", + } + assert history == [] + + def test_serve_accepts_llm_url_before_or_after_subcommand(monkeypatch): from microclaw import __main__ as cli, webserve monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "") @@ -316,23 +349,36 @@ def resolve_in_workspace(self, path): return str(self.root / path) for index, message in enumerate((anthropic_message, local_message)): path = f"script{index}.py" tools.export_session_script(None, Guard(tmp_path), path, history(message)) - outputs.append((tmp_path / path).read_text()) + outputs.append((tmp_path / path).read_text(encoding="utf-8")) assert outputs[0] == outputs[1] bad_body = replay("stream_tool_calls").content.replace(b'"arguments":"0"', b'"arguments":"?"') bad = httpx.Response(200, headers={"content-type": "text/event-stream"}, content=bad_body) - install(monkeypatch, bad) - bad_gen = agent._stream_one_round([], [], "m", []) - while True: - try: next(bad_gen) - except StopIteration as stop: - malformed = stop.value - break - bad_history = history(malformed) - failed = next(b for b in malformed.content if b.type == "tool_use" and "_microclaw_parse_error" in b.input) - bad_history[1]["content"] = [{"type": "tool_result", "tool_use_id": failed.id, - "is_error": True, "content": json.dumps({"error": failed.input["_microclaw_parse_error"]})}] + install(monkeypatch, [bad, "stream_text_reply"]) + from microclaw.agent import run_agent_iter + + def set_exposure(ctrl, guard, ms): + return {"ok": True, "ms": ms} + + def get_z_position(ctrl, guard): + return {"z_um": 12.5} + + bad_history = [] + list(run_agent_iter( + "set exposure", None, None, bad_history, model="m", tool_schemas=[], + tool_registry={"set_exposure": set_exposure, "get_z_position": get_z_position}, + )) + failed = next( + b for m in bad_history if m["role"] == "assistant" for b in m["content"] + if b.type == "tool_use" and "_microclaw_parse_error" in b.input + ) + failure_result = next( + b for m in bad_history if m["role"] == "user" and isinstance(m["content"], list) + for b in m["content"] if b.get("tool_use_id") == failed.id + ) + assert failure_result["is_error"] is True + assert "Invalid tool arguments JSON" in failure_result["content"] tools.export_session_script(None, Guard(tmp_path), "bad.py", bad_history) - bad_script = (tmp_path / "bad.py").read_text() + bad_script = (tmp_path / "bad.py").read_text(encoding="utf-8") assert "core.set_exposure" not in bad_script assert "SKIPPED: set_exposure" in bad_script diff --git a/tests/test_webserve.py b/tests/test_webserve.py index 198ccb25..e3f74b5c 100644 --- a/tests/test_webserve.py +++ b/tests/test_webserve.py @@ -1573,14 +1573,30 @@ def test_post_key_uses_local_credential_store_when_local_backend_selected(client state = {} monkeypatch.setattr(webserve, "set_llm_api_key", lambda key: state.update(live=key)) monkeypatch.setattr(webserve, "set_api_key", lambda key: pytest.fail("set Anthropic key")) - monkeypatch.setattr(credentials, "store_llm_api_key", lambda key: ("file", "/tmp/local-config")) - monkeypatch.setattr(credentials, "store_api_key", lambda key: pytest.fail("stored Anthropic key")) + monkeypatch.setattr( + credentials, "store_api_key", + lambda key, kind="anthropic": ("file", "/tmp/local-config") + if kind == "llm" else pytest.fail("stored Anthropic key"), + ) body = client.post("/api/key", json={"key": "local-secret-xyz", "persist": True}).json() assert state["live"] == "local-secret-xyz" assert body["stored_in"] == "file" assert body["suffix"] == "…-xyz" +def test_local_session_key_checks_stored_key_without_environment_value(client, monkeypatch): + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://local-server") + monkeypatch.setattr(webserve, "set_llm_api_key", lambda key: None) + monkeypatch.setattr(webserve, "set_api_key", lambda key: pytest.fail("set Anthropic key")) + monkeypatch.setattr( + credentials, "load_stored_key", + lambda kind="anthropic": ("older-local-key", "file") + if kind == "llm" else (None, None), + ) + body = client.post("/api/key", json={"key": "current-local-key", "persist": False}).json() + assert body["stale_store"] is True + + def test_post_key_can_replace_a_key_already_set(client, monkeypatch): """The chip in the header reopens the banner; the endpoint overwrites.""" stored = {} From 4509d1e57d462e4443bd36bea63ef3d0d0ce5bb4 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 12:53:15 -0400 Subject: [PATCH 14/53] design/84 84b: recorded llama.cpp /tokenize response Co-Authored-By: Claude Opus 5.5 --- tests/fixtures/llamacpp/README.md | 2 ++ tests/fixtures/llamacpp/post_tokenize.http | 4 ++++ 2 files changed, 6 insertions(+) create mode 100644 tests/fixtures/llamacpp/post_tokenize.http diff --git a/tests/fixtures/llamacpp/README.md b/tests/fixtures/llamacpp/README.md index bf53c1b1..1cfc64a8 100644 --- a/tests/fixtures/llamacpp/README.md +++ b/tests/fixtures/llamacpp/README.md @@ -8,3 +8,5 @@ SYSTEM_PROMPT and all TOOLS, streamed with `stream_options.include_usage`, except `error_bad_key` (wrong key) and `error_context_exceeded` (a 140k-token user message, no tools). `get_props` / `get_v1_models` are the GET endpoints. Replay these; do not hand-write SSE. + +`post_tokenize` is `POST /tokenize` with body `{"content": "Microclaw measures the focus_metric."}`. diff --git a/tests/fixtures/llamacpp/post_tokenize.http b/tests/fixtures/llamacpp/post_tokenize.http new file mode 100644 index 00000000..70ecd044 --- /dev/null +++ b/tests/fixtures/llamacpp/post_tokenize.http @@ -0,0 +1,4 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "Content-Type": "application/json; charset=utf-8", "Content-Length": "50", "Keep-Alive": "timeout=5, max=100"} + +{"tokens":[33464,549,654,10633,279,5077,39930,13]} \ No newline at end of file From b76a5576710da4f794f3fc8a145f089b31dbcbec Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 13:10:19 -0400 Subject: [PATCH 15/53] Derive local context budget from server --- microclaw/__main__.py | 4 +- microclaw/agent.py | 78 ++++++++++++++++++++--- microclaw/conversation.py | 17 +++-- microclaw/serve.html | 3 +- microclaw/webserve.py | 5 +- tests/test_context_budget.py | 117 +++++++++++++++++++++++++++++++++++ tests/test_local_backend.py | 5 +- 7 files changed, 208 insertions(+), 21 deletions(-) create mode 100644 tests/test_context_budget.py diff --git a/microclaw/__main__.py b/microclaw/__main__.py index 0e9437a3..e863cded 100644 --- a/microclaw/__main__.py +++ b/microclaw/__main__.py @@ -24,7 +24,6 @@ from microclaw.conversation import ( AcquisitionDiagnosticWriter, AuditLog, - ConversationStore, atomic_write_text, json_default, load_history, @@ -193,7 +192,8 @@ def run_session(args): # run_agent returns a fresh list each turn; _repl mutates this one in place # (history[:] = ...) so the finally block always sees the latest turns. history = [] - store = ConversationStore(AuditLog(history_fn_name, enabled=args.save_history)) + from microclaw.agent import make_conversation_store + store = make_conversation_store(AuditLog(history_fn_name, enabled=args.save_history)) confirmation_fn_name = history_fn_name.replace("_history.jsonl", "_confirmations.jsonl") confirmation_audit = AuditLog(confirmation_fn_name, enabled=args.save_history) usage_fn_name = history_fn_name.replace("_history.jsonl", "_usage.jsonl") diff --git a/microclaw/agent.py b/microclaw/agent.py index 3eae6043..a382c060 100644 --- a/microclaw/agent.py +++ b/microclaw/agent.py @@ -21,6 +21,7 @@ from microclaw.skills import catalog_text from microclaw.tools import TOOL_REGISTRY, execute_tool from microclaw.tools_schema import TOOLS_CACHED +from microclaw.conversation import ConversationStore # Latest available Opus at time of writing (verified against the Claude API # model reference). Override per-run with --model or the MICROCLAW_MODEL env var. @@ -63,6 +64,56 @@ def _local_get(path: str): return _get_local_http().get(url + path, headers=headers, timeout=30) MAX_OUTPUT_TOKENS = 8192 +LOCAL_CONTEXT_DEFAULT = 32768 +# 4,096 tokens is a 12% cushion over the measured 33,889-token fixed prefix, +# leaving room for knowledge-base and rig-interview text to change between turns. +LOCAL_CONTEXT_MARGIN = 4096 + + +def make_conversation_store(audit): + """Build the appropriate history budget for the selected backend.""" + if not llm_base_url(): + return ConversationStore(audit) + n_ctx = None + try: + n_ctx = _local_get("/props").json().get("default_generation_settings", {}).get("n_ctx") + except Exception: + pass + if not n_ctx: + try: + models = _local_get("/v1/models").json().get("data", []) + n_ctx = next((model.get("meta", {}).get("n_ctx") for model in models + if model.get("meta", {}).get("n_ctx")), None) + except Exception: + pass + used_default = not n_ctx + n_ctx = int(n_ctx or LOCAL_CONTEXT_DEFAULT) + system_text = "\n\n".join(block.get("text", "") for block in _system_blocks() + if block.get("type") == "text") + translated_tools = [{"type": "function", "function": { + "name": tool["name"], "description": tool.get("description", ""), + "parameters": tool["input_schema"]}} for tool in TOOLS_CACHED] + fixed_text = system_text + json.dumps(translated_tools, ensure_ascii=False, + separators=(",", ":")) + try: + response = _get_local_http().post( + llm_base_url() + "/tokenize", json={"content": fixed_text}, timeout=30, + headers=({"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} + if os.environ.get(LLM_API_KEY_ENV) else {}), + ) + response.raise_for_status() + fixed = len(response.json()["tokens"]) + except Exception: + # The measured fixed prompt is 4.18–4.75 bytes/token. Three bytes/token + # intentionally overcounts unknown servers to keep the guard conservative. + fixed = (len(fixed_text.encode("utf-8")) + 2) // 3 + high = max(2, n_ctx - fixed - MAX_OUTPUT_TOKENS - LOCAL_CONTEXT_MARGIN) + low = max(1, int(high * 0.75)) + print(f"[microclaw] Local context budget: n_ctx={n_ctx}" + f"{' (default; server did not report n_ctx)' if used_default else ''}, " + f"fixed={fixed}, high={high}, low={low}") + return ConversationStore(audit, high_water_tokens=high, + low_water_tokens=low, text_only=True) def _get_client() -> anthropic.Anthropic: @@ -771,7 +822,8 @@ def _stream_one_round(messages, system_blocks, model, tool_schemas, for attempt, delay in enumerate([0, *_RETRY_DELAYS]): delay = max(delay, retry_after) if delay: - yield {"type": "retry", "delay": delay, "attempt": attempt} + yield {"type": "retry", "delay": delay, "attempt": attempt, + "service": "Local server" if llm_base_url() else "API"} time.sleep(delay) try: if llm_base_url(): @@ -956,13 +1008,7 @@ def _local_request_body(model_messages, system_blocks, tool_schemas, model, visi else: raise ValueError(f"Cannot translate message role {role!r} to the local chat API") - payload = { - "model": model, - "messages": wire, - "max_tokens": MAX_OUTPUT_TOKENS, - "stream": True, - "stream_options": {"include_usage": True}, - } + payload = {"model": model} if tool_schemas: payload["tools"] = [ { @@ -975,6 +1021,12 @@ def _local_request_body(model_messages, system_blocks, tool_schemas, model, visi } for tool in tool_schemas ] + payload.update({ + "messages": wire, + "max_tokens": MAX_OUTPUT_TOKENS, + "stream": True, + "stream_options": {"include_usage": True}, + }) return payload @@ -1115,7 +1167,12 @@ def _stream_local_round(messages, system_blocks, model, tool_schemas, ) except Exception: _local_vision[base] = False + store = getattr(context_provider, "__self__", None) + compactions_before = getattr(store, "compaction_count", 0) model_messages = context_provider(messages) if context_provider else messages + if getattr(store, "compaction_count", 0) > compactions_before: + yield {"type": "compaction", "estimated_tokens": getattr( + store, "last_estimated_tokens", None)} payload = _local_request_body( model_messages, system_blocks, tool_schemas, model, _local_vision[base] ) @@ -1375,8 +1432,11 @@ def run_agent( elif event["type"] == "cancelled": reply = event["reason"] elif event["type"] == "retry": + service = f"{event.get('service', 'API')}" print( - f"Anthropic API overloaded — retrying in {event['delay']}s " + f"{service} unavailable — retrying in {event['delay']}s " f"(attempt {event['attempt']}/{len(_RETRY_DELAYS)})..." ) + elif event["type"] == "compaction": + print("[microclaw] Rebuilding conversation context; the next model request may take longer.") return reply, messages diff --git a/microclaw/conversation.py b/microclaw/conversation.py index 2a12d0e5..5e8f4961 100644 --- a/microclaw/conversation.py +++ b/microclaw/conversation.py @@ -391,13 +391,16 @@ def prune_transcripts(directory: str | os.PathLike[str], retention_days: int | N return removed -def estimate_tokens(messages: list[dict]) -> int: +def estimate_tokens(messages: list[dict], *, text_only: bool = False) -> int: """Return a local deterministic token *estimate*, never an exact count.""" # 2.36 UTF-8 bytes per token, measured against the API's own accounting on a # live session (design/82, block 82a's gate): billed history tokens ran 1.69x # the old bytes/4 estimate, stable at 1.67-1.71 over 59 calls. # Four was optimistic in the one direction a context guard must not be. - payload = json.dumps(jsonable(messages), ensure_ascii=False, separators=(",", ":")) + serializable = jsonable(messages) + if text_only: + serializable = _without_images(serializable) + payload = json.dumps(serializable, ensure_ascii=False, separators=(",", ":")) return math.ceil(len(payload.encode("utf-8")) / 2.36) + 8 * len(messages) @@ -543,12 +546,14 @@ class ConversationStore: def __init__(self, audit: AuditLog, *, high_water_tokens: int = DEFAULT_CONTEXT_HIGH_WATER_TOKENS, - low_water_tokens: int = DEFAULT_CONTEXT_LOW_WATER_TOKENS): + low_water_tokens: int = DEFAULT_CONTEXT_LOW_WATER_TOKENS, + text_only: bool = False): if not 0 < low_water_tokens < high_water_tokens: raise ValueError("context low-water must be positive and below high-water") self.audit = audit self.high_water_tokens = high_water_tokens self.low_water_tokens = low_water_tokens + self.text_only = text_only self._cut = 0 self._checkpoint: dict | None = None self.compaction_count = 0 @@ -559,7 +564,7 @@ def append(self, message: dict) -> None: def model_messages(self, full_history: list[dict]) -> list[dict]: current = ([self._checkpoint] if self._checkpoint else []) + full_history[self._cut:] - self.last_estimated_tokens = estimate_tokens(current) + self.last_estimated_tokens = estimate_tokens(current, text_only=self.text_only) if self.last_estimated_tokens <= self.high_water_tokens: return current @@ -578,7 +583,7 @@ def model_messages(self, full_history: list[dict]) -> list[dict]: candidate_checkpoint = _checkpoint(full_history[:end], self.audit.secrets) candidate = [candidate_checkpoint, *full_history[end:]] chosen = end - if estimate_tokens(candidate) <= self.low_water_tokens: + if estimate_tokens(candidate, text_only=self.text_only) <= self.low_water_tokens: break if chosen == self._cut: # The in-flight/recent turn alone is over budget. Splitting it would @@ -589,5 +594,5 @@ def model_messages(self, full_history: list[dict]) -> list[dict]: self._checkpoint = _checkpoint(full_history[:chosen], self.audit.secrets) self.compaction_count += 1 compacted = [self._checkpoint, *full_history[chosen:]] - self.last_estimated_tokens = estimate_tokens(compacted) + self.last_estimated_tokens = estimate_tokens(compacted, text_only=self.text_only) return compacted diff --git a/microclaw/serve.html b/microclaw/serve.html index 3d03e1a4..d5043030 100644 --- a/microclaw/serve.html +++ b/microclaw/serve.html @@ -743,7 +743,8 @@

Confirmation required

live.push({ role: "user", content: [{ type: "tool_result", tool_use_id: ev.tool_use_id, content: ev.content, is_error: ev.is_error }] }); break; - case "retry": toast("API overloaded — retrying in " + ev.delay + "s…"); break; + case "retry": toast(ev.service + " unavailable — retrying in " + ev.delay + "s…"); break; + case "compaction": toast("Rebuilding conversation context; the next model request may take longer."); break; case "cancelled": toast(ev.reason); break; case "error": toast(ev.message); break; case "acquisition_progress": { diff --git a/microclaw/webserve.py b/microclaw/webserve.py index 67d6388b..1c084e97 100644 --- a/microclaw/webserve.py +++ b/microclaw/webserve.py @@ -53,7 +53,7 @@ from microclaw import config, credentials, shortcut, tools, updates, extensions, skill_store from microclaw.authorization import RigAuthorizationError, validate_live_rig from microclaw.conversation import ( - AcquisitionDiagnosticWriter, AuditLog, ConversationStore, prune_transcripts, + AcquisitionDiagnosticWriter, AuditLog, prune_transcripts, ) from microclaw.agent import ( DEFAULT_MODEL, @@ -62,6 +62,7 @@ run_agent_iter, set_api_key, llm_base_url, + make_conversation_store, set_llm_api_key, ) from microclaw.assets import icon_bytes, load_page @@ -354,7 +355,7 @@ def _initialize(self, args): removed = prune_transcripts(".", getattr(args, "history_retention_days", None)) for path in removed: print(f"[microclaw] Pruned transcript: {path}") - self.store = ConversationStore(AuditLog(self.history_fn, enabled=self.save)) + self.store = make_conversation_store(AuditLog(self.history_fn, enabled=self.save)) # Editing credentials from a browser is offered only on loopback, the # same rule design/15 sets for the safety-config editor (v2). self.editable = args.host in LOCAL_HOSTS diff --git a/tests/test_context_budget.py b/tests/test_context_budget.py new file mode 100644 index 00000000..e1a32477 --- /dev/null +++ b/tests/test_context_budget.py @@ -0,0 +1,117 @@ +import json +import os +from pathlib import Path + +import httpx + +from microclaw import agent, conversation + +FIXTURES = Path(__file__).parent / "fixtures" / "llamacpp" + + +def recorded(name): + head, body = (FIXTURES / f"{name}.http").read_bytes().split(b"\n\n", 1) + status, headers = head.split(b"\n", 1) + return httpx.Response(int(status.split()[1]), headers=json.loads(headers), content=body) + + +def test_local_watermarks_use_props_and_tokenize_fixtures(monkeypatch, capsys): + def handler(request): + if request.url.path.endswith("/props"): + return recorded("get_props") + if request.url.path.endswith("/tokenize"): + return recorded("post_tokenize") + raise AssertionError(request.url) + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://recorded-server") + monkeypatch.setattr(agent, "_local_http", httpx.Client(transport=httpx.MockTransport(handler))) + store = agent.make_conversation_store(conversation.AuditLog(None, enabled=False)) + assert store.high_water_tokens == 131072 - 8 - agent.MAX_OUTPUT_TOKENS - 4096 + assert store.low_water_tokens == int(store.high_water_tokens * .75) + assert "n_ctx=131072" in capsys.readouterr().out + + +def test_local_watermarks_fallback_when_server_reports_neither(monkeypatch, capsys): + def handler(request): + if request.url.path.endswith("/props"): + response = recorded("get_props") + return httpx.Response(response.status_code, headers=response.headers, + content=response.content.replace(b',"n_ctx":131072', b'')) + if request.url.path.endswith("/models"): + response = recorded("get_v1_models") + return httpx.Response(response.status_code, headers=response.headers, + content=response.content.replace(b',"n_ctx":131072', b'')) + return httpx.Response(404, json={}) + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://unreported-server") + monkeypatch.setattr(agent, "_local_http", httpx.Client(transport=httpx.MockTransport(handler))) + store = agent.make_conversation_store(conversation.AuditLog(None, enabled=False)) + assert 32768 - 8192 - 4096 - store.high_water_tokens > 0 + assert store.low_water_tokens == int(store.high_water_tokens * .75) + assert "n_ctx=32768 (default; server did not report n_ctx)" in capsys.readouterr().out + + +def test_anthropic_store_retains_120k_90k_watermarks(monkeypatch): + monkeypatch.delenv("MICROCLAW_LLM_BASE_URL", raising=False) + store = agent.make_conversation_store(conversation.AuditLog(None, enabled=False)) + assert (store.high_water_tokens, store.low_water_tokens) == (120_000, 90_000) + + +def test_local_text_estimate_omits_tool_result_image_bytes(): + image = {"type": "image", "source": {"type": "base64", "media_type": "image/png", + "data": "A" * 120_000}} + history = [{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "x", + "content": [image]}]}] + text_store = conversation.ConversationStore(conversation.AuditLog(None, enabled=False), + high_water_tokens=100_000, + low_water_tokens=70_000, text_only=True) + normal_store = conversation.ConversationStore(conversation.AuditLog(None, enabled=False), + high_water_tokens=100_000, + low_water_tokens=70_000) + text_store.model_messages(history) + normal_store.model_messages(history) + assert normal_store.last_estimated_tokens - text_store.last_estimated_tokens > 40_000 + + +def test_local_request_body_prefix_is_stable_across_rounds(): + system = [{"type": "text", "text": "stable system"}] + tools = [{"name": "alpha", "description": "A", "input_schema": {"type": "object"}}] + earlier = [{"role": "user", "content": "first"}, + {"role": "assistant", "content": [{"type": "text", "text": "reply"}]}] + first = agent._local_request_body(earlier, system, tools, "m", False) + second = agent._local_request_body(earlier + [{"role": "user", "content": "next"}], + system, tools, "m", False) + left = json.dumps(first, separators=(",", ":")) + right = json.dumps(second, separators=(",", ":")) + shared = os.path.commonprefix([left, right]) + assert '"role":"system","content":"stable system"' in shared + assert json.dumps(first["tools"][0], separators=(",", ":")) in shared + for message in first["messages"]: + assert json.dumps(message, separators=(",", ":")) in shared + + +def test_compaction_event_precedes_local_request(monkeypatch): + requested = [] + def handler(request): + if request.url.path.endswith("/props"): + return recorded("get_props") + if request.url.path.endswith("/chat/completions"): + requested.append(True) + return recorded("stream_text_reply") + return httpx.Response(404, json={}) + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://recorded-server") + monkeypatch.setattr(agent, "_local_http", httpx.Client(transport=httpx.MockTransport(handler))) + store = conversation.ConversationStore(conversation.AuditLog(None, enabled=False), + high_water_tokens=80, low_water_tokens=40) + history = [] + for index in range(7): + history.extend([ + {"role": "user", "content": f"question {index} " + "x" * 100}, + {"role": "assistant", "content": [{"type": "text", "text": "answer " + "y" * 100}]}, + ]) + from microclaw.agent import run_agent_iter + events = run_agent_iter("new", None, None, history, model="m", tool_schemas=[], + context_provider=store.model_messages) + compaction = next(event for event in events if event["type"] == "compaction") + assert compaction["estimated_tokens"] is not None + assert not requested + list(events) + assert requested == [True] diff --git a/tests/test_local_backend.py b/tests/test_local_backend.py index 7e1e099c..f171d842 100644 --- a/tests/test_local_backend.py +++ b/tests/test_local_backend.py @@ -50,6 +50,8 @@ def __iter__(self): return prop if request.url.path.endswith("/models"): return replay("get_v1_models") + if request.url.path.endswith("/tokenize"): + return replay("post_tokenize") return replay("stream_text_reply") client = httpx.Client(transport=httpx.MockTransport(handler)) monkeypatch.setattr(agent, "_local_http", client) @@ -172,7 +174,8 @@ def test_retry_after_503_then_tool_calls(monkeypatch): install(monkeypatch, [response, "stream_tool_calls"]) monkeypatch.setattr(agent.time, "sleep", lambda _: None) gen = agent._stream_one_round([], [], "m", []) - assert next(gen) == {"type": "retry", "delay": 5, "attempt": 1} + assert next(gen) == {"type": "retry", "delay": 5, "attempt": 1, + "service": "Local server"} while True: try: next(gen) except StopIteration as stop: From 5a030d68bc8862f75185a5526d4aa62df66a2799 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 13:12:33 -0400 Subject: [PATCH 16/53] design/84 84b: recorded llama.cpp /apply-template response Co-Authored-By: Claude Opus 5.5 --- tests/fixtures/llamacpp/README.md | 2 ++ tests/fixtures/llamacpp/post_apply_template.http | 4 ++++ 2 files changed, 6 insertions(+) create mode 100644 tests/fixtures/llamacpp/post_apply_template.http diff --git a/tests/fixtures/llamacpp/README.md b/tests/fixtures/llamacpp/README.md index 1cfc64a8..f13844d4 100644 --- a/tests/fixtures/llamacpp/README.md +++ b/tests/fixtures/llamacpp/README.md @@ -10,3 +10,5 @@ user message, no tools). `get_props` / `get_v1_models` are the GET endpoints. Replay these; do not hand-write SSE. `post_tokenize` is `POST /tokenize` with body `{"content": "Microclaw measures the focus_metric."}`. + +`post_apply_template` is `POST /apply-template` with body `{"messages": [{"role": "system", "content": "S"}, {"role": "user", "content": "hi"}], "tools": [{"type": "function", "fu...` (one system message "S", one user message "hi", the first translated tool). On the real 83-tool prompt, `/apply-template` then `/tokenize` gave 33,864 tokens, equal to the server's own `prompt_tokens` for that request; tokenizing system text + tool JSON without the template gave 30,118. diff --git a/tests/fixtures/llamacpp/post_apply_template.http b/tests/fixtures/llamacpp/post_apply_template.http new file mode 100644 index 00000000..56635992 --- /dev/null +++ b/tests/fixtures/llamacpp/post_apply_template.http @@ -0,0 +1,4 @@ +HTTP 200 +{"Server": "llama.cpp", "Access-Control-Allow-Origin": "", "Content-Type": "application/json; charset=utf-8", "Content-Length": "2424", "Keep-Alive": "timeout=5, max=100"} + +{"prompt":"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.\n\n# Tools\n\nYou have access to the following functions:\n\n\n{\"type\": \"function\", \"function\": {\"name\": \"export_session_script\", \"description\": \"Compile the supplied append-only session record to a standalone pycro-manager Python script. Unsupported calls make the script fail loudly rather than being reconstructed. Refusals appear in not_emitted_calls with complete: false; each reason names the exact missing export capability. This is a Microclaw capability gap, never a camera, rig or hardware defect: quote the reason in your report rather than attributing it to hardware. For a diagnosis needing source detail, read the emitted artifact.\", \"parameters\": {\"type\": \"object\", \"properties\": {\"output_path\": {\"type\": \"string\"}, \"tool_use_ids\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}, \"description\": \"Optional recorded tool_use ids to emit. An id is the `toolu_...` identifier of the call itself, never a tool name: passing \\\"move_named_stage\\\" is refused as an unknown id. Take the ids from a previous export's `recorded_calls`, which lists every id beside its tool. Excluded calls remain visible as SKIPPED comments; dependencies are not inferred.\"}}, \"required\": [\"output_path\"]}}}\n\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n\n\n\nvalue_1\n\n\nThis is the value for the second parameter\nthat can span\nmultiple lines\n\n\n\n\n\nReminder:\n- Function calls MUST follow the specified format: an inner block must be nested within XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n\n\nS<|im_end|>\n<|im_start|>user\nhi<|im_end|>\n<|im_start|>assistant\n\n"} \ No newline at end of file From a3ee341fc2dd1078a2ffacc5b00e4421dbee1402 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 13:24:06 -0400 Subject: [PATCH 17/53] Count rendered local prompt budget --- microclaw/agent.py | 89 +++++++++++++++++++++--------------- tests/test_context_budget.py | 86 ++++++++++++++++++++++++++++------ 2 files changed, 124 insertions(+), 51 deletions(-) diff --git a/microclaw/agent.py b/microclaw/agent.py index a382c060..d6f3c841 100644 --- a/microclaw/agent.py +++ b/microclaw/agent.py @@ -2,6 +2,7 @@ from collections.abc import Callable, Iterator from typing import Any import json +import math import sys from datetime import datetime, timezone import os @@ -63,10 +64,25 @@ def _local_get(path: str): headers = {"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} if os.environ.get(LLM_API_KEY_ENV) else {} return _get_local_http().get(url + path, headers=headers, timeout=30) + +def _local_vision_enabled(base=None): + base = base or llm_base_url() + if not base: + return False + if base not in _local_vision: + try: + _local_vision[base] = bool( + _local_get("/props").json().get("modalities", {}).get("vision") + ) + except Exception: + _local_vision[base] = False + return _local_vision[base] + MAX_OUTPUT_TOKENS = 8192 LOCAL_CONTEXT_DEFAULT = 32768 -# 4,096 tokens is a 12% cushion over the measured 33,889-token fixed prefix, -# leaving room for knowledge-base and rig-interview text to change between turns. +# 4,096 tokens is a 12% cushion over the measured 33,864-token fixed prompt. +# It covers turn-to-turn knowledge/rig-interview changes and history-estimate +# error; the small measured history error was 3 tokens on a 2,716-token sample. LOCAL_CONTEXT_MARGIN = 4096 @@ -88,32 +104,36 @@ def make_conversation_store(audit): pass used_default = not n_ctx n_ctx = int(n_ctx or LOCAL_CONTEXT_DEFAULT) - system_text = "\n\n".join(block.get("text", "") for block in _system_blocks() - if block.get("type") == "text") - translated_tools = [{"type": "function", "function": { - "name": tool["name"], "description": tool.get("description", ""), - "parameters": tool["input_schema"]}} for tool in TOOLS_CACHED] - fixed_text = system_text + json.dumps(translated_tools, ensure_ascii=False, - separators=(",", ":")) + body = _local_request_body( + [{"role": "user", "content": "hi"}], _system_blocks(), TOOLS_CACHED, + "budget", _local_vision_enabled(), + ) try: - response = _get_local_http().post( - llm_base_url() + "/tokenize", json={"content": fixed_text}, timeout=30, - headers=({"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} - if os.environ.get(LLM_API_KEY_ENV) else {}), + headers = ({"Authorization": f"Bearer {os.environ[LLM_API_KEY_ENV]}"} + if os.environ.get(LLM_API_KEY_ENV) else {}) + rendered_response = _get_local_http().post( + llm_base_url() + "/apply-template", json=body, timeout=30, headers=headers, ) - response.raise_for_status() - fixed = len(response.json()["tokens"]) + rendered_response.raise_for_status() + tokenize_response = _get_local_http().post( + llm_base_url() + "/tokenize", + json={"content": rendered_response.json()["prompt"]}, timeout=30, + headers=headers, + ) + tokenize_response.raise_for_status() + fixed = len(tokenize_response.json()["tokens"]) except Exception: - # The measured fixed prompt is 4.18–4.75 bytes/token. Three bytes/token - # intentionally overcounts unknown servers to keep the guard conservative. - fixed = (len(fixed_text.encode("utf-8")) + 2) // 3 + # Raw request JSON measured against the 33,864-token rendered prompt; + # 2.36 bytes/token is deliberately conservative for this fallback. + fixed = math.ceil(len(json.dumps(body, ensure_ascii=False, + separators=(",", ":")).encode("utf-8")) / 2.36) high = max(2, n_ctx - fixed - MAX_OUTPUT_TOKENS - LOCAL_CONTEXT_MARGIN) low = max(1, int(high * 0.75)) print(f"[microclaw] Local context budget: n_ctx={n_ctx}" f"{' (default; server did not report n_ctx)' if used_default else ''}, " f"fixed={fixed}, high={high}, low={low}") return ConversationStore(audit, high_water_tokens=high, - low_water_tokens=low, text_only=True) + low_water_tokens=low, text_only=not _local_vision_enabled()) def _get_client() -> anthropic.Anthropic: @@ -1008,7 +1028,13 @@ def _local_request_body(model_messages, system_blocks, tool_schemas, model, visi else: raise ValueError(f"Cannot translate message role {role!r} to the local chat API") - payload = {"model": model} + payload = { + "model": model, + "messages": wire, + "max_tokens": MAX_OUTPUT_TOKENS, + "stream": True, + "stream_options": {"include_usage": True}, + } if tool_schemas: payload["tools"] = [ { @@ -1021,12 +1047,6 @@ def _local_request_body(model_messages, system_blocks, tool_schemas, model, visi } for tool in tool_schemas ] - payload.update({ - "messages": wire, - "max_tokens": MAX_OUTPUT_TOKENS, - "stream": True, - "stream_options": {"include_usage": True}, - }) return payload @@ -1160,13 +1180,7 @@ def _read_local_stream(response, base, model, iteration, usage_sink): def _stream_local_round(messages, system_blocks, model, tool_schemas, context_provider, iteration, usage_sink): base = llm_base_url() - if base not in _local_vision: - try: - _local_vision[base] = bool( - _local_get("/props").json().get("modalities", {}).get("vision") - ) - except Exception: - _local_vision[base] = False + vision = _local_vision_enabled(base) store = getattr(context_provider, "__self__", None) compactions_before = getattr(store, "compaction_count", 0) model_messages = context_provider(messages) if context_provider else messages @@ -1174,7 +1188,7 @@ def _stream_local_round(messages, system_blocks, model, tool_schemas, yield {"type": "compaction", "estimated_tokens": getattr( store, "last_estimated_tokens", None)} payload = _local_request_body( - model_messages, system_blocks, tool_schemas, model, _local_vision[base] + model_messages, system_blocks, tool_schemas, model, vision ) headers = {} key = os.environ.get(LLM_API_KEY_ENV) @@ -1217,8 +1231,11 @@ def run_agent_iter( to say so whether or not anyone was listening. `context_provider` maps the full history to the bounded view sent to the - model; identity is the default. `on_message` runs immediately after each - append, allowing a durable audit to flush records as they are produced. + model; identity is the default. Local compaction notices detect a changed + `ConversationStore.compaction_count` through `context_provider.__self__`, so + that provider must be the store's bound `model_messages` method. + `on_message` runs immediately after each append, allowing a durable audit + to flush records as they are produced. `confirmation_records`, when supplied by the browser session, is sampled around each dispatch so that tool results report their human decisions. diff --git a/tests/test_context_budget.py b/tests/test_context_budget.py index e1a32477..db1d2de9 100644 --- a/tests/test_context_budget.py +++ b/tests/test_context_budget.py @@ -1,5 +1,5 @@ import json -import os +import math from pathlib import Path import httpx @@ -16,10 +16,17 @@ def recorded(name): def test_local_watermarks_use_props_and_tokenize_fixtures(monkeypatch, capsys): + applied = [] + tokenized = [] + agent._local_vision.clear() def handler(request): if request.url.path.endswith("/props"): return recorded("get_props") + if request.url.path.endswith("/apply-template"): + applied.append(json.loads(request.content)) + return recorded("post_apply_template") if request.url.path.endswith("/tokenize"): + tokenized.append(json.loads(request.content)) return recorded("post_tokenize") raise AssertionError(request.url) monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://recorded-server") @@ -27,10 +34,19 @@ def handler(request): store = agent.make_conversation_store(conversation.AuditLog(None, enabled=False)) assert store.high_water_tokens == 131072 - 8 - agent.MAX_OUTPUT_TOKENS - 4096 assert store.low_water_tokens == int(store.high_water_tokens * .75) + assert applied[0]["messages"][0]["role"] == "system" + assert applied[0]["messages"][-1] == {"role": "user", "content": "hi"} + assert applied[0]["tools"] == agent._local_request_body( + [{"role": "user", "content": "hi"}], agent._system_blocks(), + agent.TOOLS_CACHED, "budget", False, + )["tools"] + assert tokenized == [{"content": recorded("post_apply_template").json()["prompt"]}] assert "n_ctx=131072" in capsys.readouterr().out def test_local_watermarks_fallback_when_server_reports_neither(monkeypatch, capsys): + applied = [] + agent._local_vision.clear() def handler(request): if request.url.path.endswith("/props"): response = recorded("get_props") @@ -40,11 +56,16 @@ def handler(request): response = recorded("get_v1_models") return httpx.Response(response.status_code, headers=response.headers, content=response.content.replace(b',"n_ctx":131072', b'')) + if request.url.path.endswith("/apply-template"): + applied.append(json.loads(request.content)) + return httpx.Response(404, json={}) return httpx.Response(404, json={}) monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://unreported-server") monkeypatch.setattr(agent, "_local_http", httpx.Client(transport=httpx.MockTransport(handler))) store = agent.make_conversation_store(conversation.AuditLog(None, enabled=False)) - assert 32768 - 8192 - 4096 - store.high_water_tokens > 0 + fixed_fallback = math.ceil(len(json.dumps(applied[0], ensure_ascii=False, + separators=(",", ":")).encode("utf-8")) / 2.36) + assert store.high_water_tokens == max(2, 32768 - fixed_fallback - 8192 - 4096) assert store.low_water_tokens == int(store.high_water_tokens * .75) assert "n_ctx=32768 (default; server did not report n_ctx)" in capsys.readouterr().out @@ -71,21 +92,54 @@ def test_local_text_estimate_omits_tool_result_image_bytes(): assert normal_store.last_estimated_tokens - text_store.last_estimated_tokens > 40_000 +def test_local_store_image_counting_follows_shared_vision_probe(monkeypatch): + for vision in (False, True): + agent._local_vision.clear() + def handler(request): + if request.url.path.endswith("/props"): + response = recorded("get_props") + if vision: + return httpx.Response(response.status_code, headers=response.headers, + content=response.content.replace( + b'"vision":false', b'"vision":true')) + return response + if request.url.path.endswith("/apply-template"): + return recorded("post_apply_template") + if request.url.path.endswith("/tokenize"): + return recorded("post_tokenize") + return httpx.Response(404, json={}) + monkeypatch.setenv("MICROCLAW_LLM_BASE_URL", "http://recorded-server") + monkeypatch.setattr(agent, "_local_http", httpx.Client(transport=httpx.MockTransport(handler))) + store = agent.make_conversation_store(conversation.AuditLog(None, enabled=False)) + assert store.text_only is (not vision) + assert agent._local_vision["http://recorded-server"] is vision + history = [{"role": "user", "content": [{"type": "tool_result", + "tool_use_id": "image", "content": [{"type": "image", + "source": {"type": "base64", "media_type": "image/png", + "data": "A" * 1000}}]}]}] + store.model_messages(history) + assert store.last_estimated_tokens == conversation.estimate_tokens( + history, text_only=not vision + ) + + def test_local_request_body_prefix_is_stable_across_rounds(): system = [{"type": "text", "text": "stable system"}] tools = [{"name": "alpha", "description": "A", "input_schema": {"type": "object"}}] - earlier = [{"role": "user", "content": "first"}, - {"role": "assistant", "content": [{"type": "text", "text": "reply"}]}] - first = agent._local_request_body(earlier, system, tools, "m", False) - second = agent._local_request_body(earlier + [{"role": "user", "content": "next"}], - system, tools, "m", False) - left = json.dumps(first, separators=(",", ":")) - right = json.dumps(second, separators=(",", ":")) - shared = os.path.commonprefix([left, right]) - assert '"role":"system","content":"stable system"' in shared - assert json.dumps(first["tools"][0], separators=(",", ":")) in shared - for message in first["messages"]: - assert json.dumps(message, separators=(",", ":")) in shared + round_n = [{"role": "user", "content": "first"}, + {"role": "assistant", "content": [ + {"type": "text", "text": "calling alpha"}, + {"type": "tool_use", "id": "call-1", "name": "alpha", + "input": {"z": 2, "a": 1}}, + ]}, + {"role": "user", "content": [{"type": "tool_result", + "tool_use_id": "call-1", + "content": "done"}]}] + round_n_plus_one = round_n + [{"role": "user", "content": "next"}] + first = agent._local_request_body(round_n, system, tools, "m", False) + second = agent._local_request_body(round_n_plus_one, system, tools, "m", False) + assert second["messages"][:len(first["messages"])] == first["messages"] + assert second["tools"] == first["tools"] def test_compaction_event_precedes_local_request(monkeypatch): @@ -108,8 +162,10 @@ def handler(request): {"role": "assistant", "content": [{"type": "text", "text": "answer " + "y" * 100}]}, ]) from microclaw.agent import run_agent_iter + context_provider = store.model_messages + assert context_provider.__self__ is store events = run_agent_iter("new", None, None, history, model="m", tool_schemas=[], - context_provider=store.model_messages) + context_provider=context_provider) compaction = next(event for event in events if event["type"] == "compaction") assert compaction["estimated_tokens"] is not None assert not requested From 60c0bb90ff4cfc56c34aa0a4975a87b296ebd6d8 Mon Sep 17 00:00:00 2001 From: Ian Dobbie Date: Fri, 2 Oct 2026 13:43:52 -0400 Subject: [PATCH 18/53] Allow keyless local backend in browser --- microclaw/serve.html | 40 +++++++++++++++++++++++++++------------- microclaw/webserve.py | 15 +++++++++++---- tests/test_webserve.py | 38 +++++++++++++++++++++++++++++++++++++- 3 files changed, 75 insertions(+), 18 deletions(-) diff --git a/microclaw/serve.html b/microclaw/serve.html index d5043030..6cbe1254 100644 --- a/microclaw/serve.html +++ b/microclaw/serve.html @@ -162,7 +162,7 @@

Pair this browser