Skip to content

Local model - #62

Open
iandobbie wants to merge 53 commits into
Micro-Claw:mainfrom
iandobbie:local-model
Open

iandobbie wants to merge 53 commits into
Micro-Claw:mainfrom
iandobbie:local-model

Conversation

@iandobbie

Copy link
Copy Markdown
Collaborator

Adding the ability to connect to a local server using the openAI API. Tested extensivley with QWEN 3.8-28B

iandobbie and others added 30 commits October 23, 2026 16:56
…oposed)

Notebook only, no code. Records the 2026-10-01 llama.cpp/Qwen spike against
9e9271a, decisions D1-D8, blocks 84a/84b, gate 84, and two operator questions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…elps)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recorded by the coordinator from a real server; the runner sandbox has no
network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t the end)

Codex revision-002 addressed G1-G7 and was killed by the coordinator's
2-hour background limit during its final verification step, after its
targeted run (665 passed, 2 sandbox socket failures). Committed by the
coordinator as found; not yet reviewed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0ecbbcc moved the store's construction after the key load so the budget
probe is authenticated; _add_audit_secret skipped the not-yet-built store,
so a stored key was no longer redacted from the history JSONL. Coordinator
fix (CLAUDE.md step 7); the test fails on 0ecbbcc for exactly that reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One driven session on the llama.cpp workstation with Micro-Manager's demo
config. The scorer reads the session's history and usage files; the budget
limb reads the server's n_ctx itself and asks make_conversation_store, and
reports NOT EXERCISED when it cannot. Selftest: a real run_agent_iter /
ConversationStore / AuditLog session against recorded llama.cpp responses,
with a mutant per graded limb (12 passed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…required

befe1a7 made setBusy read keyRequired, declared further down with the key UI.
The recovery JS harness lifts setBusy alone and raised ReferenceError; in the
page every call runs after the declaration, but the state belongs with hasKey.
A /api/key payload without key_required is now read as required (the
Anthropic rule), not as optional. Coordinator fix (CLAUDE.md step 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-1 test asserted the max(2, ...) clamp that 0ecbbcc (L4) replaced
with an announced 4096-token floor; the runner's targeted run did not
include this file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dows

Git for Windows' default core.autocrlf rewrote them to CRLF at checkout and
every replay failed (gate 84 selftest: 3 failed, 9 errors on the workstation;
reproduced here by converting a copy).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… clone

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed for D9

The budget limb skipped compaction records and so passed gate 84 round 1,
whose checkpoints were committed above high-water (F2). It is now two graded
limbs - every request fits n_ctx with the reply reserve, and no compaction is
committed above high-water - plus a report of requests D9 sent as is. Re-scored
on round 1's real artifacts: 8/9, failing the D9 limb at records 10 and 11.
The selftest's good session is resized so a fold can clear high-water (at
1200/600 none can under D9). Also a corrected comment in _stream_local_round.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same session and server settings as round 1, re-runnable on the machine that
ran round 1 (the safety bounds step no longer refuses an existing file), O4
rewritten for 84c's two toasts, evidence to block84c-evidence. Pinned to f26df3e.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
iandobbie and others added 20 commits October 23, 2026 17:08
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n notes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
design/84 84a imports httpx at the top of microclaw/agent.py, but httpx was
declared only in the test extra; it had arrived transitively through
anthropic, which now depends on httpx2. A fresh non-editable .[serve] install
- what install.bat builds - raised ModuleNotFoundError on import of
microclaw.agent, on either backend. The gates installed .[serve,test], so
none saw it. Measured and fixed 2026-10-06; the new test fails on the old
pyproject for exactly that reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PowerShell clears a variable by assigning ''; 9372a89 forwarded that as an
effort of "", which the measured template rejects. Coordinator fix; the test
fails on 9372a89 for exactly that reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le notices, console errors

From gate 84 round 3 (workstation, 2026-10-06): the compaction toast showed
'6,106.033 calibrated tokens'; persistent notices closed on the click that
selects their text, so they could not be copied; and the serve window printed
only 'seq 5 error' for a context-exceeded refusal. Rebuild tokens are rounded at
the source, persistent notices close from their own button and allow text
selection, and the console prints an error event's message. The served-page
test checks the notice wiring as text; nothing in the suite executes the button,
and the console error line has no test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the select does not offer

The shared budget-refresh helper had been inserted between two import
statements. And a startup MICROCLAW_LLM_REASONING_EFFORT outside the four
offered levels (Qwen's own xhigh, say) left the select showing nothing, so the
control misstated the level in force; it is now listed as the startup setting.
Coordinator fixes (CLAUDE.md step 7); the page assertion fails on e55761a.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured against the workstation 2026-10-22: switching to medium re-processed
the whole prompt (33,831 tokens), switching back to the default re-processed 4,
because this llama.cpp build kept the earlier prompt cached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@iandobbie

Copy link
Copy Markdown
Collaborator Author

Current PR has references to design/84* which conflicts with an exisitng plan point 84. I have moved some of the refernce to #91 but not all. This also affects the 70-carried-forward-register.md where I shifted the issues found to points 160-163.

I will clean up the branch to make the references all point to desgin/91 but it is a bit of work.

@iandobbie

Copy link
Copy Markdown
Collaborator Author

Ok, I think I have updated most of it. I think some references to the carry forward register are out of sync as I had to renumber mine as new ones had been added inthe last few weeks. I dont think this will affect the code in any way.

…ex R160-R164

R161 seen again at the default limit=50 (Qwen3.5-9B, n_ctx 65536): a 70,881-token
request refused; the recent-turn floor kept the bulk unfoldable. Fix direction and
follow-up ownership recorded. R17's MEDIUM rested on the browser showing the
error; it is a self-clearing toast and the operator saw nothing, so HIGH.
R165: a reply with only reasoning ends the turn silently and is stored empty.
R160-R164 added to the provenance and work-queue tables.
@zacsimile

Copy link
Copy Markdown
Member

Tested d91eaec on our demo machine (RTX 3070 Ti, 8 GB) with Qwen3.5-4B (Q8_0, -c 131072) and Qwen3.5-9B (Q4_K_M, -c 65536) under llama.cpp. The local path works: every call went to the local server, the key flow, prompt-cache reuse and the server-derived budget all behaved, and the gate scored 7/9 both times. The two misses were the compaction limbs, which were never reached. The 9B was clearly the more usable of the two.

We found a few things that we'll handle ourselves after merge, so there's nothing to change in this PR. I've pushed a register commit to this branch (2118d15) so they're tracked:

  • R161 happened again. With the 9B at 64k, the default 50-entry read_hook_log (~16k tokens) was read in turn 3 and again in turn 4, and the server refused a 70,881-token request. Compaction couldn't help, because main's rule never folds the two most recent turns. The row now has this second occurrence and a fix direction.
  • R17 is raised to HIGH. The refusal appeared only as a toast that clears itself, so the operator saw nothing.
  • New R165. In 3 of 4 turns the 4B produced only reasoning, so the turn ended with no visible answer and an empty assistant message was stored. The 9B didn't do this.
  • R160–R164 are now in the register's two tables, where they were missing.

One thing about the gate script: its "every request fit" check reads the usage records, and a refused request never gets one, so it reported PASS in the session where a request was refused.

Evidence (history, usage, gate, llama-server and serve logs) on request.

New docs/local-llm-model.md: serving a model with llama.cpp (what each
llama-server flag is for, with the ones Microclaw needs), the three
MICROCLAW_LLM_* settings and how to set them persistently (not in
updater-launcher.ps1, which every install replaces), startup, what to
expect, troubleshooting, and the models tested. Linked from the README
docs table; browser-gui.md gains a paragraph on the key chip, model chip
and reasoning-level selector with a local model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants