Repository navigation
Agent readline + history, and fix the OOM-compact no-op - #206
Merged
Merged
Conversation
…he OOM-compact no-op Two things: finishing the readline integration a prior session left half-done, and a compact bug that turned one OOM into CUDA corruption. ## readline The earlier attempt had mangled shard.yml (broke ameba's indentation, a dependency on a nonexistent lib/readline path, a stray targets block) and no real integration. I first assumed readline was stdlib because `crystal eval 'require "readline"'` succeeded -- it was not; it succeeded only because the crystal-readline shard was already installed in lib/. The shard is declared as a DEVELOPMENT dependency, since the shainet library never uses it -- only the bundled agent example does -- so consumers of the shard do not pull libreadline. read_line_edited backs the existing reader with the shard: arrow-key history, in-line editing, and a persistent ~/.shainet_agent_history capped at 1000 entries. The paste splicing and multi-line joining in read_submission are untouched -- readline only replaces the raw gets under them. On a non-tty (piped input, tests) it falls back to gets, since readline needs a terminal. The shard's API is minimal and shaped the code: readline(prompt, add_history) records live entries via the flag, but there is NO public method to add or read history back. So preloading the file seeds libreadline directly through its LibReadline.add_history binding, and a private @history array mirrors entries for saving because the list cannot be read out otherwise. Verified: preload, dedup against the last entry, blank-line skip, and the 1000-cap all hold. The REPL and the idle-interrupt handler no longer print their own "You >" -- readline draws the prompt so its editing works within it, and printing it twice would duplicate it. ## compact OOM recovery compacted to @max_context * 0.5. VRAM runs out well BELOW the context limit, so the target was above the failure point: observed OOM at 17817 tokens with a 40960 context made the target 20480, compact! returned 0 (before <= target), and the identical prompt was retried straight into an illegal memory access that corrupted CUDA state. It now targets 60% of the token count that actually failed, and if it cannot shrink below that (one huge message) it stops cleanly instead of retrying into the same wall. Gates: ameba 174/0, formatted, agent builds and links readline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
readline line editing with persistent history (crystal-readline as a dev dependency), plus a fix for OOM recovery compacting ABOVE the failure point and retrying into CUDA corruption -- now targets 60% of the failing token count and stops cleanly if it cannot shrink below it. Gates: ameba 174/0, agent builds and links readline.