Skip to content

fix: read vLLM-SR's and llama.cpp's too-long errors as too long - #40

Open
subin9 wants to merge 1 commit into
osolmaz:mainfrom
subin9:fix/too-long-errors
Open

subin9 wants to merge 1 commit into
osolmaz:mainfrom
subin9:fix/too-long-errors

Conversation

@subin9

@subin9 subin9 commented Oct 30, 2026

Copy link
Copy Markdown

When a request is too long, switch asks the view function for a smaller view and tries again. That only happens when TOO_LONG in src/models/pi-ai.ts matches the error, and it misses what two System One servers send, so the step just fails.

vLLM-SR answers 400 with code state_too_long. llama.cpp answers 400 exceed_context_size_error ("request (N tokens) exceeds the available context size") for a causal model like Kev-0.8B, and 500 "input (N tokens) is too large to process" for an encoder like Julia-1, whose whole input has to fit in one batch. The tests use the messages pi-ai returned for each of them. This adds context size, too[ _]long and too large to process to the pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant