Skip to content

fix(auth-cursor): stop reporting the run's usage total on replies - #4

Open
eric-x-liu wants to merge 1 commit into
UNICKCHENG:mainfrom
eric-x-liu:fix/auth-cursor-no-run-usage
Open

eric-x-liu wants to merge 1 commit into
UNICKCHENG:mainfrom
eric-x-liu:fix/auth-cursor-no-run-usage

Conversation

@eric-x-liu

Copy link
Copy Markdown

Problem

Cursor reports token usage once per run, and one run spans every tool round of a client turn: the plugin parks the run on each client tool call and resumes it when the tool result arrives. The plugin puts that run total on the final reply as if it were this request's usage (convertUsage(result.usage) in buildCompletion / buildStreamChunk). Tool-call replies carry no usage, so the last reply of a turn shows the whole turn's context counted again for every round.

To reproduce, send a ~1.3K-token prompt through CLIProxyAPI, answer three client tool calls, then get a text answer. The tool-call replies carry no usage, and the final reply carries prompt_tokens: 13490.

In an agent client on cursor/grok-4.7 (500K window), one 17-call turn whose prompts were about 40–58K each ended with:

model=cursor/grok-4.7 in=1746716 out=11236

Clients that size their context from prompt_tokens read that as overflow. This one compressed its context at the start of every message, even though the real context was about 35K.

Change

  • Replies, streaming and non-streaming, carry no usage, the same as tool-call replies already did. OpenAI-compatible callers such as LiteLLM estimate usage per request when it is missing.
  • The run total still goes to the cursor request completed log line (request_log.go is unchanged).
  • chatCompletion.Usage is removed. Tests assert that no reply has a usage field.
  • README: one line under "Other limits".

Testing

  • go vet ./..., gofmt -l . (clean), go test ./... pass on main.
  • Deployed behind LiteLLM, the same probe now shows LiteLLM's estimate on the final reply (1,320 prompt tokens), and the agent client logs normal per-call sizes with no compression on the next turn.

Trade-offs

  • CLIProxyAPI's host usage statistics no longer get Cursor token counts. The log line keeps them.
  • Through LiteLLM, a non-streaming request reports zeros rather than an estimate, because LiteLLM only estimates for streams.
  • convertUsage adds cache_read_tokens to input_tokens for the log line. I haven't checked whether Cursor's input_tokens already includes the cache reads.

🤖 Generated with Claude Code

Cursor reports token usage once per run, and one run spans every tool
round of a client turn. The plugin put that run total on the final reply
as if it were this request's usage, so it counted the turn's whole context
again for each round. Clients that size their context from usage read it as
overflow: Hermes logged in=1,746,716 for a ~58K request on a 500K-window
model and compressed its context at the start of every turn.

Replies now carry no usage, the same as tool-call replies already did.
LiteLLM and other OpenAI-compatible callers estimate it per request. The
run total stays on the `cursor request completed` log line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant