Skip to content

feat: onboard waits longer for approval, retries an unreachable token endpoint, and reports how sign-ins end - #175

Open
bitpshr wants to merge 2 commits into
mainfrom
paul/onboard-1-7-2
Open

bitpshr wants to merge 2 commits into
mainfrom
paul/onboard-1-7-2

Conversation

@bitpshr

@bitpshr bitpshr commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Since 1.7.1, the Windows misses we fixed are mostly gone. Three onboard problems are left in the sign-in, on every OS. This fixes them:

  • Late approvals. The sign-in waited 5 minutes. About 2.5% of Authorize clicks come later than that, and every one of them reached nothing. Of the 88 clicks more than 5 minutes after the page opened since Oct 19, 55 were within 15 minutes and 80 within 30. onboard now waits 30 minutes. auth login still waits 5.
  • A token endpoint that can't be reached. About 2% of sign-ins end on server_error without a grant, at the same rate in every country. Many take about 11 s, which is the 10 s connect timeout, and a retry in a new tab usually works. When the token request never connected, the code wasn't used, so the CLI now tries once more after 1 s. A request that did connect is never sent again. If both tries fail to connect, the error page gets temporarily_unavailable instead of server_error, so these failures can be counted apart from the rest.
  • A limited key called another account's. When /v1/user refused the copied key (401 or 403), onboard told the agent the key "belongs to a different account". A key without user_read is refused there but works for everything else. It's now could_not_verify ("the key couldn't be checked, so it wasn't stored"). "A different account" is kept for a key that does open another account.

It also makes the CLI say how each sign-in ended (this was #188). About 1 in 10 Authorize clicks still never ends with the CLI confirming the key, and the biggest group (about 45%) looks from the page like "the browser never reached the CLI", which could be a CLI the agent already stopped, a timeout, a port problem or something else. The CLI knows, but can't tell us: that sign-in never got a token.

  • Each sign-in's outcome gets a fixed label, kept in the project's session folder: timed-out, declined, port-busy, replaced, restarted, stopped (the worker went away before replying; stopped-after once the browser step was done), exchange-connect / -timeout / -refused / -server / -other (the token exchange now names its failure; exchange-connect is the one retried above), and for a sign-in that worked, how the key hand-off went (created, no-key, clipboard-error, wrong-account, unverified, ...).
  • The next approval link carries it: cli_prev=<outcome> and cli_prev_id, the first 16 hex characters of the SHA-256 of the earlier sign-in's state. The page computes the same hash from its own link, so a report can be matched to the click it explains; the state itself never leaves the machine.
  • The next onboard requests carry it too, as a signin/<outcome> User-Agent tag. About 42% of failed clicks are followed by a new approval page within a day and 56% by a signed-in CLI call, 74% by one or the other, so between the two most failures get explained, including people who give up and add a key by hand.
  • Which coding agent ran the CLI, from a variable the agent sets: cli_agent on the approval link and agent/<name> on every request. claude-code, claude-desktop, cursor, codex, gemini-cli or vscode. Claude Code's CLAUDECODE and CLAUDE_CODE_ENTRYPOINT are checked first-hand; the others come from each tool's docs and are worth confirming in the eval harness.

The reporting follows the 1.8.0 telemetry switch (--telemetry off, ELEVENLABS_TELEMETRY=off or DO_NOT_TRACK=1 turns it off), and only fixed words are sent. Outcomes older than a day aren't reported. The README and --telemetry help say what's sent. The page and the API keep the new values once elevenlabs/xi#58352 merges; until then they're just dropped.

Rebased on main (1.9.0). No version bump: the SDK regeneration sets it now, so this goes out with the next release.

Testing

  • cargo test: all 3,345 tests pass on macOS, including the onboard integration tests. New or changed tests:
    • an endpoint that's unreachable at first gets the code on the second try, exactly once;
    • one that stays unreachable is reported as unreachable (and labelled connect), after the retry;
    • a code that reached the endpoint is never sent again, whether the endpoint refused it (labelled refused) or closed without answering;
    • a refused key is "couldn't check", and a key that opens another account is still "different account";
    • the agent label for each agent's variables, and none for a plain terminal or a blank value;
    • every way a sign-in can fail before the clipboard step maps to its label, including each token-exchange failure;
    • every reply maps to a label, unknown values become sign-in-error or other, and every label fits the API's tag pattern ([a-z-]{1,16});
    • the last sign-in is its reply, or stopped / stopped-after when the worker left none, and nothing while one is running;
    • a new sign-in keeps how the last one ended (restarted when this command stopped it), and a tampered file reports nothing;
    • the approval link and the User-Agent carry the new values, and only with telemetry on.
    • Breaking the retry rule both ways (retry after any error, never retry) makes the retry tests fail.
  • End to end on macOS with real Chrome: the token endpoint was down when the callback arrived and came up 400 ms later. This branch signed in after about 1 s; 1.7.1 showed server_error after 20 ms.
  • End to end with the built binary, in a throwaway home folder: with a previous sign-in that timed out and CLAUDECODE=1, onboard connect --no-browser printed an approval link with cli_agent=claude-code, cli_prev=timed-out and cli_prev_id=ba7816bf8f01cfea.
  • Windows probe on a windows-latest runner with real Chrome and Edge, on the retry and longer-wait code (before the rebase and the reporting were added):
    • Every earlier scenario (other programs on the port, older sign-ins, old tabs, agent jobs, idle and empty connections, reserved ports, Deny) gives the same result as on main.
    • Token endpoint down when the callback arrives, up 3 s later: this branch signed in, while 1.7.1 showed server_error. With only a 400 ms outage, both signed in, because Windows itself keeps retrying a refused connect for about 2 s.

A side effect of the longer wait: a sign-in nobody approves keeps its link and its port for up to 30 minutes. Running onboard connect again in that project shows the same link, as it did within the 5 minutes before.

@bitpshr
bitpshr requested a review from PaulAsjes October 23, 2026 14:29
@bitpshr
bitpshr marked this pull request as ready for review October 23, 2026 14:30
@bitpshr
bitpshr requested a balanced review from Copilot October 23, 2026 14:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review identified no blocking issues in the focused reliability changes or regression coverage; tests were not executed during this review.

No open issues 🎉

What changed in this PR

This PR improves onboarding sign-in reliability and error reporting in the ElevenLabs CLI for version 1.7.2.

Changes:

  • Extends onboarding approval waits to 30 minutes while leaving auth login at 5 minutes.
  • Retries token requests once after connection failures and distinguishes unreachable endpoints from other failures.
  • Reports unverifiable clipboard keys separately from account mismatches and adds regression coverage.
File Description
cli/​elevenlabs/​workflow/​onboard/​tests.rs Updates timeout expectations and tests key-verification outcomes.
cli/​elevenlabs/​workflow/​onboard/​session.rs Extends the onboarding approval wait to 30 minutes.
cli/​elevenlabs/​workflow/​onboard/​connect.rs Logs why clipboard-key verification failed.
cli/​elevenlabs/​workflow/​onboard/​auth.rs Distinguishes unidentified accounts from confirmed account mismatches.
cli/​elevenlabs/​workflow/​loopback.rs Adds connection-failure retries, error classification, and regression tests.
Cargo.toml Bumps the package version to 1.7.2.
Cargo.lock Synchronizes the locked package version.

Review effort: Balanced

…oken endpoint again

- The sign-in waits 30 minutes instead of 5. Some people approve 10 to 30
  minutes after the page opens, and those clicks reached nothing.
- A token request that never connected is sent once more after 1 s; the code
  never got there, so it's still good. A request that connected is never sent
  again. When both tries fail to connect, the error page gets
  `temporarily_unavailable` instead of `server_error`.
- A copied key that /v1/user refuses is "couldn't be checked", not "belongs to
  a different account": a key without user_read is refused there but works
  otherwise. The reason goes to the worker's log.
@bitpshr
bitpshr force-pushed the paul/onboard-1-7-2 branch from 37fc273 to 5020e6c Compare November 6, 2026 16:36
@bitpshr bitpshr changed the title fix: onboard waits longer for the approval and tries an unreachable token endpoint again (1.7.2) feat: onboard waits longer for approval, retries an unreachable token endpoint, and reports how sign-ins end Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants