Skip to content

feat: onboard says how the last sign-in ended and which coding agent ran it - #188

Closed
bitpshr wants to merge 1 commit into
mainfrom
paul/onboard-signin-report
Closed

bitpshr wants to merge 1 commit into
mainfrom
paul/onboard-signin-report

Conversation

@bitpshr

@bitpshr bitpshr commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Summary

About 1 in 10 Authorize clicks in agent-first onboarding never ends with the CLI confirming the key, and about 7% of people hit it on their first try. When it happens, the CLI knows what went wrong, but it can't tell us: the sign-in never got a token, so there's no API call, and the approval page only sees the click. From the page's side, the biggest group (about 45%) looks like "the browser never reached the CLI", which could be a CLI the agent already stopped, a timeout, a port problem or something else.

This makes the CLI say how each sign-in ended, the next time it can:

  • Each sign-in's outcome gets a fixed label, kept in the project's session folder as before: timed-out, declined, port-busy, replaced, restarted, stopped (the worker went away before replying, before or after the browser step: stopped-after), exchange-connect / -timeout / -refused / -server / -other (the token exchange now names its failure), and for a sign-in that worked, how the key hand-off went (created, no-key, clipboard-error, wrong-account, unverified, ...).
  • The next approval link carries it: cli_prev=<outcome> and cli_prev_id, the first 16 hex characters of the SHA-256 of the earlier sign-in's state. The page can compute the same hash from its own link, so a report can be matched to the click it explains; the state itself never leaves the machine.
  • The next onboard requests carry it too, as a signin/<outcome> User-Agent tag. About 42% of failed clicks are followed by a new approval page within a day and 56% by a signed-in CLI call (connect, check or test), 74% by one or the other, so between the two most failures get explained, including people who give up and add a key by hand.
  • Which coding agent ran the CLI, from a variable the agent sets: cli_agent on the approval link and agent/<name> on every request. claude-code, claude-desktop, cursor, codex, gemini-cli or vscode. Claude Code's CLAUDECODE and CLAUDE_CODE_ENTRYPOINT are checked first-hand; the others come from each tool's docs and are worth confirming in the eval harness.

All of it follows the 1.8.0 telemetry switch: --telemetry off, ELEVENLABS_TELEMETRY=off or DO_NOT_TRACK=1 turns it off, and only fixed words are sent. Outcomes older than a day aren't reported. The README and --telemetry help say what's sent.

The API and the approval page need to keep the new values: the page in elevenlabs/xi#58352, the User-Agent tags in a small change next to #59230's tag parsing. Nothing breaks without them; the values are just dropped.

No version bump: this can go out with the next release.

Testing

  • cargo test: all 3,342 tests pass on macOS, including the onboard integration tests. New tests:
    • the agent label for each agent's variables, and none for a plain terminal or a blank value;
    • every way a sign-in can fail before the clipboard step maps to its label, including each token-exchange failure;
    • every reply maps to a label, unknown values become sign-in-error or other, and every label fits the API's tag pattern ([a-z-]{1,16});
    • the last sign-in is its reply, or stopped / stopped-after when the worker left none, and nothing while one is running;
    • a new sign-in keeps how the last one ended (restarted when this command stopped it), and a tampered file reports nothing;
    • the approval link and the User-Agent carry the new values, and only with telemetry on.
  • End to end with the built binary, in a throwaway home folder: with a previous sign-in that timed out and CLAUDECODE=1, onboard connect --no-browser printed an approval link with cli_agent=claude-code, cli_prev=timed-out and cli_prev_id=ba7816bf8f01cfea (the first 8 bytes of SHA-256 of that sign-in's state).
  • Not tried on Windows yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant