Repository navigation
Conversation
…a server move freeze The server move freeze rejected every non-GET request outside /api/v1/server/move with 503 server_moving, including plugin RPCs, which are always POST. bb connect long-polls bb-account.v1.waitForStatusChange (returning at least every 25 s) and reads bb-account.v1.status and bb-account.v1.connectCredential. Its account client maps any 503 to AccountUnavailableError, so AccountLink treated the frozen server as signed out and closed the tunnel, dropping every daemon session that reaches the server through bb connect, including the move target. The move then failed at "Send data to <target>: Host is not connected". Exempt exactly those three read-only bb-account RPC paths from the freeze. Writable bb-account RPCs (fetch, adoptConnectCredential, confirmRefusedCredential, login, redeem, sign-out) stay frozen. Fixes get-bb#4880
Contributor
|
Thanks @theFong for the PR. This repo needs approval before a PR. Join the #contributors channel in Discord (https://discord.gg/kvBU6tJhcJ) and explain what you want to open a PR for and why. A maintainer then adds you to the allow list. This PR now closes on its own. Reopen it or open a new one after approval. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human comments
What was wrong
During a
bb server move,serverMoveFreezeMiddlewarerejects every/api/v1/*request that isn't GET/HEAD/OPTIONS or under/api/v1/server/movewith503 server_moving(apps/server/src/services/server-move/freeze.ts). Plugin RPC is alwaysPOST /api/v1/plugins/<id>/rpc/<method>, so the connect plugin's reads of the bb-account plugin were frozen too. The move keeps both plugins running on purpose (CONNECT_HOLD_SOURCES). Connect long-pollsbb-account.v1.waitForStatusChange, which returns at least every 25 s (LONG_POLL_TIMEOUT_MS), and then readsbb-account.v1.status. Its account client maps any 503 toAccountUnavailableError(plugins/connect/src/account-client.ts), andAccountLinktreats that as signed out and callsonAccount(null), which closes the tunnel. Every daemon that reaches the server through bb connect loses its session, including the move target, so any connect move still frozen when the current long-poll returns fails with "Send data to : Host is not connected". This has the same root cause as #4837 (the freeze blocks a bb-account RPC the move depends on), but it hits a different call. #4838 fixes only the grant ordering. This is likely also what caused the mid-move1001 tunnel disconnectedin #4394.What changed
apps/server/src/services/server-move/freeze.ts:isServerMoveFreezeExemptnow also exempts three exact paths,/api/v1/plugins/bb-account/rpc/bb-account.v1.status,.../bb-account.v1.waitForStatusChangeand.../bb-account.v1.connectCredential. Their handlers only read state:account.status(), a long-poll on the status revision, and returning the stored credential. Everything else on bb-account stays frozen:bb-account.v1.fetch,adoptConnectCredential,confirmRefusedCredential,login.*,redeemCodeandsignOut. These can write state that the export snapshot would lose. The server doesn't import plugin packages and had no constant for these method names or the bb-account plugin ID, so the paths are an exact-match set local tofreeze.ts. Look-alike methods, other plugin IDs, trailing segments and prefixed paths don't match.docs/server-move-plan.md: the "At the freeze" bullet now says the read-only bb account RPCs that bb connect polls stay available, so the tunnel stays up. Thebb guide machinestext doesn't describe what the freeze exempts, so it is unchanged.HOST_DAEMON_PROTOCOL_VERSIONchanges.This implements fix 1 from the issue. Fix 2 is left as a possible follow-up hardening: connect's account client could treat
503 server_movingas transient instead ofAccountUnavailableError, soAccountLinkkeeps the last known account and retries. This PR does not include therunStopWorkgrant-ordering change from #4838. The two PRs touch the same test file, but #4838's diff applies cleanly on top of this branch (git apply --3way --check).How you verified
New tests:
test/server-move/freeze.test.tsisServerMoveFreezeExempt).bb-account.v1.fetch,adoptConnectCredential,confirmRefusedCredential,login.start,redeemCode,signOut,bb-account.v1.statusX,bb-account.v1.status/extra,bb-account.v1.waitForStatusChange/, the same method underconnectandbb-account-evil,/api/v1/plugins/bb-account/cli, a prefixed path, andPOST /api/v1/threads.serverMoveFreezeMiddlewareandcreateServerErrorHandler).fetch,adoptConnectCredentialandconfirmRefusedCredentialstill return503 server_moving.test/server-move/coordinator.test.tscoordinator.isFrozen()as inserver.ts.waitForStatusChange,statusandconnectCredentialreturn 200 andfetchreturns 503.Fail before (new tests on
mainat 300a08b,freeze.tsunchanged):Pass after:
pnpm exec oxfmt --checkpasses on the changed TypeScript files.On my first full-suite run after the fix, one test failed: the existing "runs a direct move in step order and hands the server to the target". Its
expect.polltimed out at the default 1 s. Two re-runs passed 100/100. That test never touches a bb-account path, and on unmodifiedmainit takes about 2.5 s under the same suite load, so the failure was timing and not this change.Manual check on a real move, done while investigating #4880: this exemption (plus #4838) was patched into the shipped bb 0.45.0 bundle. A
bb server movefrom macOS to an Ubuntu machine enrolled over bb connect then completed in 39 s. There were no bb-account 503s, the tunnel stayed up for the whole freeze, and daemon sessions closed only at the switch. Without the exemption, the tunnel dropped 9 s into the freeze, when connect's in-flightwaitForStatusChangelong-poll returned into a 503, and the move failed at "Send data" (logs in #4880).Fixes #4880
Related: #4837 and #4838 (same freeze-vs-bb-account cause, different call), #4394 (likely the same mid-move tunnel drop).