fix(openai): move to the next account when the stripped retry fails for the account's own reason

.downstream/gpt6-models-and-prompts.md

@@ -66,7 +66,7 @@ Claude OAuth 伪装系统块只由上游设置决定：设置 → 网关的 `ena
 
 侧栏“扩展功能 → 推理恢复”（`/admin/reasoning-recovery`）：一个全局开关和保存按钮，默认开启，没有账号级选项；页面打开时读取一次，读取失败时显示错误、不显示开关。存为 `settings.reasoning_recovery_config` 一行 JSON（`{"enabled": <bool>}`），没有该行即开启；`GET/PUT /api/v1/admin/reasoning-recovery` 以标准响应封装读写 `{"enabled": <bool>}`，管理员认证、无二次验证，保存记入操作审计，请求体不是严格的 `{"enabled": <bool>}` 时返回 400 且不保存。请求路径只读内存中的布尔值（保存即替换，60 秒后台刷新以同步多实例），不查询数据库，每次转发开始时读取一次。启动时读取失败或存储值不是严格的 `{"enabled": <bool>}` 则停止启动；运行中刷新失败保留当前值。
 
-开启时，OpenAI 平台的 API Key、OAuth 和 Setup Token 账号（含自动透传）在原生 Responses（含 compact）与 Chat Completions 转 Responses 的 HTTP 请求上生效：上游以结构化错误码 `invalid_encrypted_content` 或 `thinking_signature_invalid` 拒绝、且尚无语义输出时，去掉推理项的 `encrypted_content`，在同一账号重发一次：上游用 `input[N]` 指明条目时只去掉该项，否则去掉请求里全部推理项的密文。上游未用 `input[N]` 指明条目、而请求里还有其他密文载体（压缩项、多智能体 `agent_message` 的正文段）时，只有上游错误消息按条目 id 点名的是带密文的推理项才恢复，否则不恢复；这条点名规则只在推理项 id 随请求发出的路径上成立（API Key 账号与自动透传），OAuth / Setup Token 的非透传请求发出前会去掉推理项 id，遇到其他密文载体仍不恢复。其他载体的密文任何情况下都不改动。被去掉的密文按来源在 Redis 记忆 24 小时，期间同源请求发送前即去掉；记忆按每批 64 条读写，读与写共用每个请求 100 毫秒的 I/O 预算。本账号修不好时换号：在 `POST /v1/responses`（含 compact 与自动透传）上，上游报告请求里的密文无法解密或校验（结构化错误码、同类错误消息，或 `could not be decrypted` 的措辞），而本账号无法修复（不能恢复，或去掉推理密文后的重发、发送前已去掉密文的请求仍得到这类拒绝），且客户端尚未收到本次尝试的任何输出时，请求原样交给分组里的下一个账号——密文只能由签发它的上游组织解开，别的账号可能直接接受。每个请求最多因此换号 3 次（同时计入正常换号预算），下一个账号各自按上述规则恢复一次；被换下的账号不计健康惩罚，也不做同账号重试。换号上限或可选账号用尽时，客户端收到第一个报告密文问题的账号的那个错误；途中某个账号给出自己的终态（例如读得懂密文但请求本身无效、恢复重发遇到 5xx）或网关侧容量限制时，返回该结果。没有账号给出回答时，被这次换号挪到后续账号上的粘性绑定回到第一个报告密文问题的账号，调度器保留在别处的绑定不动；有账号接受或已开始输出时，会话留在该账号。普通的请求校验错误、带 `previous_response_id` 或 `conversation` 的请求、上游状态为 401/402/403/407/429 时不换号；先试过透传账号再换到非透传账号时，沿用既有规则整块去掉加密推理项。经原生 Responses 路径转发的其他平台账号同样适用；Chat Completions 转 Responses 不换号。Ops 续接诊断用 `recovery.account_mismatch` 标出这样结束的尝试。Messages 桥接与 WebSocket 不恢复。关闭时不重发也不预先去除，签名拒绝仍作为请求级终态返回，不切换账号。
+开启时，OpenAI 平台的 API Key、OAuth 和 Setup Token 账号（含自动透传）在原生 Responses（含 compact）与 Chat Completions 转 Responses 的 HTTP 请求上生效：上游以结构化错误码 `invalid_encrypted_content` 或 `thinking_signature_invalid` 拒绝、且尚无语义输出时，去掉推理项的 `encrypted_content`，在同一账号重发一次：上游用 `input[N]` 指明条目时只去掉该项，否则去掉请求里全部推理项的密文。上游未用 `input[N]` 指明条目、而请求里还有其他密文载体（压缩项、多智能体 `agent_message` 的正文段）时，只有上游错误消息按条目 id 点名的是带密文的推理项才恢复，否则不恢复；这条点名规则只在推理项 id 随请求发出的路径上成立（API Key 账号与自动透传），OAuth / Setup Token 的非透传请求发出前会去掉推理项 id，遇到其他密文载体仍不恢复。其他载体的密文任何情况下都不改动。被去掉的密文按来源在 Redis 记忆 24 小时，期间同源请求发送前即去掉；记忆按每批 64 条读写，读与写共用每个请求 100 毫秒的 I/O 预算。本账号修不好时换号：在 `POST /v1/responses`（含 compact 与自动透传）上，上游报告请求里的密文无法解密或校验（结构化错误码、同类错误消息，或 `could not be decrypted` 的措辞），而本账号无法修复（不能恢复，或去掉推理密文后的重发、发送前已去掉密文的请求仍得到这类拒绝），且客户端尚未收到本次尝试的任何输出时，请求原样交给分组里的下一个账号——密文只能由签发它的上游组织解开，别的账号可能直接接受。每个请求最多因此换号 3 次（同时计入正常换号预算），下一个账号各自按上述规则恢复一次；被换下的账号不计健康惩罚，也不做同账号重试。换号上限或可选账号用尽时，客户端收到第一个报告密文问题的账号的那个错误；途中某个账号给出自己的终态（例如读得懂密文但请求本身无效）或网关侧容量限制时，返回该结果。没有账号给出回答时，被这次换号挪到后续账号上的粘性绑定回到第一个报告密文问题的账号，调度器保留在别处的绑定不动；有账号接受或已开始输出时，会话留在该账号。普通的请求校验错误、带 `previous_response_id` 或 `conversation` 的请求、上游状态为 401/402/403/407/429 时不换号；先试过透传账号再换到非透传账号时，沿用既有规则整块去掉加密推理项。经原生 Responses 路径转发的其他平台账号同样适用；Chat Completions 转 Responses 不换号。Ops 续接诊断用 `recovery.account_mismatch` 标出因密文读不了而交给下一个账号的尝试。恢复重发因账号自身的原因失败时也换号（同样只在 `POST /v1/responses`，含 compact 与自动透传；Chat Completions 转 Responses 不换号）：重发已发出、失败属于该路径首次发送时就会换号的那一类，且客户端尚未收到本次尝试的任何输出时，请求按普通账号失败交给下一个账号——下一个账号收到原样请求，本账号不再重试，换号预算同普通换号。哪些失败算这一类以各路径首次发送的换号规则为准：非透传包括额度用尽、限流、鉴权失败、5xx 与流内的上游失败；自动透传的规则更窄，OAuth / Setup Token 透传的普通 5xx、不带账号状态信息的 401/403 等不在其内，仍在本账号结束。Ops 续接诊断用 `recovery.account_failure` 标出这样交给下一个账号的尝试。账号健康：失败的分类与首次发送相同（请求预算被拒 402/403 `balance_insufficient` 只算本次请求，结构化的 `model_not_supported` 只算该账号与模型，`budget_exceeded` 429 算账号，其余池模式可重试的状态不计入 API Key 健康熔断），调度失败照常上报，流内失败照常写入账号状态；重发的 HTTP 错误应答则不执行转发层在首次发送时的账号状态写入（标记限流、置为错误、停用、预算用尽标记、API Key 的瞬时失败计数），也不开启 OAuth 429 的同账号重试窗口，留给之后的首次发送处理，其中首次发送因同时停用账号而把池模式可重试状态计入账号的情形按不计入处理。处理器侧同账号重试用尽后的冷却照常调用，它只对 OAuth / Setup Token 账号起作用（硬额度 429 会写入额度状态）；OpenAI API Key 账号在 HTTP 错误应答下不被冷却，下一个请求仍可能先发给它，由那次首次发送写入状态。重发遇到请求校验错误、拒绝类结果、传输错误、响应头超时或未归类为可换号的错误时，仍在本账号结束；重发再次被报告密文无法解密时，按上文的密文规则换号。Messages 桥接与 WebSocket 不恢复。关闭时不重发也不预先去除，签名拒绝仍作为请求级终态返回，不切换账号。
 
 迁移 `267_purge_account_reasoning_options.sql` 一次性删除各账号 `extra` 中原有的两个账号级推理选项键（键名见该文件）；此后没有代码读写它们，已存的账号级取值不带入全局开关。Chat Completions 转 Responses 的请求只由客户端消息转换而来，网关不保存响应中的推理项。
 

backend/internal/handler/openai_continuation_account_mismatch_test.go

@@ -26,7 +26,10 @@ type continuationMismatchUpstream struct {
 	bodies     [][]byte
 	accepting  map[int64]bool
 	replies    map[int64]continuationMismatchReply
-	rejection  continuationMismatchReply
+	// sequence scripts an account's answers call by call; when it runs out the
+	// account falls back to replies and rejection.
+	sequence  map[int64][]continuationMismatchReply
+	rejection continuationMismatchReply
 }
 
 type continuationMismatchReply struct {
@@ -44,12 +47,18 @@ func (u *continuationMismatchUpstream) Do(req *http.Request, _ string, accountID
 	u.mu.Lock()
 	u.accountIDs = append(u.accountIDs, accountID)
 	u.bodies = append(u.bodies, body)
+	var next *continuationMismatchReply
+	if pending := u.sequence[accountID]; len(pending) > 0 {
+		next, u.sequence[accountID] = &pending[0], pending[1:]
+	}
 	u.mu.Unlock()
-	if u.accepting[accountID] {
+	if next == nil && u.accepting[accountID] {
 		return openAIOfficialHTTPSuccess(openAIOfficialHTTPAttempt{accountID: accountID, path: req.URL.Path, body: body}), nil
 	}
 	reply, scripted := u.replies[accountID]
-	if !scripted {
+	if next != nil {
+		reply = *next
+	} else if !scripted {
 		reply = u.rejection
 	}
 	if reply.stream != "" {
@@ -125,9 +134,14 @@ func newContinuationMismatchHandler(t *testing.T, upstream service.HTTPUpstream,
 }
 
 func serveContinuationMismatch(t *testing.T, h *OpenAIGatewayHandler, body string) *responseRecorderResult {
+	t.Helper()
+	return serveContinuationMismatchSession(t, h, body, "one-conversation")
+}
+
+func serveContinuationMismatchSession(t *testing.T, h *OpenAIGatewayHandler, body, session string) *responseRecorderResult {
 	t.Helper()
 	c, recorder := newReplayableEndpointContext(t, openAIOfficialHTTPGroupID, http.MethodPost, "/v1/responses", []byte(body))
-	c.Request.Header.Set("session_id", "one-conversation")
+	c.Request.Header.Set("session_id", session)
 	h.Responses(c)
 	return &responseRecorderResult{code: recorder.Code, body: recorder.Body.String()}
 }
@@ -155,6 +169,10 @@ func continuationMismatchBoundAccount(t *testing.T, cache *openAIOfficialHTTPSti
 // whose body is ciphertext the gateway cannot remove.
 const continuationMismatchBody = `{"model":"gpt-5.1","stream":false,"input":[{"type":"reasoning","id":"rs_a","encrypted_content":"cipher-reasoning","summary":[]},{"type":"agent_message","id":"amsg_b","author":"/root","recipient":"/root/worker","content":[{"type":"input_text","text":"Payload:"},{"type":"encrypted_content","encrypted_content":"cipher-agent"}]}]}`
 
+// A history whose only ciphertext is reasoning: any rejection of it can be
+// repaired by the stripped retry, also where reasoning ids are not sent.
+const continuationMismatchReasoningOnlyBody = `{"model":"gpt-5.1","stream":false,"input":[{"type":"reasoning","id":"rs_a","encrypted_content":"cipher-reasoning","summary":[]},{"type":"message","role":"user","content":[{"type":"input_text","text":"continue"}]}]}`
+
 func TestOpenAIGatewayHandler_ContinuationCiphertextMismatchMovesToAnAccountThatAccepts(t *testing.T) {
 	gin.SetMode(gin.TestMode)
 	for _, test := range []struct {
@@ -277,3 +295,95 @@ func TestOpenAIGatewayHandler_ContinuationCiphertextMismatchKeepsTheSessionOnAnA
 	require.Contains(t, response.body, "partial-answer")
 	require.Equal(t, int64(2), continuationMismatchBoundAccount(t, cache))
 }
+
+// The stripped retry can fail for a reason of the account's own. The answer is
+// then the same as on a first send: the request goes to the next account, which
+// receives it unchanged, and the first account is not asked a third time.
+func TestOpenAIGatewayHandler_ContinuationStrippedRetryAccountFailureMovesToTheNextAccount(t *testing.T) {
+	gin.SetMode(gin.TestMode)
+	for _, test := range []struct {
+		name        string
+		stream      bool
+		accountType string
+		body        string
+		first       continuationMismatchReply
+		retry       continuationMismatchReply
+	}{
+		// Production, 2026-10-05: the retry is accepted at HTTP level and fails in the stream.
+		{"credit_exhausted_in_stream", true, service.AccountTypeAPIKey, continuationMismatchBody, continuationMismatchNamedReasoning,
+			continuationMismatchReply{status: http.StatusOK, stream: "event: response.failed\ndata: {\"type\":\"response.failed\",\"response\":{\"id\":\"resp_failed\",\"status\":\"failed\",\"error\":{\"code\":\"credit_balance_exhausted\",\"message\":\"You have no credits remaining. do-not-leak-billing\"}}}\n\n"}},
+		{"payment_required", false, service.AccountTypeAPIKey, continuationMismatchBody, continuationMismatchNamedReasoning,
+			continuationMismatchReply{status: http.StatusPaymentRequired, payload: `{"error":{"type":"insufficient_quota","code":"insufficient_quota","message":"do-not-leak-billing"}}`}},
+		// An OAuth account would otherwise get a same-account retry for a 5xx.
+		{"server_error_on_oauth", true, service.AccountTypeOAuth, continuationMismatchReasoningOnlyBody, continuationMismatchUnrecoverable,
+			continuationMismatchReply{status: http.StatusServiceUnavailable, payload: `{"error":{"type":"server_error","message":"do-not-leak-provider"}}`}},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			upstream := &continuationMismatchUpstream{
+				accepting: map[int64]bool{2: true},
+				sequence:  map[int64][]continuationMismatchReply{1: {test.first, test.retry}},
+				rejection: continuationMismatchUnrecoverable,
+			}
+			handler, cache := newContinuationMismatchHandler(t, upstream, continuationMismatchAccounts(2, test.accountType))
+			body := test.body
+			if test.stream {
+				body = strings.Replace(body, `"stream":false`, `"stream":true`, 1)
+			}
+			response := serveContinuationMismatch(t, handler, body)
+
+			require.Equal(t, []int64{1, 1, 2}, upstream.accountIDs, "first send, stripped retry, then the next account")
+			require.False(t, gjson.GetBytes(upstream.bodies[1], "input.0.encrypted_content").Exists(), "the retry on the first account is the stripped one")
+			require.Equal(t, gjson.GetBytes(upstream.bodies[0], "input").Raw, gjson.GetBytes(upstream.bodies[2], "input").Raw, "the next account receives the request unchanged")
+			require.Equal(t, http.StatusOK, response.code, response.body)
+			require.NotContains(t, response.body, "do-not-leak")
+			require.NotContains(t, response.body, "response.failed")
+			if test.stream {
+				require.Contains(t, response.body, "response.completed")
+			} else {
+				require.Contains(t, response.body, "healthy answer")
+			}
+			require.Equal(t, int64(2), continuationMismatchBoundAccount(t, cache))
+		})
+	}
+}
+
+// An account failure of the stripped retry is an ordinary account failure, not
+// a mismatch: when nobody answers, the last account's failure is the answer and
+// the session stays where selection put it.
+func TestOpenAIGatewayHandler_ContinuationStrippedRetryAccountFailureIsNotAMismatch(t *testing.T) {
+	gin.SetMode(gin.TestMode)
+	upstream := &continuationMismatchUpstream{
+		sequence: map[int64][]continuationMismatchReply{1: {continuationMismatchNamedReasoning,
+			{status: http.StatusPaymentRequired, payload: `{"error":{"type":"insufficient_quota","code":"insufficient_quota","message":"do-not-leak-billing"}}`}}},
+		replies: map[int64]continuationMismatchReply{2: {status: http.StatusServiceUnavailable, payload: `{"error":{"type":"server_error","message":"do-not-leak-provider"}}`}},
+	}
+	handler, cache := newContinuationMismatchHandler(t, upstream, continuationMismatchAccounts(2, service.AccountTypeAPIKey))
+	response := serveContinuationMismatch(t, handler, continuationMismatchBody)
+
+	require.Equal(t, []int64{1, 1, 2}, upstream.accountIDs)
+	require.GreaterOrEqual(t, response.code, http.StatusInternalServerError, response.body)
+	require.NotEqual(t, service.OpenAIContinuationStateUnavailableCode, gjson.Get(response.body, "error.code").String())
+	require.NotContains(t, response.body, "do-not-leak")
+	require.Equal(t, int64(2), continuationMismatchBoundAccount(t, cache), "no mismatch, so no binding is moved back")
+}
+
+// An OAuth account that failed on its stripped retry gets the cooldown of an
+// account whose same-account attempts are used up: after a 401 the next request
+// is not sent to it. (For an OpenAI API-key account that cooldown does nothing.)
+func TestOpenAIGatewayHandler_ContinuationStrippedRetryAccountFailureCoolsTheAccount(t *testing.T) {
+	gin.SetMode(gin.TestMode)
+	upstream := &continuationMismatchUpstream{
+		accepting: map[int64]bool{1: true, 2: true},
+		// An authentication failure blocks the whole account for a while, which
+		// the next selection can be seen to respect.
+		sequence: map[int64][]continuationMismatchReply{1: {continuationMismatchUnrecoverable,
+			{status: http.StatusUnauthorized, payload: `{"error":{"type":"authentication_error","message":"credentials rejected"}}`}}},
+	}
+	handler, _ := newContinuationMismatchHandler(t, upstream, continuationMismatchAccounts(2, service.AccountTypeOAuth))
+	body := strings.Replace(continuationMismatchReasoningOnlyBody, `"stream":false`, `"stream":true`, 1)
+	serveContinuationMismatch(t, handler, body)
+	require.Equal(t, []int64{1, 1, 2}, upstream.accountIDs)
+
+	serveContinuationMismatchSession(t, handler, body, "another-conversation")
+	require.Equal(t, []int64{1, 1, 2, 2}, upstream.accountIDs, "account 1 would accept again, but is cooling")
+}