Repository navigation
ci(mobile): require Flutter tests to run and pass on app PRs, including forks - #21432
Conversation
Require successful selected Dart coverage before handling deferred heavy jobs, run the ordinary suite every nightly, and close recovered suite failure issues. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Budget four additional fork Dart runs per day and preserve the cheap-tier step summary while requiring successful selected and nightly Flutter coverage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c75eb64ff6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if [[ "$result" == success ]]; then | ||
| if [[ -n "$issue_number" ]]; then | ||
| gh issue close "$issue_number" --repo "$REPO" --reason completed |
There was a problem hiding this comment.
Because the workflow’s concurrency group includes github.run_id for every non-PR event, main pushes and nightly runs can overlap. If an older successful run reaches this branch after a newer failing run has created or updated the same suite issue, it closes the issue even though the latest result is red; the reverse ordering can also reopen an issue after recovery. Serialize these reconciliations or compare run/SHA recency before closing the issue.
Useful? React with 👍 / 👎.
Report passed Dart coverage, use exact issue titles from the paginated API, exercise recovery and skipped-push behavior, and align the cap rationale with the approved forecast. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 participant
Problem
The 2026-10-13 contributor audit found 46 merged fork app PRs without Dart coverage over its full September 13–October 13 window. Since the September 28
ci:fullgate, 39 of 59 fork app PRs merged without Dart (28 skipped, 11 absent). Seven of those PRs predate the gate. Five red Flutter tests reached main in the October 13 merges, and the next non-app push reported green because it skipped the suite.Changes
Dart Analyze & Testsjob for selected app changes on every PR, including forks withoutci:full, and every nightly. It runs the analyzer ratchet and regular Flutter suite.Mobile Release Eligibilityunderalways(). Before fork deferral or general skip handling, require selected/nightly Dart coverage to concludesuccess; skipped, failed, cancelled, and missing results fail.ci:fullexplanation, the CI budget forecast and Linux cap (13,550 → 13,620 minutes/day), and tier-contract assertions while retaining the cheap-tier summary notice.This repairs the existing shared mobile eligibility primitive rather than adding another guard or workflow. It would have blocked the skipped Dart coverage on #20798 and the missing coverage on #21302 once required.
Zero-job root cause
Fork workflow approval expired. The run page's actual annotation on 38304462940 and 38233046442 says: “This workflow run required approval but was not approved before it expired.”
gh run viewcalls this a likely workflow-file issue, but the API shows failure with zero jobs/check runs and no reusable workflow references. Run 38304462940 and successful same-repo 38446875839 use the identical workflow blob419ae0624cea5d087415c283635505937d68fcf8.The same expired-approval annotation was verified for #21324 (38276262726), #21340 (38301133647), #21341 (38302471715), #21343 (38303715580), #20869 (37670747921), #20451 (37122531710), and #15088 (35480665984). This is fork approval expiry, not a fork-only YAML rejection.
action_requiredis the pending approval state; expiry turns it into failure without jobs. The exact organization approval policy remains unreadable with the current token. No policy or branch-protection changes are included.For maintainers
After this lands and Mobile Release Eligibility becomes a required check on main, an app PR cannot merge until the Flutter tests have actually run and passed. Contributors no longer need
ci:fullfor those tests.ci:fullstill turns on the iOS and emulator jobs, plus the other existing expensive fork checks. For first-time or outside contributors whose runs require GitHub approval, maintainers must click Approve and run before jobs can start. Once eligibility is required, a PR with an unapproved run stays pending because the required eligibility check has not been created, and that blocks merging. This is the intended outcome. Expired approval also leaves the required check missing and blocks merging.The coordinator will make
Mobile Release Eligibilitya required status check on main after merge. This PR does not change branch protection or rulesets.To verify forks after merge, use a fork app PR whose merge ref includes this workflow revision, without
ci:full. Approve its pending GitHub workflow run if needed, then verifyDart Analyze & Testsruns andMobile Release Eligibilityreflects its result on the exact head SHA. Existing runs may retain the old workflow: rerunning an old run alone is not proof of the new gate. Coordinate an author update/rebase when necessary; do not push to their fork. A genuinely failing Dart run must yield failed eligibility. Unrelated PRs should keep passing eligibility with Dart skipped. No other contributors' labels, comments, forks, or workflow runs were changed in this lane.Runner cost and risks
The audit estimated 59 fork app PRs in 15 days, about four/day, each up to the existing 30-minute Ubuntu timeout: roughly 120 extra Linux runner-minutes/day at the timeout ceiling, plus one regular Dart nightly (up to 30 minutes/day). No new macOS work. At the inventory's retained 17-minute p50, four additional PR runs/day estimate 68 Linux minutes/day; repeated pushes/reruns can increase this.
Existing open fork app PRs can become blocked once the coordinator requires the gate; flaky Flutter tests will now block merges. Fork workflow approval remains necessary where GitHub requires it. Green nightlies close recovered suite issues; skipped suites do not erase failure history. This still does not run regular Flutter on every unrelated main push.
Validation
actionlint .github/workflows/mobile-app-checks.yml: passed.GITHUB_STEP_SUMMARYdeferral notice, nightly error text, and actual issue-bot Bash with a mockghfor recovery closure, skipped-push preservation, and non-matching-title preservation.python3 .github/scripts/check_ci_budget.py --refreshand budget validation: passed; Linux cap is 13,620 minutes/day.secrets.*, OIDC, protected environment, or write permission in the Dart job. The main/nightly issue writer remains guarded from PR runs.OMI_PR_BODY_FILE=/tmp/ci-fork-gate-pr-body.md make preflight: 26 checks passed.scripts/pr-preflight --pr-body-file /tmp/ci-fork-gate-pr-body.md: 26 checks passed. History-dependent failure-class ratchet explicitly skips because the checkout is shallow.04026662f0737d62ced830b06df82aef08c65dbd: Mobile App Checks run 38453891951 completed successfully. Detect Changes, Generated Files, Dart Analyze & Tests, Seeded Journeys (hermetic), Android Compile Smoke, Android JVM Unit Tests, Android Emulator Acceptance, iOS Compile Check, and Mobile Release Eligibility all succeeded. Dart Tests (Pacific/Kiritimati) and Post-merge mobile failure issues were correctly skipped on this PR. This PR changes three.github/files and the contribution guide, with no app source changes.Product invariants affected
none
Failure-Class: none
🤖 Generated with Claude Code