diff --git a/.automation/backport-skip b/.automation/backport-skip new file mode 100644 index 00000000000000..45f4777c562617 --- /dev/null +++ b/.automation/backport-skip @@ -0,0 +1,11 @@ +# Edera commits on edera/mainline that must never be backported to +# edera/6.18-lts, one subject per line, exactly as the commit has it. +# +# Each pull request merged into edera/mainline is backported on its own +# (.github/workflows/lts-backport.yml); label it no-backport to opt out. This +# file is for commits that did not arrive through such a pull request, or +# were ported to 6.18 in a form the subject match cannot see. A commit that +# is listed here, or belongs to a no-backport pull request, does not hold up +# later backports that touch the same files. +# +# Blank lines and lines starting with # are ignored. diff --git a/.automation/skills/mainline-backport/SKILL.md b/.automation/skills/mainline-backport/SKILL.md new file mode 100644 index 00000000000000..fe1d9506994b16 --- /dev/null +++ b/.automation/skills/mainline-backport/SKILL.md @@ -0,0 +1,206 @@ +--- +name: mainline-backport +description: Backport of one merged edera/mainline pull request to edera/6.18-lts, run unattended in CI by .github/workflows/lts-backport.yml. Covers the cherry-picks, adapting a commit to the older kernel, spotting a dependency on another backport, build checks, and the report that becomes the backport pull request. +--- + +# Mainline pull request backport + +Edera kernel work lands on `edera/mainline` first, one pull request at a time, +and every merged pull request is proposed for `edera/6.18-lts` as its own +backport pull request. You are preparing one of those. The workflow has +already worked out which of the pull request's commits the LTS tree still +lacks and hands you that list. + +The goal is that each backport changes **exactly what its mainline commit +changes**. Where 6.18 differs from mainline enough that the same change does +not apply, you adapt it the way a careful stable maintainer would and you say +exactly what you did. + +This runs unattended. Nobody is watching while you work, and nobody can answer +a question. Everything a reviewer needs has to end up in the report. + +## What happens to your result + +You cannot push, and nothing you write lands on `edera/6.18-lts`: a maintainer +merges the backport pull request, or does not. When you finish, the workflow +independently: + +1. runs `verify-backport.sh`, which checks from git alone that your branch + fast-forwards the LTS tip, is linear, and holds one commit per requested + commit, in order, each naming its source with a `(cherry picked from + commit )` trailer, keeping its source's subject, and changing the + same files by the same +/- lines; +2. builds x86_64 and arm64 `vmlinux` with `kernel-build.sh`. + +Both results go into the pull request next to your report. A commit you had +to adapt **will** show up as adapted there. That is expected, not a failure; +your report explains it. Never reword a subject, merge two commits into one, +or split one into two: backports are recognised later by subject, and a +renamed one would be offered again. + +## Inputs + +The workflow gives you, in the prompt: + +- `MAINLINE_PR`: the number and title of the merged pull request. +- `OLD_TIP`: the `edera/6.18-lts` commit to build on (already checked out). +- `REQUEST`: a file listing the pull request's commits to backport, one per + line as ` TAB `, oldest first. Their objects are already + fetched. Commits the LTS tree already has, and automation-only commits, are + not in it. +- `PENDING`: a file listing every Edera commit on `edera/mainline` that the + LTS tree lacks, in the same format, oldest first. The request's own commits + are among them. The workflow has already checked that nothing earlier in + this list touches the request's files; you look for the dependencies a file + match cannot see. +- `RESULT`: the local branch name your result must end up on. +- `REPORT`: the path your report must be written to. + +The automation's own scripts are in `.rebase-tools/scripts/`, copied from the +commit this run started from. Use those, never the copy inside the tree. + +### How to run commands here + +Your shell commands are checked against an allowlist that matches the +command's first word. Shell variables, `$(...)`, `<(...)` and `VAR=x cmd` +prefixes will be refused, so `OLD_TIP`, `RESULT`, `` and `<...>` below +are placeholders: substitute the literal value. Put scratch files, worktrees +and build output under `.rebase-scratch/` in the repository root; the +kernel's `.gitignore` ignores it. + +This is a blobless clone: file contents are fetched from GitHub the first time +a command needs them, so the first checkout or `git show` can be slow. That is +not a hang. + +## 1. Read the request + +```sh +cat REQUEST +cat PENDING +git show --stat # for each requested commit +``` + +For each commit, read its message and diff, and check what it depends on: +does it call a helper, use a struct field or a Kconfig symbol that 6.18 +lacks? If so, where does that come from? + +- From an earlier commit in the request: fine, it is backported first. +- From a commit in `PENDING` that is not in the request: this backport has to + **wait** for that one. Stop here; see "If it has to wait". +- From upstream (Linus' tree) only: see the next section. + +## 2. Cherry-pick, in order + +```sh +git switch -c RESULT OLD_TIP +git cherry-pick -x # one at a time, oldest first +``` + +`-x` is required: the trailer it adds is how the checker pairs your commit +with its source. If you have to amend, keep the trailer and the subject. + +When a pick conflicts, or applies but would not build or behave the same on +6.18: + +- Read what differs between the two kernels around the change, and adapt the + change so it does on 6.18 what it does on mainline. Prefer the smallest + adaptation; do not backport an unrelated upstream refactor to make the + patch apply. +- If the conflict is with Edera code that only a pending commit brings, + that is a dependency: **wait**, as above. +- If the commit depends on an upstream change 6.18 lacks and cannot be + adapted without it, do not pull that upstream change in. Leave the commit + out (`git cherry-pick --abort` for that one, then continue with the next) + and make it a **Needs a decision** item saying what it needs. +- If the commit does not apply to 6.18 at all (it fixes code only mainline + has), leave it out the same way and say so. +- If you cannot tell what the right adaptation is, make the most conservative + one you can defend, mark it **UNSURE** in the report, and explain both + readings. +- Never use `-X ours`, `-X theirs`, or `git checkout --ours/--theirs` on a + whole file to make a conflict go away. +- Never commit anything that is not a backport of a requested commit. If a + backport needs a 6.18-only fix to build, fold it into that backport and + report it as an adaptation. + +## 3. Build + +```sh +bash .rebase-tools/scripts/kernel-build.sh x86_64 .rebase-scratch/obj-x86 4 +bash .rebase-tools/scripts/kernel-build.sh arm64 .rebase-scratch/obj-arm64 4 +``` + +If a build fails because of a backport, fix it **in that backport**: drive the +todo list with sed, `git -c sequence.editor="sed -i 's/^pick /edit +/'" rebase -i OLD_TIP`, amend (keeping subject and trailer), then `git +rebase --continue`. Never rebase below OLD_TIP. + +If a build fails on OLD_TIP too, it is not yours: build OLD_TIP the same way +in a `.rebase-scratch/` worktree to confirm, and report it as pre-existing. + +## 4. Check yourself + +```sh +bash .rebase-tools/scripts/verify-backport.sh OLD_TIP RESULT REQUEST +``` + +Every commit it lists as adapted or left out must be accounted for in your +report. If one is not, you changed something you did not mean to: find it and +undo it. + +## 5. Report + +Write the report to the `REPORT` path, in Markdown. Its **first line** is the +status, exactly one of: + +- `Status: proposed` when the RESULT branch holds a backport for review; +- `Status: waiting` when it has to wait for another backport (no RESULT + branch); +- `Status: not-applicable` when none of the request applies to 6.18 (no + RESULT branch). + +The rest becomes the pull request body (or the comment on the mainline pull +request when there is no branch), so lead with what a reviewer must decide. +No preamble, no sign-off. + +```markdown +Status: proposed + +## Summary +One paragraph: commits requested, backported unchanged, adapted, left out, +and whether you expect the checker to pass. + +## Needs a decision +One bullet per question, each starting **UNSURE:**, with both readings and +what you committed. Every left-out commit is one. "None." if there is +nothing. + +## Adapted commits +Per commit: subject, what differs on 6.18, what you changed and why. "None." +if none. + +## Left out +Per commit: subject and why. "None." if none. + +## Builds +x86_64, arm64: pass, fail (with the first error), or pre-existing failure +(confirmed on OLD_TIP). +``` + +Then stop. The workflow reads the branch, not the working tree. + +## If it has to wait + +Do not create the RESULT branch. Write the report with `Status: waiting`, +and under `## Waiting on` list each pending commit it needs, as `` `` + ``, with one line on what it needs from it. The workflow labels the +mainline pull request `backport-waiting` and tries again once other backports +land. + +## If nothing applies, or you cannot finish + +If none of the request applies to 6.18, do not create the RESULT branch and +write the report with `Status: not-applicable`, explaining why. If you could +not produce a backport for any other reason, do not create the branch either +and explain what went wrong; the first line is then whatever fits best, and +a missing branch is treated as needing a human either way. diff --git a/.automation/skills/upstream-rebase/SKILL.md b/.automation/skills/upstream-rebase/SKILL.md new file mode 100644 index 00000000000000..5f22a00da6d391 --- /dev/null +++ b/.automation/skills/upstream-rebase/SKILL.md @@ -0,0 +1,246 @@ +--- +name: upstream-rebase +description: Nightly rebase of edera/6.18-lts onto stable linux-6.18.y, run unattended in CI by .github/workflows/upstream-rebase.yml. Covers the replay, conflict resolution, build checks, the audit of upstream fixes that meet the series, and the report the workflow publishes. +--- + +# Nightly upstream rebase + +You are rebasing the Edera downstream kernel series (around 80 commits: +Hyper-V-nested-on-Xen, the Xen PV-IOMMU, NUMA-aware Xen backends, 2 MiB +ballooning, OpenPaX, and assorted fixes) on `edera/6.18-lts` onto the stable +`linux-6.18.y` branch. The goal is a replay that changes **nothing downstream**: every +difference in the final tree must come from the upstream delta alone. Where +that is impossible, because upstream and downstream touched the same code, you +resolve the conflict the way a careful kernel maintainer would and you say +exactly what you did. + +This runs unattended. Nobody is watching while you work, and nobody can answer +a question. Everything a reviewer needs has to end up in the report. + +## What happens to your result + +You cannot push, and nothing you write can cause the branch to be pushed. +Your report can only stop it. When you finish, the workflow independently: + +1. runs `verify-upstream-rebase.sh` against your branch, which checks from git + alone that the series sits on the upstream tip, is linear, lost or gained + no commit, and changed the tree by exactly the upstream delta; +2. builds x86_64 and arm64 `vmlinux` with `kernel-build.sh`, which turns on + every downstream feature as a built-in. + +If all of that passes **and your report's "Needs a decision" section is +`None.` with nothing marked UNSURE anywhere** (checked by +`report-needs-decision.sh`), the downstream branch is force-pushed to your +branch. Otherwise your branch goes up as a pull request with your report as +the body, and a maintainer reads it. + +So a question you raise is never lost to an automatic push. Raise one whenever +a maintainer should see something before the branch moves, even when the +rebase itself is clean, for example an upstream change that reaches a +downstream feature without touching the same lines. Do not use the word UNSURE +for anything else. + +So: a conflict you resolve **will** show up as drift and **will** go to a +human. That is the intended outcome, not a failure. Do not try to make a +resolution look clean, and never drop, squash, reorder or reword a downstream +commit to get past the checks. A rebase that reaches review with an honest +report is a success; one that hides a judgement call is not. + +## Inputs + +The workflow gives you, in the prompt: + +- `DOWNSTREAM`: the branch being rebased, `edera/6.18-lts`. +- `OLD_TIP`: its current commit (already checked out). +- `UPSTREAM`: the upstream commit to rebase onto, fetched as the local branch + `upstream-target`. +- `RESULT`: the local branch name your result must end up on. +- `REPORT`: the path your report must be written to. + +The automation's own scripts are in `.rebase-tools/scripts/`, copied from the +default branch. Use those, never a copy inside the tree you are rebasing: the +tree is what is under test. + +### How to run commands here + +Your shell commands are checked against an allowlist that matches the +command's first word. Shell variables, `$(...)`, `<(...)` and `VAR=x cmd` +prefixes will be refused, so in every snippet below `OLD_TIP`, `UPSTREAM`, +`RESULT`, `MB` and `<...>` are placeholders: substitute the literal SHA or +path, or a path you choose. Put scratch files, worktrees and build output +under `.rebase-scratch/` in the repository root (`mkdir -p .rebase-scratch`); +the kernel's `.gitignore` ignores it and nothing reads it after you. + +This is a blobless clone: file contents are fetched from GitHub the first time +a command needs them, so the first `git log -p` or checkout over a range is +slower than you expect. That is not a hang. + +## 1. Survey + +```sh +git merge-base OLD_TIP UPSTREAM # call the result MB +git rev-list --count --no-merges MB..OLD_TIP # downstream commits +git log --oneline --no-merges MB..UPSTREAM # what is new upstream +``` + +The upstream range is large (one or more stable releases, often hundreds of +commits each), so do not read it all. Find where it meets the +series: + +```sh +git diff --name-only --output=.rebase-scratch/down.files MB..OLD_TIP +git diff --name-only --output=.rebase-scratch/up.files MB..UPSTREAM +comm -12 .rebase-scratch/down.files .rebase-scratch/up.files # both sorted already +git log --oneline --no-merges MB..UPSTREAM -- +``` + +Read the messages and diffs of those upstream commits. They are where a +textually clean replay can still be semantically wrong: a helper downstream +calls that changed its locking or return convention, a struct field that +moved, a Kconfig symbol that was renamed. Note them; you come back to them in +step 4. + +The series may contain a merge commit from an old pull request. Rebasing +flattens it into its commits, which is expected and the checker accepts it; +do not use `--rebase-merges`. + +## 2. Rebase + +```sh +git switch -c RESULT OLD_TIP +git rebase --onto UPSTREAM MB RESULT +``` + +When a commit conflicts: + +- Read the upstream change that caused it **and** the downstream commit's + intent (its message, and the rest of its diff). Resolve so the downstream + commit does what it did before, on top of what upstream now does. +- Upstream wins on fixes. If a stable backport changed the code a downstream + commit edits, keep every check, lock, ordering constraint + and error path the fix introduced, and fit the downstream change around it. +- If upstream now contains the downstream change itself (it was upstreamed and + backported to stable), let the downstream commit go empty and let git drop + it. Record that, with the upstream commit. +- If you cannot tell what the right resolution is, do not guess silently. Make + the most conservative resolution you can defend, mark it **UNSURE** in the + report, and explain both readings. +- Never use `-X ours`, `-X theirs`, `--skip`, or `git checkout --ours/--theirs` + on a whole file to make a conflict go away. + +For each conflict, record: the downstream commit (subject), the files, the +upstream commit it collided with, and in a sentence or two what you kept and +why. + +## 3. Build + +The workflow re-runs these builds itself, but run them here so you can fix +what your resolutions broke. Each takes a while; run them once the rebase is +complete, not after every commit. + +```sh +bash .rebase-tools/scripts/kernel-build.sh x86_64 .rebase-scratch/obj-x86 4 +bash .rebase-tools/scripts/kernel-build.sh arm64 .rebase-scratch/obj-arm64 4 +``` + +Exit status 3 means a downstream option no longer survives `olddefconfig`: +upstream renamed or re-depended a Kconfig symbol the series relies on (or that +the build script names). Find out which, and say so in the report; if the +build script's list is what is stale, that is a **Needs a decision** item, +not something to work around. + +If a build fails because of the rebase (a resolution, or an upstream change a +downstream commit has not caught up with), fix it **in the downstream commit +that is wrong**. Interactive editors do not work here, so drive the todo list +with sed: `git -c sequence.editor="sed -i 's/^pick /edit /'" rebase +-i UPSTREAM`, amend, then `git rebase --continue`. Do not add a fix-up commit +on top unless there is no single commit it belongs to; if you must, its +subject starts with `rebase: ` and the report says why. + +If a build fails on the pristine upstream tree too, it is not yours: build +UPSTREAM the same way in a `.rebase-scratch/` worktree to confirm, and report +it as pre-existing. + +## 4. Audit what upstream brought in + +For every upstream commit flagged in step 1, check that its change **survived +the replay**. Ancestry is not enough: a downstream commit replayed on top can +edit the very lines a fix added. + +Every stable commit is a fix; those with a `CVE-` reference matter most. For +each one that touched a file the series also touches: + +```sh +git log --oneline UPSTREAM..RESULT -- +``` + +If that returns anything, read the **final tree**, not the commit graph: +confirm the fix's checks and ordering are intact and that downstream code +added after it is covered by them. Grep tree-wide for any function or flag +the fix deliberately removed, in case a downstream commit brought it back. + +Also look for semantic collisions that did not conflict: an upstream change +to a function's contract (locking, refcounting, return values, a new required +call) where a downstream commit calls that function. Each one you cannot rule +out is an UNSURE item. + +## 5. Check yourself + +Run the same checker the workflow will run, so the report can explain its +result rather than be surprised by it: + +```sh +bash .rebase-tools/scripts/verify-upstream-rebase.sh OLD_TIP RESULT UPSTREAM +``` + +Every drift line and every changed commit it lists must be accounted for in +your report by a conflict resolution, a dropped-empty commit or a build fix. +If one is not, you made a change you did not mean to: find it and undo it. + +## 6. Report + +Write the report to the `REPORT` path, in Markdown. It becomes a pull request +body when a human has to look, so lead with what they must decide. No +preamble, no sign-off. + +```markdown +## Summary +One paragraph: the tree, upstream range (N commits, short SHAs and the +`git describe` of each end), downstream commits replayed, conflicts resolved, +commits dropped, and whether you expect the checker to pass. + +## Needs a decision +One bullet per question, each starting **UNSURE:**, with both readings and +what you committed. "None." if there is nothing; anything else here stops the +automatic push. + +## Conflicts resolved +Per conflict: downstream commit, files, colliding upstream commit, what you +kept and why. "None." if none. + +## Dropped downstream commits +Subject and the upstream commit that made it empty. "None." if none. + +## Build fixes +What broke, which commit you fixed it in, why. "None." if none. + +## Upstream commits that meet the series +`short-sha subject` for each upstream commit that touched a file the series +touches, and for each fix among them your reading of the final tree. "None." +if none. + +## Builds +x86_64, arm64: pass, fail (with the first error), or pre-existing failure +(confirmed on pristine upstream). +``` + +Then stop. Leave RESULT checked out or not; the workflow reads the branch, +not the working tree. + +## If you cannot finish + +If the rebase cannot be completed (a conflict you cannot resolve at all, a +toolchain that will not run), run `git rebase --abort`, do not create the +RESULT branch, and write the report explaining where you stopped and why. The +workflow treats a missing RESULT branch as a failed night and opens an issue +with your report. diff --git a/.github/scripts/backport-blockers.sh b/.github/scripts/backport-blockers.sh new file mode 100755 index 00000000000000..6588478a0fcf96 --- /dev/null +++ b/.github/scripts/backport-blockers.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +# Works out what a merged edera/mainline pull request still needs on +# edera/6.18-lts, and what it has to wait for first. +# +# Backports to edera/6.18-lts land in the order their commits landed on +# edera/mainline. A pull request is backported only once every Edera commit +# that came before it on mainline, is still missing from the LTS tree, and +# touches a file it touches has been backported (or skipped). Otherwise its +# commits could be applied on top of code they were never written against. +# +# Usage: +# backport-blockers.sh +# +# is what pending-backports.sh prints for the whole of +# edera/mainline: the mainline commits the LTS tree lacks, oldest first. +# lists the pull request's own commits the same way (" +# TAB ", oldest first). A request commit still needs backporting when +# it matches a pending commit, and those are written to : +# +# - by patch-id first, which is how a rebase-merged commit normally matches +# its copy on mainline; +# - otherwise, if the LTS series does not already have its patch-id, by +# subject, ignoring a trailing version tag. The patch-id check keeps a +# commit that is already on the LTS tree from claiming a different +# pending commit that happens to share its subject. +# +# Prints the pending commits the pull request has to wait for, oldest first, +# and nothing when it can go ahead. + +set -euo pipefail + +if [ $# -ne 5 ] || [ ! -r "$1" ] || [ ! -r "$2" ]; then + echo "usage: $0 " >&2 + exit 2 +fi +pending=$1 +request=$2 +l_tip=$(git rev-parse --verify "$3^{commit}") +l_base=$(git merge-base "$l_tip" "$(git rev-parse --verify "$4^{commit}")") +todo=$5 + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +norm() { sed -E 's/[[:space:]]*\((v?[0-9]+\.[0-9]+(\.[0-9y]+)?(-lts)?)\)$//'; } +pid() { git show "$1" | git patch-id --stable | cut -d' ' -f1; } +files() { git show --format= --name-only "$1"; } + +git log --no-merges --format='commit %H' --patch "$l_base..$l_tip" | + git patch-id --stable | cut -d' ' -f1 | sort -u >"$work/lts-pids" + +# Pending commits, numbered, with patch-ids. +n=0 +while IFS=$'\t' read -r sha subject; do + n=$((n + 1)) + printf '%s\t%s\t%s\t%s\n' "$n" "$sha" "$subject" "$(pid "$sha")" +done <"$pending" >"$work/pending" + +# Each request commit claims the first pending commit it matches that nothing +# else has claimed. +: >"$todo" +: >"$work/claimed" +while IFS=$'\t' read -r sha subject; do + s=$(printf '%s' "$subject" | norm) + p=$(pid "$sha") + by=subject + grep -qxF -- "$p" "$work/lts-pids" && by=none + hit=$(awk -F'\t' -v s="$s" -v p="$p" -v by="$by" ' + FILENAME == ARGV[1] { claimed[$0] = 1; next } + ($1 in claimed) { next } + $4 == p { print $1; found = 1; exit } + by == "subject" && $3 == s && !first { first = $1 } + END { if (!found && first) print first } + ' "$work/claimed" "$work/pending") + if [ -n "$hit" ]; then + echo "$hit" >>"$work/claimed" + printf '%s\t%s\n' "$sha" "$s" >>"$todo" + files "$sha" >>"$work/files" + fi +done <"$request" +[ -s "$todo" ] || exit 0 +sort -u -o "$work/files" "$work/files" +first=$(sort -n "$work/claimed" | head -n1) + +awk -F'\t' -v first="$first" ' + FILENAME == ARGV[1] { claimed[$0] = 1; next } + $1 < first && !($1 in claimed) { print $2 "\t" $3 } +' "$work/claimed" "$work/pending" | while IFS=$'\t' read -r sha subject; do + if files "$sha" | grep -qxFf "$work/files"; then + printf '%s\t%s\n' "$sha" "$subject" + fi +done diff --git a/.github/scripts/kernel-build.sh b/.github/scripts/kernel-build.sh new file mode 100755 index 00000000000000..2a4c38dd68cb80 --- /dev/null +++ b/.github/scripts/kernel-build.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# Builds the kernel the way the nightly automation judges a branch. +# +# defconfig alone compiles almost none of the Edera series: Hyper-V nesting, +# Xen dom0 and backends, the PV-IOMMU, OpenPaX and the NUMA-aware Xen paths +# are all off in it. This turns them on as built-ins and builds vmlinux, so a +# replay that breaks any of them, or links against a symbol upstream removed, +# fails here. Building only a subdirectory would miss those link errors. +# +# The rebase skill runs this same script before they finish, so +# what Claude checks and what the workflow checks cannot drift apart. +# +# Usage: +# kernel-build.sh [jobs] +# +# Run from the top of a kernel tree. is created if missing and may be +# reused between runs. For arm64 it uses CROSS_COMPILE (default +# aarch64-linux-gnu-) unless LLVM is set, in which case it builds with clang. +# +# Exit status: make's on a build failure; 3 if an option it asked for did not +# survive olddefconfig (a Kconfig dependency the series no longer satisfies). + +set -euo pipefail + +if [ $# -lt 2 ] || [ $# -gt 3 ]; then + echo "usage: $0 [jobs]" >&2 + exit 2 +fi +arch=$1 +objdir=$2 +jobs=${3:-$(nproc)} + +common=( + XEN XEN_BACKEND XEN_DOM0 XEN_BALLOON XEN_GNTDEV XEN_GRANT_DEV_ALLOC + NUMA ACPI_NUMA MEMORY_HOTPLUG MEMORY_HOTREMOVE ZONE_DEVICE + XEN_UNPOPULATED_ALLOC XEN_BACKEND_NUMA_AFFINITY + XEN_NETDEV_FRONTEND XEN_NETDEV_BACKEND XEN_BLKDEV_FRONTEND XEN_BLKDEV_BACKEND + NET_9P NET_9P_XEN 9P_FS + DRM DRM_VIRTIO_GPU FW_CFG_SYSFS + SECURITY OPENPAX OPENPAX_SOFTMODE OPENPAX_XATTR_PAX_FLAGS OPENPAX_MPROTECT + OPENPAX_EMUTRAMP +) +case $arch in +x86_64) + karch=x86 + # make reads LLVM from the environment; x86 is always the gcc build. + unset LLVM + cross=() + opts=( + "${common[@]}" + HYPERVISOR_GUEST PARAVIRT XEN_PV XEN_PVHVM_GUEST PCI_XEN + XEN_PCIDEV_FRONTEND XEN_PCIDEV_BACKEND XEN_IOMMU + HYPERV HYPERV_VMBUS HYPERV_NET PCI_HYPERV PCI_HYPERV_INTERFACE + ) + ;; +arm64) + karch=arm64 + if [ -n "${LLVM:-}" ]; then + cross=(LLVM=1) + else + cross=(CROSS_COMPILE="${CROSS_COMPILE:-aarch64-linux-gnu-}") + fi + opts=("${common[@]}") + ;; +*) + echo "unknown arch: $arch" >&2 + exit 2 + ;; +esac + +mkdir -p "$objdir" +mk() { make -s ARCH="$karch" "${cross[@]}" O="$objdir" "$@"; } + +mk defconfig +enable=() +for o in "${opts[@]}"; do enable+=(--enable "$o"); done +scripts/config --file "$objdir/.config" "${enable[@]}" +mk olddefconfig + +# scripts/config writes whatever it is told; olddefconfig then quietly drops +# anything whose dependencies are not met. Catch that, or a Kconfig change +# could switch a downstream feature off without failing the build. +lost=() +for o in "${opts[@]}"; do + grep -qx "CONFIG_$o=y" "$objdir/.config" || lost+=("$o") +done +if [ "${#lost[@]}" -ne 0 ]; then + echo "kernel-build: not enabled after olddefconfig ($arch): ${lost[*]}" >&2 + exit 3 +fi + +mk -j"$jobs" vmlinux +echo "kernel-build: $arch vmlinux built in $objdir" diff --git a/.github/scripts/pending-backports.sh b/.github/scripts/pending-backports.sh new file mode 100755 index 00000000000000..c6ae95a0669378 --- /dev/null +++ b/.github/scripts/pending-backports.sh @@ -0,0 +1,80 @@ +#!/usr/bin/env bash +# Lists the Edera commits on edera/mainline that edera/6.18-lts does not have. +# +# Edera kernel work lands on edera/mainline first and flows down to +# edera/6.18-lts. Both trees are rebased onto their upstream nightly, so commit +# hashes are no use for telling which commits already made it across. A commit +# counts as present on the LTS tree when its series there has: +# +# - a commit with the same subject, ignoring a trailing version tag such as +# " (6.18)" that a port may add (the k-th mainline commit with a subject +# pairs with the k-th LTS one, so repeated subjects are counted, not +# merged); or +# - a commit with the same patch-id, which catches a port whose subject was +# reworded but whose change was not. +# +# A commit that must never reach the LTS tree (it fixes code only mainline +# has, say) is listed by subject in the skip file, one per line (the +# backport workflow adds the commits of mainline pull requests labelled +# no-backport); blank lines +# and lines starting with # are ignored. Commits that change only .github/ or +# .automation/ are never offered: both trees carry the automation, it is +# landed on each by hand, and the backport is not allowed to touch it. +# +# Usage: +# pending-backports.sh [] +# +# Prints " TAB " for each pending commit, oldest first, and +# nothing when the trees agree. Merge commits are ignored on both sides: their +# content is in the commits they merged. + +set -euo pipefail + +if [ $# -lt 4 ] || [ $# -gt 5 ]; then + echo "usage: $0 []" >&2 + exit 2 +fi + +m_tip=$(git rev-parse --verify "$1^{commit}") +m_base=$(git merge-base "$m_tip" "$(git rev-parse --verify "$2^{commit}")") +l_tip=$(git rev-parse --verify "$3^{commit}") +l_base=$(git merge-base "$l_tip" "$(git rev-parse --verify "$4^{commit}")") +skip=${5:-/dev/null} + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +# TAB TAB , oldest first. +series() { + git log --no-merges --reverse --format='commit %H' --patch "$1" | + git patch-id --stable | awk '{ print $2 "\t" $1 }' >"$work/pid.map" + git log --no-merges --reverse --format='%H%x09%s' "$1" | + awk -F'\t' 'NR == FNR { pid[$1] = $2; next } + { + s = $2 + sub(/[ \t]*\((v?[0-9]+\.[0-9]+(\.[0-9y]+)?(-lts)?)\)$/, "", s) + print $1 "\t" s "\t" ($1 in pid ? pid[$1] : "-") + }' "$work/pid.map" - +} +series "$m_base..$m_tip" >"$work/mainline" +series "$l_base..$l_tip" >"$work/lts" +# Mainline commits whose every changed path is automation. +git log --no-merges --format='commit %H' --name-only "$m_base..$m_tip" | + awk '/^commit / { if (c != "" && only) print c; c = $2; only = 0; seen = 0; next } + NF { if (!seen) only = 1; seen = 1; if ($0 !~ /^\.(github|automation)\//) only = 0 } + END { if (c != "" && only) print c }' >"$work/automation" +grep -vE '^[[:space:]]*(#|$)' "$skip" | sed -E 's/^[[:space:]]+|[[:space:]]+$//g' >"$work/skip" || true + +awk -F'\t' ' + FILENAME == ARGV[1] { auto[$0] = 1; next } + FILENAME == ARGV[2] { skip[$0] = 1; next } + FILENAME == ARGV[3] { have[$2]++; if ($3 != "-") bypid[$3] = $2; next } + { + if (($1 in auto) || ($2 in skip)) next + # A patch-id match uses up the LTS commit it matched, under the + # subject that commit has there, so it cannot also pair with another. + if ($3 in bypid) { have[bypid[$3]]--; delete bypid[$3]; next } + if (have[$2] > 0) { have[$2]--; next } + print $1 "\t" $2 + } +' "$work/automation" "$work/skip" "$work/lts" "$work/mainline" diff --git a/.github/scripts/report-needs-decision.sh b/.github/scripts/report-needs-decision.sh new file mode 100755 index 00000000000000..e15b51979fd3b7 --- /dev/null +++ b/.github/scripts/report-needs-decision.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# Decides whether the nightly rebase report asks a human for a decision. +# +# The upstream-rebase skill tells Claude to list anything it was unsure about +# under a "## Needs a decision" section, each item marked UNSURE. A rebase can +# be clean by every mechanical measure (no drift, builds pass) and still raise +# such a question: an upstream change that interacts with a downstream feature +# without touching the same lines. The workflow does not push those +# automatically; it opens a pull request so a maintainer sees the question +# before the branch moves. +# +# The report counts as asking for a decision when either: +# - its "Needs a decision" section has any content other than "None.", or +# - any line marks something UNSURE. +# Both err towards a pull request: a stray "UNSURE" costs one review, a missed +# question costs an unreviewed kernel change. +# +# Usage: +# report-needs-decision.sh +# +# Prints the section's content when it asks for a decision. Exit status: +# 0 a decision is needed +# 1 no decision is needed +# 2 the report cannot be read + +set -euo pipefail + +if [ $# -ne 1 ] || [ ! -r "$1" ]; then + echo "usage: $0 " >&2 + exit 2 +fi + +awk ' + BEGIN { insec = 0; body = ""; unsure = 0 } + /UNSURE/ { unsure = 1 } + /^##?[ \t]/ { + if (tolower($0) ~ /^##?[ \t]+needs a decision[ \t]*$/) { insec = 1; next } + insec = 0 + } + insec { + line = $0 + gsub(/^[ \t]+|[ \t]+$/, "", line) + if (line == "") next + if (tolower(line) ~ /^(_?none\.?_?|n\/a\.?)$/) next + body = body $0 "\n" + } + END { + if (body != "" || unsure) { printf "%s", body; exit 0 } + exit 1 + } +' "$1" diff --git a/.github/scripts/retry-backports.sh b/.github/scripts/retry-backports.sh new file mode 100755 index 00000000000000..5eaca535f0b508 --- /dev/null +++ b/.github/scripts/retry-backports.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# Dispatches backports of merged edera/mainline pull requests again. +# +# - Every mainline pull request labelled backport-waiting: something it was +# waiting for may have landed on edera/6.18-lts since. +# - With --stale, also every open backport pull request GitHub can no +# longer merge: a rebase of edera/6.18-lts rewrote the history it was +# built on, so it is prepared again from the new tip. +# +# Each one is a run of kernel-nightly.yml on edera/6.18-lts with backport_pr +# set (lts-backport.yml). Needs GH_TOKEN with actions write and pull requests +# read, and GITHUB_REPOSITORY. +# +# Usage: retry-backports.sh [--stale] +# +# Nothing to retry is the usual case and exits 0. + +set -euo pipefail + +SOURCE=edera/mainline +TARGET=edera/6.18-lts +PREFIX=backport/edera-6.18-lts/pr- +R=$GITHUB_REPOSITORY + +{ + gh pr list --repo "$R" --base "$SOURCE" --state merged --label backport-waiting \ + --limit 200 --json number -q '.[].number' + if [ "${1:-}" = --stale ]; then + gh pr list --repo "$R" --base "$TARGET" --state open --limit 200 \ + --json headRefName,mergeable \ + -q ".[] | select(.headRefName | startswith(\"$PREFIX\")) | select(.mergeable == \"CONFLICTING\") | .headRefName" | + sed "s|^$PREFIX||" + fi +} | { grep -xE '[0-9]+' || true; } | sort -un | while read -r n; do + echo "Dispatching the backport of #$n" + gh workflow run kernel-nightly.yml --repo "$R" --ref "$TARGET" -f backport_pr="$n" +done diff --git a/.github/scripts/tests/backport.test.sh b/.github/scripts/tests/backport.test.sh new file mode 100755 index 00000000000000..9c650aba454d5b --- /dev/null +++ b/.github/scripts/tests/backport.test.sh @@ -0,0 +1,225 @@ +#!/usr/bin/env bash +# Tests for .github/scripts/pending-backports.sh, backport-blockers.sh and +# verify-backport.sh. +# +# Builds a small throwaway repository with two upstreams (an "lts" line and a +# "mainline" line that has moved on), an Edera series on each, and checks +# that pending-backports.sh finds exactly the mainline commits the LTS series +# lacks, and that backport-blockers.sh holds a pull request back only for +# earlier pending commits that touch its files. It then backports them +# honestly and in each way the checker exists to catch. A backport is only +# proposed on verify-backport.sh's say-so, so every damaged backport must be +# flagged or refused and the honest one must pass. +# +# Run from anywhere: bash .github/scripts/tests/backport.test.sh +set -uo pipefail + +HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +PENDING="$HERE/../pending-backports.sh" +VERIFY="$HERE/../verify-backport.sh" +BLOCKERS="$HERE/../backport-blockers.sh" + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT +mkdir "$WORK/repo" && cd "$WORK/repo" || exit 1 + +failures=0 +ok() { echo "ok $1"; } +bad() { + echo "FAIL $1" >&2 + [ -z "${2:-}" ] || printf ' %s\n' "${2//$'\n'/$'\n' }" >&2 + failures=$((failures + 1)) +} + +git init -q -b lts-up . +git config user.name test +git config user.email test@example.invalid +commit() { git add -A && git commit -qm "$1"; } + +# Upstreams: mainline is the LTS base plus an unrelated change. +seq 1 60 >a.c +seq 1 60 >b.c +commit "upstream: base" +git switch -q -c mainline-up +echo "upstream main" >u.c +commit "upstream: mainline moves on" + +# Edera on LTS: one commit both trees share, one ported with a version tag, +# one reworded but identical in content, one LTS-only, and a repeated subject +# once. +git switch -q -c lts lts-up +sed -i 's/^5$/5 shared/' a.c +commit "xen: shared fix" +sed -i 's/^15$/15 ported/' a.c +commit "9p/xen: share one frontend (6.18)" +sed -i 's/^25$/25 reworded/' a.c +commit "xen: lts wording of a fix" +echo lts-only >l.c +commit "drm: lts only" +sed -i 's/^30$/30 dep one/' b.c +commit "apply dep patch" +lts_tip=$(git rev-parse HEAD) + +# Edera on mainline: the same, plus three new commits (one with the repeated +# subject, one to be skipped), a merge, and an automation-only commit, which +# is never offered. +git switch -q -c main mainline-up +sed -i 's/^5$/5 shared/' a.c +commit "xen: shared fix" +sed -i 's/^15$/15 ported/' a.c +commit "9p/xen: share one frontend" +sed -i 's/^25$/25 reworded/' a.c +commit "xen: mainline wording of a fix" +sed -i 's/^30$/30 dep one/' b.c +commit "apply dep patch" +git switch -q -c side +sed -i 's/^40$/40 new one/' b.c +commit "xen: new on mainline" +git switch -q main +git merge -q --no-ff --no-edit side >/dev/null +sed -i 's/^50$/50 dep two/' b.c +commit "apply dep patch" +echo mainline-only >m.c +commit "xen: mainline only" +mkdir -p .github/workflows +echo "on: push" >.github/workflows/x.yml +commit "automation: a workflow" +main_tip=$(git rev-parse HEAD) + +printf '# never backport\n\n xen: mainline only \n' >"$WORK/skip" +want=$(printf '%s\n' "xen: new on mainline" "apply dep patch") +got=$(bash "$PENDING" "$main_tip" mainline-up "$lts_tip" lts-up "$WORK/skip" | cut -f2) +if [ "$got" = "$want" ]; then ok "pending lists only what LTS lacks, oldest first"; else + bad "pending lists only what LTS lacks, oldest first" "got: $got"; fi + +got=$(bash "$PENDING" "$main_tip" mainline-up "$lts_tip" lts-up | cut -f2 | tail -n1) +if [ "$got" = "xen: mainline only" ]; then ok "without a skip file, skipped commits are pending"; else + bad "without a skip file, skipped commits are pending" "got: $got"; fi + +got=$(bash "$PENDING" "$lts_tip" lts-up "$lts_tip" lts-up) +if [ -z "$got" ]; then ok "a tree against itself has nothing pending"; else + bad "a tree against itself has nothing pending" "got: $got"; fi + +# Blockers. A "pull request" here is one mainline commit, as the request +# file lists it. +bash "$PENDING" "$main_tip" mainline-up "$lts_tip" lts-up >"$WORK/pending-all" +nth() { git log --reverse --format=%H --grep="^$1\$" "$main_tip" | sed -n "${2:-1}p"; } +pr() { for c in "$@"; do printf '%s\t%s\n' "$c" "$(git log -1 --format=%s "$c")"; done >"$WORK/pr"; } +blockers() { bash "$BLOCKERS" "$WORK/pending-all" "$WORK/pr" "$lts_tip" lts-up "$WORK/todo" | cut -f2; } +check() { # check + local got + got=$(blockers) + if [ "$got" != "$2" ]; then + bad "$1" "blockers: $got" + elif [ "$(grep -c . "$WORK/todo")" -ne "$3" ]; then + bad "$1" "todo: $(cat "$WORK/todo")" + else + ok "$1" + fi +} + +pr "$(nth 'apply dep patch' 2)" +check "an earlier pending commit on the same file blocks" "xen: new on mainline" 1 + +pr "$(nth 'apply dep patch' 1)" +check "a commit already on LTS has nothing to do, even with a repeated subject" "" 0 + +pr "$(nth 'xen: new on mainline')" +check "the oldest pending commit is not blocked" "" 1 + +pr "$(nth 'xen: new on mainline')" "$(nth 'apply dep patch' 2)" +check "a pull request does not wait for its own commits" "" 2 + +pr "$(nth 'xen: mainline only')" +check "earlier pending commits on other files do not block" "" 1 + +pr "$(nth 'automation: a workflow')" +check "an automation-only pull request has nothing to do" "" 0 + +printf 'xen: new on mainline\n' >"$WORK/skip2" +bash "$PENDING" "$main_tip" mainline-up "$lts_tip" lts-up "$WORK/skip2" >"$WORK/pending-all" +pr "$(nth 'apply dep patch' 2)" +check "a skipped commit does not block" "" 1 + +bash "$PENDING" "$main_tip" mainline-up "$lts_tip" lts-up "$WORK/skip" >"$WORK/request" + +expect() { + # expect [] + local desc=$1 want=$2 got out + out=$(bash "$VERIFY" "$lts_tip" "$3" "$WORK/request") + got=$? + if [ "$got" -ne "$want" ]; then + bad "$desc: exit $got, want $want" "$out" + elif [ $# -ge 4 ] && ! grep -qF -- "$4" <<<"$out"; then + bad "$desc: output lacks '$4'" "$out" + else + ok "$desc" + fi +} + +pick_all() { # pick_all + git switch -q -c "$1" "$lts_tip" + cut -f1 "$WORK/request" | while read -r c; do git cherry-pick -x "$c" >/dev/null; done +} + +pick_all good +good=$(git rev-parse HEAD) +expect "honest backport passes" 0 "$good" "All 2 requested commits backported" + +got=$(bash "$PENDING" "$main_tip" mainline-up "$good" lts-up "$WORK/skip") +if [ -z "$got" ]; then ok "after the backport nothing is pending"; else + bad "after the backport nothing is pending" "got: $got"; fi + +# Adapted: the backport changes different lines from its source. +git switch -q -c adapted "$good" +echo extra >>b.c +git commit -qa --amend --no-edit +expect "adapted commit needs review" 1 "$(git rev-parse HEAD)" "| adapted |" + +# Retitled: same change, new subject, which pending would not recognise. +git switch -q -c retitled "$good" +git commit -q --amend -m "apply a dependency patch + +(cherry picked from commit $(sed -n 2p "$WORK/request" | cut -f1))" +expect "retitled commit needs review" 1 "$(git rev-parse HEAD)" "| retitled |" + +# Left out: only the first requested commit. +git switch -q -c leftout "$good~1" +expect "left-out commit needs review" 1 "$(git rev-parse HEAD)" "| left-out |" + +# Added: an extra commit with no trailer. +git switch -q -c added "$good" +echo sneaky >s.c +commit "innocent looking" +expect "added commit needs review" 1 "$(git rev-parse HEAD)" "| added |" + +# Out of order. +git switch -q -c reordered "$lts_tip" +git cherry-pick -x "$(sed -n 2p "$WORK/request" | cut -f1)" >/dev/null +git cherry-pick -x "$(sed -n 1p "$WORK/request" | cut -f1)" >/dev/null +expect "out-of-order backport is refused" 2 "$(git rev-parse HEAD)" "| stray |" + +# A trailer naming a commit nobody asked for. +git switch -q -c stray "$good" +echo x >x.c +git add x.c +git commit -qm "xen: mainline only + +(cherry picked from commit $main_tip)" +expect "unrequested source is refused" 2 "$(git rev-parse HEAD)" "| stray |" + +# Not a fast-forward of the LTS tip. +git switch -q -c rewritten "$lts_tip~1" +cut -f1 "$WORK/request" | while read -r c; do git cherry-pick -x "$c" >/dev/null; done +expect "rewritten LTS history is refused" 2 "$(git rev-parse HEAD)" "does not contain the LTS tip" + +# A merge on top. +git switch -q -c merged "$good" +git merge -q --no-ff --no-edit side >/dev/null +expect "merge on top is refused" 2 "$(git rev-parse HEAD)" "merge commit" + +if [ "$failures" -ne 0 ]; then + echo "$failures failure(s)" >&2 + exit 1 +fi +echo "all passed" diff --git a/.github/scripts/tests/report-needs-decision.test.sh b/.github/scripts/tests/report-needs-decision.test.sh new file mode 100755 index 00000000000000..b2ab3547a2f1b6 --- /dev/null +++ b/.github/scripts/tests/report-needs-decision.test.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# Tests for .github/scripts/report-needs-decision.sh. +# +# A clean nightly rebase is force-pushed to its downstream branch unless this script says +# Claude's report asks for a decision, so a report that does ask must never +# read as one that does not. +# +# Run from anywhere: bash .github/scripts/tests/report-needs-decision.test.sh +set -uo pipefail + +HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +SCRIPT="$HERE/../report-needs-decision.sh" + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT + +failures=0 +expect() { + # expect (report on stdin) + local desc=$1 want=$2 got + cat >"$WORK/report.md" + bash "$SCRIPT" "$WORK/report.md" >/dev/null 2>&1 + got=$? + if [ "$got" -eq "$want" ]; then + echo "ok $desc" + else + echo "FAIL $desc: exit $got, want $want" >&2 + failures=$((failures + 1)) + fi +} + +expect "no section, nothing unsure" 1 <<'EOF' +## Summary +All 221 commits replayed cleanly. + +## Conflicts resolved +None. +EOF + +expect "section says None." 1 <<'EOF' +## Summary +Clean. + +## Needs a decision +None. + +## Conflicts resolved +None. +EOF + +expect "section empty before the next heading" 1 <<'EOF' +## Needs a decision + +## Conflicts resolved +None. +EOF + +expect "section with an item" 0 <<'EOF' +## Summary +Clean replay. + +## Needs a decision +- Upstream 0822a2e890 now reaches Arm dom0 vPCI; see below. + +## Conflicts resolved +None. +EOF + +expect "UNSURE outside the section" 0 <<'EOF' +## Summary +Clean replay. + +## Conflicts resolved +- foo.c: kept downstream's lock order. **UNSURE** whether upstream intended otherwise. +EOF + +expect "heading case and trailing space" 0 <<'EOF' +## NEEDS A DECISION +- something +EOF + +expect "a level-3 heading stays inside the section" 0 <<'EOF' +## Needs a decision +### Arm vPCI +- something +## Builds +pass +EOF + +expect "a report with one UNSURE item asks for a decision" 0 <<'EOF' +## Summary +Rebased cleanly. + +## Needs a decision +- **UNSURE: upstream `0123456789ab` ("xen/events: rework lateeoi handling") now reaches the downstream _on_node bind helpers.** + +## Conflicts resolved +None. +EOF + +bash "$SCRIPT" "$WORK/missing.md" >/dev/null 2>&1 +if [ $? -eq 2 ]; then echo "ok unreadable report is an error"; else + echo "FAIL unreadable report is an error" >&2 + failures=$((failures + 1)) +fi + +if [ "$failures" -ne 0 ]; then + echo "$failures failure(s)" >&2 + exit 1 +fi +echo "all passed" diff --git a/.github/scripts/tests/retry-backports.test.sh b/.github/scripts/tests/retry-backports.test.sh new file mode 100755 index 00000000000000..5f900c5c1672b6 --- /dev/null +++ b/.github/scripts/tests/retry-backports.test.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# Tests for .github/scripts/retry-backports.sh against a fake gh. +# +# The fake answers the two pull request listings from WAITING and STALE and +# records every dispatch, so each case checks exactly which backports the +# script started, and that having nothing to retry is a success. +# +# Run from anywhere: bash .github/scripts/tests/retry-backports.test.sh +set -uo pipefail + +HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +RETRY="$HERE/../retry-backports.sh" + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT +mkdir "$WORK/bin" +cat >"$WORK/bin/gh" <<'GH' +#!/usr/bin/env bash +case "$1 $2" in +"pr list") + case " $* " in + *" --label backport-waiting "*) [ -z "$WAITING" ] || printf '%s\n' $WAITING ;; + *) [ -z "$STALE" ] || printf '%s\n' $STALE ;; + esac ;; +"workflow run") echo "$*" >>"$DISPATCHED" ;; +*) echo "unexpected: gh $*" >&2; exit 2 ;; +esac +GH +chmod +x "$WORK/bin/gh" +export PATH="$WORK/bin:$PATH" GITHUB_REPOSITORY=edera-dev/linux DISPATCHED="$WORK/dispatched" + +failures=0 +ok() { echo "ok $1"; } +bad() { + echo "FAIL $1" >&2 + [ -z "${2:-}" ] || printf ' %s\n' "${2//$'\n'/$'\n' }" >&2 + failures=$((failures + 1)) +} + +# check +check() { + local name=$1 args=$2 out got + : >"$DISPATCHED" + # shellcheck disable=SC2086 # $args is zero or one word + if ! out=$(WAITING=$3 STALE=$4 bash "$RETRY" $args 2>&1); then + bad "$name" "exited non-zero: $out" + return + fi + got=$(sed -E 's/.*backport_pr=([0-9]+).*/\1/' "$DISPATCHED" | xargs) + if [ "$got" = "$5" ]; then ok "$name"; else bad "$name" "dispatched '$got', expected '$5'"; fi +} + +check "nothing waiting, no --stale" "" "" "" "" +check "nothing waiting, nothing stale" --stale "" "" "" +check "waiting pull requests are dispatched" "" "7 12" "" "7 12" +check "stale backports are dispatched by number" --stale "" \ + "backport/edera-6.18-lts/pr-23 backport/edera-6.18-lts/pr-9" "9 23" +check "stale backports are ignored without --stale" "" "" "backport/edera-6.18-lts/pr-23" "" +check "a pull request both waiting and stale goes once" --stale "23" \ + "backport/edera-6.18-lts/pr-23" "23" + +if [ "$failures" -ne 0 ]; then + echo "$failures failure(s)" >&2 + exit 1 +fi +echo "all passed" diff --git a/.github/scripts/tests/verify-upstream-rebase.test.sh b/.github/scripts/tests/verify-upstream-rebase.test.sh new file mode 100755 index 00000000000000..fdbdf7b5f86583 --- /dev/null +++ b/.github/scripts/tests/verify-upstream-rebase.test.sh @@ -0,0 +1,136 @@ +#!/usr/bin/env bash +# Tests for .github/scripts/verify-upstream-rebase.sh. +# +# Builds a small throwaway repository with an upstream line and a downstream +# series on it, rebases the series onto a newer upstream, and then damages +# copies of that result in each way the checker exists to catch. The nightly +# rebase force-pushes edera/6.18-lts on this script's say-so, so every +# damaged copy must be refused, and every honest replay must pass. +# +# Run from anywhere: bash .github/scripts/tests/verify-upstream-rebase.test.sh +set -uo pipefail + +HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +SCRIPT="$HERE/../verify-upstream-rebase.sh" + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT +cd "$WORK" || exit 1 + +failures=0 +expect() { + # expect [] + local desc=$1 want=$2 got out + out=$(bash "$SCRIPT" "$3" "$4" "$5") + got=$? + if [ "$got" -ne "$want" ]; then + echo "FAIL $desc: exit $got, want $want" >&2 + printf ' %s\n' "${out//$'\n'/$'\n' }" >&2 + failures=$((failures + 1)) + elif [ $# -ge 6 ] && ! grep -qF -- "$6" <<<"$out"; then + echo "FAIL $desc: output lacks '$6'" >&2 + printf ' %s\n' "${out//$'\n'/$'\n' }" >&2 + failures=$((failures + 1)) + else + echo "ok $desc" + fi +} + +git init -q -b upstream . +git config user.name test +git config user.email test@example.invalid +commit() { git add -A && git commit -qm "$1"; } + +# Upstream: two files with room for both sides to edit. +seq 1 40 >a.c +seq 1 40 >b.c +commit "upstream: base" +base=$(git rev-parse HEAD) + +# Downstream series. Two commits share a subject, as real series do. +git switch -q -c downstream +sed -i 's/^10$/10 downstream/' a.c +commit "downstream: edit a" +sed -i 's/^30$/30 downstream/' b.c +commit "apply dep patch" +echo "new" >c.c +commit "downstream: add c" +sed -i 's/^35$/35 downstream/' b.c +commit "apply dep patch" +old=$(git rev-parse HEAD) + +# Upstream moves on in two steps. The first edits only lines far from +# anything downstream touches; the second edits line 12, which sits inside the +# hunk context of "downstream: edit a" (line 10) without conflicting with it. +git switch -q upstream +sed -i 's/^1$/1 upstream/' b.c +commit "upstream: edit b far away" +far=$(git rev-parse HEAD) +sed -i 's/^12$/12 upstream/' a.c +commit "upstream: edit a near downstream" +up=$(git rev-parse HEAD) + +replay() { # replay + git switch -q -c "$1" "$old" && git rebase -q --onto "$2" "$base" "$1" +} + +replay far "$far" +expect "replay with untouched context passes" 0 "$old" "$(git rev-parse HEAD)" "$far" "identical patch-ids" +expect "a no-op rebase passes" 0 "$old" "$old" "$base" "identical patch-ids" + +# Context moved but the commit's own lines did not: still clean. +replay rebased "$up" +good=$(git rev-parse HEAD) +expect "replay with moved context passes" 0 "$old" "$good" "$up" "1 changed only in hunk context" + +# Damage 1: one of the two same-subject commits edited. +git switch -q -c tamper "$good" +# shellcheck disable=SC2016 # expanded by the shell git rebase starts +git rebase -q --exec 'if [ "$(git log -1 --format=%s)" = "apply dep patch" ] && grep -q "35 downstream" b.c; then echo extra >>b.c; git commit -qa --amend --no-edit; fi' "$up" +expect "edited commit with a repeated subject fails" 1 "$old" "$(git rev-parse HEAD)" "$up" "1 commit(s) lost, added, or with different +/- lines" + +# Damage 2: a commit dropped. +git switch -q -c dropped "$good" +drop=$(git log --format='%H %s' "$up..HEAD" | awk '/downstream: add c/ { print $1 }') +git rebase -q --onto "$drop^" "$drop" dropped +expect "dropped commit fails" 1 "$old" "$(git rev-parse HEAD)" "$up" "downstream: add c" + +# Damage 3: a commit added. +git switch -q -c added "$good" +echo "sneaky" >d.c +commit "innocent looking" +expect "added commit fails" 1 "$old" "$(git rev-parse HEAD)" "$up" "innocent looking" + +# Damage 4: not on the upstream tip. +expect "stale base is refused" 2 "$old" "$old" "$up" "not based on the upstream tip" + +# Damage 5: a merge commit in the series. +git switch -q -c merged "$good" +git switch -q -c side "$up" +echo side >e.c +commit "side" +git switch -q merged +git merge -q --no-edit side +expect "merge in the series is refused" 2 "$old" "$(git rev-parse HEAD)" "$up" "merge commit" + +# A series that merged a feature branch, as both kernel trees once did: the +# first rebase flattens it, and that must still pass, since no commit is +# lost and the tree moves by exactly the upstream delta. +git switch -q -c withmerge "$old" +git switch -q -c feature "$old~1" +sed -i 's/^20$/20 feature/' a.c +commit "feature: edit a" +git switch -q withmerge +git merge -q --no-ff --no-edit feature >/dev/null +sed -i 's/^25$/25 after merge/' b.c +commit "downstream: after the merge" +mold=$(git rev-parse HEAD) +git switch -q -c flattened "$mold" +git rebase -q --onto "$up" "$base" flattened +expect "flattening a merged series passes" 0 "$mold" "$(git rev-parse HEAD)" "$up" "All 6 downstream commits replayed" + +if [ "$failures" -ne 0 ]; then + echo "$failures failure(s)" >&2 + exit 1 +fi +echo "all passed" diff --git a/.github/scripts/verify-backport.sh b/.github/scripts/verify-backport.sh new file mode 100755 index 00000000000000..e5cc7964777ea0 --- /dev/null +++ b/.github/scripts/verify-backport.sh @@ -0,0 +1,177 @@ +#!/usr/bin/env bash +# Checks a backport of edera/mainline commits onto edera/6.18-lts, without +# trusting whoever did it. +# +# The backport of a merged edera/mainline pull request lets a model +# cherry-pick its commits, but nothing it says about its own result is taken +# on faith. This script is the gate. It proves, from git alone, that: +# +# 1. the new branch fast-forwards the LTS tip the backport started from; +# 2. the commits on top are linear (no merge commits); +# 3. every one of them is a backport of a requested commit, in the requested +# order, each named by its "(cherry picked from commit )" trailer, +# with nothing extra in between and nothing requested left out; +# 4. each keeps its source's subject, so pending-backports.sh recognises it +# next time instead of offering it again; +# 5. each changes the same files by the same +/- lines as its source. Hunk +# offsets and context may differ; the change itself may not. +# +# A commit that had to be adapted to the older kernel fails check 5. That is +# the point: such a commit is not wrong, but the pull request has to say so, +# so its reviewer knows to read it closely. +# +# Usage: +# verify-backport.sh +# +# holds the commits asked for, as pending-backports.sh prints +# them: " TAB ", oldest first. +# +# Writes a Markdown report to stdout. Exit status: +# 0 clean: every requested commit backported unchanged +# 1 differences: well-formed (1-3 hold for what is there) but a commit was +# adapted, retitled, left out, or added; flagged in the pull request +# 2 broken: 1 or 2 failed, a trailer names something not requested or out +# of order, or the inputs are unusable; not proposed at all + +set -euo pipefail + +if [ $# -ne 3 ] || [ ! -r "$3" ]; then + echo "usage: $0 " >&2 + exit 2 +fi + +old=$(git rev-parse --verify "$1^{commit}") +new=$(git rev-parse --verify "$2^{commit}") +requested=$3 + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +status=0 +fail() { [ "$status" -ge "$1" ] || status=$1; } +short() { git rev-parse --short=12 "$1"; } +oneline() { git log -1 --format='%h %s' "$1"; } +# The subject without a trailing version tag such as " (6.18)", as +# pending-backports.sh compares them. +subject() { git log -1 --format=%s "$1" | sed -E 's/[[:space:]]*\((v?[0-9]+\.[0-9]+(\.[0-9y]+)?(-lts)?)\)$//'; } +own_change() { + git show --format= "$1" | grep -E '^([+-]|diff )' | grep -vE '^(\+\+\+|---) ' || true +} + +echo "| | |" +echo "| --- | --- |" +echo "| LTS tip before | \`$(short "$old")\` |" +echo "| LTS tip after | \`$(short "$new")\` |" +echo "| Commits requested | $(grep -c . "$requested" || true) |" +echo + +# 1. Fast-forward. +if git merge-base --is-ancestor "$old" "$new"; then + echo "- [x] New branch fast-forwards the LTS tip." +else + echo "- [ ] **New branch does not contain the LTS tip** \`$(short "$old")\`." + fail 2 +fi + +# 2. Linearity. +if [ "$status" -lt 2 ]; then + merges=$(git rev-list --merges "$old..$new" | wc -l) + if [ "$merges" -eq 0 ]; then + echo "- [x] Backported commits are linear." + else + echo "- [ ] **$merges merge commit(s) on top of the LTS tip.**" + fail 2 + fi +fi + +if [ "$status" -ge 2 ]; then + echo + echo "Stopped: the remaining checks assume a linear series on the LTS tip." + exit "$status" +fi + +# 3-5. Walk the new commits and the request list together. A commit names its +# source by trailer; it must be the next requested commit, or a later one (the +# ones skipped over were left out). +cut -f1 "$requested" | while read -r sha; do git rev-parse --verify "$sha^{commit}"; done >"$work/want" +git rev-list --reverse "$old..$new" >"$work/got" + +: >"$work/rows" +next=1 +n_want=$(wc -l <"$work/want") +while read -r c; do + src=$(git log -1 --format=%B "$c" | + sed -nE 's/^\(cherry picked from commit ([0-9a-f]{40})\)$/\1/p' | tail -n1) + if [ -z "$src" ]; then + echo "added $c -" >>"$work/rows" + fail 1 + continue + fi + pos=$(grep -nxF "$src" "$work/want" | cut -d: -f1 | head -n1 || true) + if [ -z "$pos" ] || [ "$pos" -lt "$next" ]; then + echo "stray $c $src" >>"$work/rows" + fail 2 + continue + fi + while [ "$next" -lt "$pos" ]; do + echo "left-out - $(sed -n "${next}p" "$work/want")" >>"$work/rows" + fail 1 + next=$((next + 1)) + done + next=$((pos + 1)) + if [ "$(subject "$c")" != "$(subject "$src")" ]; then + echo "retitled $c $src" >>"$work/rows" + fail 1 + elif [ "$(own_change "$c")" != "$(own_change "$src")" ]; then + echo "adapted $c $src" >>"$work/rows" + fail 1 + else + echo "same $c $src" >>"$work/rows" + fi +done <"$work/got" +while [ "$next" -le "$n_want" ]; do + echo "left-out - $(sed -n "${next}p" "$work/want")" >>"$work/rows" + fail 1 + next=$((next + 1)) +done + +count() { awk -F'\t' -v k="$1" '$1 == k' "$work/rows" | wc -l; } +n_same=$(count same) +if [ "$n_same" -eq "$n_want" ] && [ "$(wc -l <"$work/got")" -eq "$n_want" ]; then + echo "- [x] All $n_want requested commits backported in order, each with its source's subject and identical +/- lines." +else + echo "- [ ] **Backport differs from the request:** $n_same of $n_want unchanged;" \ + "$(count adapted) adapted, $(count retitled) retitled, $(count left-out) left out," \ + "$(count added) added, $(count stray) not requested or out of order." +fi + +if [ "$n_same" -ne "$(wc -l <"$work/rows")" ]; then + echo + echo "| | Backport | Mainline source |" + echo "| --- | --- | --- |" + while IFS=$'\t' read -r verdict c src; do + [ "$verdict" = same ] && continue + a='(none)'; b='(none)' + [ "$c" = - ] || a=$(oneline "$c") + [ "$src" = - ] || b=$(oneline "$src") + echo "| $verdict | \`$a\` | \`$b\` |" + done <"$work/rows" +fi + +if grep -q '^adapted' "$work/rows"; then + echo + echo "
Adapted commits (source +/- lines vs. backport)" + echo + echo '```diff' + awk -F'\t' '$1 == "adapted" { print $2 "\t" $3 }' "$work/rows" | + while IFS=$'\t' read -r c src; do + echo "### $(oneline "$c")" + diff -u --label mainline --label backport \ + <(own_change "$src") <(own_change "$c") || true + done | head -n 400 + echo '```' + echo + echo "
" +fi + +exit "$status" diff --git a/.github/scripts/verify-upstream-rebase.sh b/.github/scripts/verify-upstream-rebase.sh new file mode 100755 index 00000000000000..7a493aad81c0c8 --- /dev/null +++ b/.github/scripts/verify-upstream-rebase.sh @@ -0,0 +1,207 @@ +#!/usr/bin/env bash +# Checks a rebased downstream branch against the branch it replaces, without +# trusting whoever did the rebase. +# +# The nightly upstream-rebase workflow lets a model drive the rebase, but +# nothing it says about its own result is taken on faith. This script is the +# gate. It proves, from git alone, that: +# +# 1. the new branch sits directly on the upstream tip it claims to; +# 2. the downstream series is linear (no merge commits slipped in); +# 3. no downstream commit was lost, added, or changed: patch-ids match, or, +# where they do not, the commit's own +/- lines are identical and only a +# hunk's context anchor moved; +# 4. the tree changed by exactly the upstream delta and nothing else (only +# the +/- lines are compared, so moved hunks and blob hashes do not raise +# false alarms). +# +# A rebase that needed conflict resolution will normally fail check 4: the +# resolution is, by definition, a change to downstream lines. That is the +# point. Such a result is not wrong, but it is not "clean", and a human has to +# read it before it replaces the branch. +# +# Usage: +# verify-upstream-rebase.sh +# +# Writes a Markdown report to stdout. Exit status: +# 0 clean: all four checks pass +# 1 drift: the branch is well-formed (1-2) but 3 or 4 found differences +# 2 broken: 1 or 2 failed, or the inputs are unusable + +set -euo pipefail + +if [ $# -ne 3 ]; then + echo "usage: $0 " >&2 + exit 2 +fi + +old=$(git rev-parse --verify "$1^{commit}") +new=$(git rev-parse --verify "$2^{commit}") +up=$(git rev-parse --verify "$3^{commit}") +old_base=$(git merge-base "$old" "$up") + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT + +status=0 +fail() { [ "$status" -ge "$1" ] || status=$1; } +short() { git rev-parse --short=12 "$1"; } +changed_lines() { grep -E '^[+-]' | grep -vE '^(\+\+\+|---)' || true; } +changed_files() { grep -E '^(\+\+\+|---) ' || true; } + +echo "| | |" +echo "| --- | --- |" +echo "| Old tip | \`$(short "$old")\` |" +echo "| New tip | \`$(short "$new")\` |" +echo "| Old upstream base | \`$(short "$old_base")\` |" +echo "| New upstream base | \`$(short "$up")\` |" +echo + +# 1. Base. +if [ "$(git merge-base "$new" "$up")" = "$up" ]; then + echo "- [x] New branch is based on the upstream tip." +else + echo "- [ ] **New branch is not based on the upstream tip** \`$(short "$up")\`." + fail 2 +fi + +# 2. Linearity. +merges=$(git rev-list --merges "$up..$new" | wc -l) +if [ "$merges" -eq 0 ]; then + echo "- [x] Downstream series is linear." +else + echo "- [ ] **Downstream series contains $merges merge commit(s).**" + fail 2 +fi + +if [ "$status" -ge 2 ]; then + echo + echo "Stopped: the remaining checks assume a linear series on the upstream tip." + exit "$status" +fi + +# 3. Commit set. +# +# Each commit is keyed by its subject and how many earlier commits in the +# series share that subject, so the k-th "apply dep patch" before pairs with +# the k-th after. A rebase keeps the series order, so this pairs every commit +# with its own replay even when subjects repeat. +# +# A changed patch-id is fine when the commit's own +/- lines are unchanged and +# only a hunk's context anchor moved, which is what happens wherever upstream +# edited the same region. Anything else is a real difference. +series() { + git log --no-merges --reverse --format='commit %H' --patch "$1" | + git patch-id --stable | awk '{ print $2 "\t" $1 }' >"$work/pid.map" + git log --no-merges --reverse --format='%H%x09%s' "$1" | + awk -F'\t' 'NR == FNR { pid[$1] = $2; next } + { k = $2 "#" ++seen[$2]; print k "\t" $1 "\t" ($1 in pid ? pid[$1] : "-") }' \ + "$work/pid.map" - +} +series "$old_base..$old" >"$work/old.series" +series "$up..$new" >"$work/new.series" +n_old=$(wc -l <"$work/old.series") +n_new=$(wc -l <"$work/new.series") + +# key -> "old-commit new-commit verdict"; verdict is same, check, lost, added. +awk -F'\t' 'NR == FNR { oc[$1] = $2; op[$1] = $3; order[++n] = $1; next } + { + if ($1 in oc) { print oc[$1], $2, (op[$1] == $3 ? "same" : "check"); delete oc[$1] } + else print "-", $2, "added" + } + END { for (i = 1; i <= n; i++) if (order[i] in oc) print oc[order[i]], "-", "lost" }' \ + "$work/old.series" "$work/new.series" >"$work/pairs" + +own_change() { + local patch + patch=$(git show --format= "$1") + changed_files <<<"$patch" + changed_lines <<<"$patch" +} +: >"$work/moved" +: >"$work/changed" +while read -r c d verdict; do + case $verdict in + same) ;; + check) + if [ "$(own_change "$c")" = "$(own_change "$d")" ]; then + echo "$c $d" >>"$work/moved" + else + echo "$c $d" >>"$work/changed" + fi ;; + *) echo "$c $d" >>"$work/changed" ;; + esac +done <"$work/pairs" + +if [ "$n_old" -eq "$n_new" ] && [ ! -s "$work/changed" ]; then + if [ -s "$work/moved" ]; then + echo "- [x] All $n_new downstream commits replayed;" \ + "$(wc -l <"$work/moved") changed only in hunk context (listed below)." + else + echo "- [x] All $n_new downstream commits replayed with identical patch-ids." + fi +else + echo "- [ ] **Downstream commit set changed:** $n_old before, $n_new after;" \ + "$(wc -l <"$work/changed") commit(s) lost, added, or with different +/- lines." + fail 1 +fi + +# 4. Drift. +git diff "$old..$new" >"$work/rebase.diff" +git diff "$old_base..$up" >"$work/upstream.diff" +changed_files <"$work/rebase.diff" >"$work/rebase.files" +changed_files <"$work/upstream.diff" >"$work/upstream.files" +changed_lines <"$work/rebase.diff" >"$work/rebase.lines" +changed_lines <"$work/upstream.diff" >"$work/upstream.lines" + +if cmp -s "$work/rebase.files" "$work/upstream.files" && + cmp -s "$work/rebase.lines" "$work/upstream.lines"; then + echo "- [x] Zero downstream drift: the tree changed by exactly the upstream delta." +else + echo "- [ ] **Downstream drift:** the tree changed by more than the upstream delta." + fail 1 +fi + +# Details for anything that did not match. +oneline() { if [ "$1" = - ]; then echo "(none)"; else git log -1 --format='%h %s' "$1"; fi; } +if [ -s "$work/moved" ]; then + echo + echo "
Commits whose hunk context moved (own +/- lines identical)" + echo + while read -r c d; do + echo "- \`$(oneline "$c")\` → \`$(git rev-parse --short "$d")\`" + done <"$work/moved" + echo + echo "
" +fi +if [ -s "$work/changed" ]; then + echo + echo "
Commits lost, added, or whose own +/- lines changed" + echo + echo "| Before | After |" + echo "| --- | --- |" + while read -r c d; do + echo "| \`$(oneline "$c")\` | \`$(oneline "$d")\` |" + done <"$work/changed" + echo + echo "
" +fi + +if ! cmp -s "$work/rebase.lines" "$work/upstream.lines" || + ! cmp -s "$work/rebase.files" "$work/upstream.files"; then + echo + echo "
Drift (changed lines: upstream delta vs. actual change)" + echo + echo '```diff' + { + diff -u --label upstream-files --label actual-files \ + "$work/upstream.files" "$work/rebase.files" || true + diff -u --label upstream-lines --label actual-lines \ + "$work/upstream.lines" "$work/rebase.lines" || true + } | head -n 400 + echo '```' + echo + echo "
" +fi + +exit "$status" diff --git a/.github/workflows/automation-selftest.yml b/.github/workflows/automation-selftest.yml new file mode 100644 index 00000000000000..0f5b41444b775e --- /dev/null +++ b/.github/workflows/automation-selftest.yml @@ -0,0 +1,71 @@ +name: Automation self-test + +# Tests the scripts the nightly rebase relies on to decide whether to +# force-push edera/6.18-lts, and the ones the backports rely on to decide what +# to propose, so a change that weakens any of them must not land quietly: +# +# - verify-upstream-rebase.sh: honest replays pass, including one that +# flattens a merged series; edited, dropped, added and merged commits and +# a stale base are all refused; +# - report-needs-decision.sh: any question in Claude's report, or anything +# marked UNSURE, stops the automatic push; +# - pending-backports.sh, backport-blockers.sh and verify-backport.sh: only +# what the LTS tree lacks is offered, in mainline order, and a damaged +# backport is flagged or refused. +# +# It runs only when the automation itself changes. + +on: + pull_request: + types: [opened, synchronize, ready_for_review] + branches: ['edera/**'] + paths: + - '.github/scripts/**' + - '.github/workflows/upstream-rebase.yml' + - '.github/workflows/kernel-nightly.yml' + - '.github/workflows/rebase-land.yml' + - '.github/workflows/lts-backport.yml' + - '.github/workflows/lts-backport-merged.yml' + - '.github/workflows/automation-selftest.yml' + - '.automation/**' + +permissions: + contents: read + +concurrency: + group: automation-selftest-${{ github.event.pull_request.number }} + cancel-in-progress: true + +jobs: + selftest: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + # Only the automation is needed, not the kernel. + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + persist-credentials: false + sparse-checkout: | + .github + .automation + + - name: Rebase checker tests + run: bash .github/scripts/tests/verify-upstream-rebase.test.sh + + - name: Decision detector tests + run: bash .github/scripts/tests/report-needs-decision.test.sh + + - name: Backport tests + run: bash .github/scripts/tests/backport.test.sh + + - name: Backport retry tests + run: bash .github/scripts/tests/retry-backports.test.sh + + - name: Shellcheck + run: shellcheck .github/scripts/*.sh .github/scripts/tests/*.sh diff --git a/.github/workflows/kernel-nightly.yml b/.github/workflows/kernel-nightly.yml new file mode 100644 index 00000000000000..9c5135045f42c9 --- /dev/null +++ b/.github/workflows/kernel-nightly.yml @@ -0,0 +1,114 @@ +name: Nightly kernel maintenance + +# Keeps edera/6.18-lts current: +# +# - nightly, rebases it onto stable linux-6.18.y (upstream-rebase.yml), then +# retries waiting backports and refreshes the backport pull requests the +# rebase left behind (retry-backports.sh); +# - with backport_pr set, proposes the backport of that merged +# edera/mainline pull request instead (lts-backport.yml). +# lts-backport-trigger.yml on edera/mainline dispatches this for every +# pull request that merges there. +# +# Claude does the rebase and the backports. Independent checks decide what +# happens to each result; see those workflows. +# +# GitHub runs scheduled workflows only from the default branch, edera/mainline, +# so this copy has no schedule of its own. lts-rebase-trigger.yml on +# edera/mainline dispatches it nightly; this file and the one on +# edera/mainline share a path, which is how that dispatch finds this copy. +# +# Repository configuration this needs: +# vars.REBASE_APP_CLIENT_ID, secrets.REBASE_APP_PRIVATE_KEY +# A GitHub App installed on this repository with contents, pull requests +# and workflows write, allowed to bypass the protection rules on +# edera/6.18-lts. GITHUB_TOKEN cannot do this job: it cannot push +# commits that touch .github/workflows/ (every rebase rewrites the ones +# that add them), and pull requests it opens do not trigger other +# workflows. +# vars.REBASE_FEDERATION_RULE_ID +# The workload identity federation rule for the rebase. It must accept +# this workflow's OIDC subject (a dispatched run on edera/6.18-lts). +# vars.ANTHROPIC_ORGANIZATION_ID, ANTHROPIC_SERVICE_ACCOUNT_ID, +# ANTHROPIC_WORKSPACE_ID +# The rest of the Claude API federation identity. + +on: + workflow_dispatch: + inputs: + force: + description: Rebase even if an open rebase pull request already covers this upstream tip + type: boolean + default: false + backport_pr: + description: Instead of the rebase, backport this merged edera/mainline pull request (its number) + type: string + default: '' + +permissions: + contents: read + +concurrency: + # Per tree: groups are repository-wide, and edera/mainline's copy of this + # workflow uses plain kernel-nightly. Each backport has its own group, so + # they run alongside the rebase and each other; a newer dispatch for the + # same pull request replaces one still queued. + group: kernel-nightly-edera/6.18-lts${{ inputs.backport_pr != '' && format('-backport-{0}', inputs.backport_pr) || '' }} + cancel-in-progress: false + +jobs: + lts: + if: inputs.backport_pr == '' + uses: ./.github/workflows/upstream-rebase.yml + permissions: + contents: read + pull-requests: read + issues: write + id-token: write + with: + downstream: edera/6.18-lts + key: 6.18-lts + upstream_url: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git + upstream_branch: linux-6.18.y + force: ${{ inputs.force == true }} + secrets: inherit + + # After the rebase, whatever it did. + retry: + needs: lts + if: always() && inputs.backport_pr == '' + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + actions: write + pull-requests: read + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + persist-credentials: false + sparse-checkout: .github + + - name: Retry and refresh backports + env: + GH_TOKEN: ${{ github.token }} + run: bash .github/scripts/retry-backports.sh --stale + + backport: + if: inputs.backport_pr != '' + uses: ./.github/workflows/lts-backport.yml + permissions: + contents: read + actions: write + issues: write + pull-requests: write + id-token: write + with: + pr: ${{ inputs.backport_pr }} + secrets: inherit diff --git a/.github/workflows/lts-backport-merged.yml b/.github/workflows/lts-backport-merged.yml new file mode 100644 index 00000000000000..d77121a6a984ec --- /dev/null +++ b/.github/workflows/lts-backport-merged.yml @@ -0,0 +1,45 @@ +name: Retry waiting backports + +# When a backport pull request merges into edera/6.18-lts, a mainline pull +# request waiting for it (labelled backport-waiting) may now be able to go +# ahead, so every waiting one is dispatched again (retry-backports.sh). The +# nightly retries them too; this just saves the wait. +# +# pull_request_target runs this file as it is on edera/6.18-lts, never the +# pull request's copy, and nothing here checks out the pull request. + +on: + pull_request_target: + types: [closed] + branches: [edera/6.18-lts] + +permissions: {} + +jobs: + retry: + if: >- + github.event.pull_request.merged + && startsWith(github.event.pull_request.head.ref, 'backport/edera-6.18-lts/') + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + actions: write + pull-requests: read + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ github.event.pull_request.base.ref }} + persist-credentials: false + sparse-checkout: .github + + - name: Retry waiting backports + env: + GH_TOKEN: ${{ github.token }} + run: bash .github/scripts/retry-backports.sh diff --git a/.github/workflows/lts-backport.yml b/.github/workflows/lts-backport.yml new file mode 100644 index 00000000000000..e60c42b6a3a4c5 --- /dev/null +++ b/.github/workflows/lts-backport.yml @@ -0,0 +1,723 @@ +name: LTS backport + +# Proposes the backport of one merged edera/mainline pull request to +# edera/6.18-lts, as a pull request. Called by kernel-nightly.yml when it is +# dispatched with backport_pr: by lts-backport-trigger.yml on edera/mainline +# when a pull request merges there, and by retry-backports.sh to retry or +# refresh one. +# +# Edera kernel work lands on edera/mainline first and every merged pull +# request flows down, unless it is labelled no-backport. The order is kept: +# backport-blockers.sh holds a pull request back while an earlier Edera commit +# that touches its files is still missing from edera/6.18-lts, and Claude +# holds it back for a dependency a file match cannot see. Either way the +# mainline pull request is labelled backport-waiting and retried later. +# +# Claude cherry-picks the commits, following +# .automation/skills/mainline-backport/SKILL.md. Nothing it says about its own +# result decides what happens to it. The model runs read-only; its branch +# leaves as a git bundle and is judged by scripts from the commit this run +# started from: +# +# - verify-backport.sh proves the branch fast-forwards the LTS tip and holds +# exactly the requested commits, in order, each with its source's subject +# and its source's +/- lines; +# - kernel-build.sh builds x86_64 and arm64 vmlinux. +# +# The result always goes up as a backport/edera-6.18-lts/pr- pull request +# with both results and Claude's report; nothing here pushes edera/6.18-lts. +# The mainline pull request gets one status comment, kept up to date, and is +# labelled backport-needs-attention when there is nothing to propose and a +# human has to decide. + +on: + workflow_call: + inputs: + pr: + description: Number of the merged edera/mainline pull request + type: string + required: true + +permissions: + contents: read + +env: + SOURCE: edera/mainline + SOURCE_UPSTREAM_URL: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + SOURCE_UPSTREAM_BRANCH: master + TARGET: edera/6.18-lts + TARGET_UPSTREAM_URL: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git + TARGET_UPSTREAM_BRANCH: linux-6.18.y + PR: ${{ inputs.pr }} + BRANCH: backport/edera-6.18-lts/pr-${{ inputs.pr }} + +jobs: + prepare: + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + pull-requests: read + outputs: + state: ${{ steps.plan.outputs.state }} + old: ${{ steps.plan.outputs.old }} + head: ${{ steps.plan.outputs.head }} + source: ${{ steps.plan.outputs.source }} + title: ${{ steps.plan.outputs.title }} + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ env.TARGET }} + fetch-depth: 0 + filter: blob:none + persist-credentials: false + + - name: Trusted tools + run: | + set -euo pipefail + git fetch --no-tags --filter=blob:none origin "$GITHUB_SHA" + mkdir "$RUNNER_TEMP/tools" + git archive "$GITHUB_SHA" .github/scripts .automation | tar -x -C "$RUNNER_TEMP/tools" --strip-components=1 + + - name: Plan + id: plan + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + out=$RUNNER_TEMP/out + mkdir -p "$out" + tools=$RUNNER_TEMP/tools + finish() { # finish + echo "state=$1" >> "$GITHUB_OUTPUT" + printf '%s\n' "$2" >"$out/message.md" + printf '### Backport of #%s\n\n%s\n' "$PR" "$2" >> "$GITHUB_STEP_SUMMARY" + exit 0 + } + if ! [[ $PR =~ ^[0-9]+$ ]]; then + echo "::error::backport_pr must be a pull request number, not '$PR'." + exit 1 + fi + old=$(git rev-parse HEAD) + echo "old=$old" >> "$GITHUB_OUTPUT" + + pr=$(gh api "repos/$GITHUB_REPOSITORY/pulls/$PR") + title=$(jq -r .title <<<"$pr" | tr -d '\r\n') + echo "title=$title" >> "$GITHUB_OUTPUT" + if [ "$(jq -r .merged <<<"$pr")" != true ] || [ "$(jq -r .base.ref <<<"$pr")" != "$SOURCE" ]; then + finish ignored "#$PR is not a merged pull request into \`$SOURCE\`." + fi + case $(jq -r .head.ref <<<"$pr") in + rebase/*) finish ignored "#$PR is a nightly rebase, not Edera work." ;; + esac + if jq -e '.labels | map(.name) | index("no-backport")' <<<"$pr" >/dev/null; then + finish skipped "#$PR is labelled \`no-backport\`, so it is not backported to \`$TARGET\`." + fi + + # The pull request's own commits, as they were reviewed. Their hashes + # stay valid when the nightly rebase rewrites edera/mainline. + git fetch --no-tags --filter=blob:none origin "refs/pull/$PR/head" + head=$(git rev-parse FETCH_HEAD) + echo "head=$head" >> "$GITHUB_OUTPUT" + gh api --paginate "repos/$GITHUB_REPOSITORY/pulls/$PR/commits" -q '.[].sha' >"$out/pr-commits" + while read -r c; do + git merge-base --is-ancestor "$c" "$head" + printf '%s\t%s\n' "$c" "$(git log -1 --format=%s "$c")" + done <"$out/pr-commits" >"$out/request" + + git fetch --no-tags --filter=blob:none origin "$SOURCE" + src=$(git rev-parse FETCH_HEAD) + echo "source=$src" >> "$GITHUB_OUTPUT" + git fetch --no-tags --filter=blob:none "$SOURCE_UPSTREAM_URL" "$SOURCE_UPSTREAM_BRANCH" + src_up=$(git rev-parse FETCH_HEAD) + git fetch --no-tags --filter=blob:none "$TARGET_UPSTREAM_URL" "$TARGET_UPSTREAM_BRANCH" + tgt_up=$(git rev-parse FETCH_HEAD) + + # Never backported: the skip file, and the commits of every mainline + # pull request labelled no-backport. + cp "$tools/backport-skip" "$out/skip" + gh pr list --repo "$GITHUB_REPOSITORY" --base "$SOURCE" --state merged \ + --label no-backport --limit 500 --json number -q '.[].number' | + while read -r n; do + gh api --paginate "repos/$GITHUB_REPOSITORY/pulls/$n/commits" -q '.[].commit.message | split("\n")[0]' + done >>"$out/skip" + + bash "$tools/scripts/pending-backports.sh" "$src" "$src_up" "$old" "$tgt_up" "$out/skip" >"$out/pending" + bash "$tools/scripts/backport-blockers.sh" "$out/pending" "$out/request" "$old" "$tgt_up" "$out/todo" >"$out/blockers" + + if [ ! -s "$out/todo" ]; then + finish skipped "Nothing to backport: every commit of #$PR is already on \`$TARGET\` (\`${old:0:12}\`), changes only automation, or is no longer an Edera commit on \`$SOURCE\`." + fi + if [ -s "$out/blockers" ]; then + { + echo "Waiting: earlier Edera commits on \`$SOURCE\` that touch the same files are not on \`$TARGET\` yet, and have to be backported first:" + echo + while IFS=$'\t' read -r sha subject; do echo "- \`${sha:0:12}\` $subject"; done <"$out/blockers" + echo + echo "This is retried when a backport lands on \`$TARGET\`, and nightly. If one of them should never be backported, label its pull request \`no-backport\`, or list its subject in \`.automation/backport-skip\` on \`$TARGET\`." + } >"$out/blocked.md" + finish waiting "$(cat "$out/blocked.md")" + fi + { + echo "### Backport of #$PR to \`$TARGET\` (\`${old:0:12}\`)" + echo + while IFS=$'\t' read -r sha subject; do echo "- \`${sha:0:12}\` $subject"; done <"$out/todo" + } >> "$GITHUB_STEP_SUMMARY" + echo "state=run" >> "$GITHUB_OUTPUT" + + - name: Upload + if: always() + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: backport-plan + path: ${{ runner.temp }}/out/ + if-no-files-found: error + retention-days: 30 + + backport: + needs: prepare + if: needs.prepare.outputs.state == 'run' + runs-on: ubuntu-latest + timeout-minutes: 240 + # Read-only on purpose: the model runs here. It reaches the Claude API + # through OIDC federation, and that is the only thing id-token is for. + permissions: + contents: read + id-token: write + outputs: + produced: ${{ steps.collect.outputs.produced }} + result: ${{ steps.collect.outputs.result }} + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ needs.prepare.outputs.old }} + fetch-depth: 0 + filter: blob:none + persist-credentials: false + + # The skill and scripts come from the commit this run started from. + # The kernel's .gitignore ignores dot-directories, so .rebase-tools/ + # cannot end up in a commit. + - name: Trusted tools + env: + HEAD_SHA: ${{ needs.prepare.outputs.head }} + SRC: ${{ needs.prepare.outputs.source }} + run: | + set -euo pipefail + git fetch --no-tags --filter=blob:none origin "$GITHUB_SHA" "$SRC" "refs/pull/$PR/head" + git cat-file -e "$HEAD_SHA^{commit}" + mkdir .rebase-tools + git archive "$GITHUB_SHA" .github/scripts .automation | tar -x -C .rebase-tools --strip-components=1 + git config --global user.name "edera-lts-backport" + git config --global user.email "noreply@edera.dev" + + - name: Download the plan + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport-plan + path: ${{ runner.temp }}/in + + - name: Install toolchains + run: | + set -euo pipefail + sudo apt-get update + sudo apt-get install -y --no-install-recommends \ + build-essential bc bison flex libelf-dev libssl-dev python3 \ + gcc-aarch64-linux-gnu + + - name: Normalize identifiers + id: ids + env: + RULE: ${{ vars.REBASE_FEDERATION_RULE_ID }} + ORG: ${{ vars.ANTHROPIC_ORGANIZATION_ID }} + SVC: ${{ vars.ANTHROPIC_SERVICE_ACCOUNT_ID }} + WS: ${{ vars.ANTHROPIC_WORKSPACE_ID }} + run: | + set -euo pipefail + strip() { printf '%s' "$1" | tr -d '[:space:]'; } + { + echo "rule=$(strip "$RULE")" + echo "org=$(strip "$ORG")" + echo "svc=$(strip "$SVC")" + echo "ws=$(strip "$WS")" + } >> "$GITHUB_OUTPUT" + # An unset variable otherwise surfaces as an opaque API failure a + # second into the model step, with nothing in the log to say why. + missing=0 + for n in REBASE_FEDERATION_RULE_ID:RULE ANTHROPIC_ORGANIZATION_ID:ORG ANTHROPIC_SERVICE_ACCOUNT_ID:SVC ANTHROPIC_WORKSPACE_ID:WS; do + eval "v=\$${n#*:}" + if [ -z "$(strip "$v")" ]; then + echo "::error::vars.${n%%:*} is empty or not visible to this repository." + missing=1 + fi + done + exit "$missing" + + - name: Backport + id: claude + continue-on-error: true + uses: anthropics/claude-code-action@6fed3ca145920b639991cb756090506e1bcaf515 # v1.0.245 + with: + # The job token, which is read-only here. Without it the action + # would exchange OIDC for its own app token, which can write. + github_token: ${{ github.token }} + anthropic_federation_rule_id: '${{ steps.ids.outputs.rule }}' + anthropic_organization_id: '${{ steps.ids.outputs.org }}' + anthropic_service_account_id: '${{ steps.ids.outputs.svc }}' + anthropic_workspace_id: '${{ steps.ids.outputs.ws }}' + allowed_bots: 'github-actions' + prompt: | + Backport a merged edera/mainline pull request to edera/6.18-lts by + following .rebase-tools/skills/mainline-backport/SKILL.md exactly. + Read it in full before you start. + + MAINLINE_PR=#${{ inputs.pr }}: ${{ needs.prepare.outputs.title }} + OLD_TIP=${{ needs.prepare.outputs.old }} (${{ env.TARGET }}, checked out) + REQUEST=${{ runner.temp }}/in/todo + PENDING=${{ runner.temp }}/in/pending + RESULT=backport-result + REPORT=${{ runner.temp }}/backport-report.md + + You cannot push and must not try. The workflow verifies and + builds your branch independently afterwards and proposes it as a + pull request. + claude_args: | + --model claude-opus-5-5 + --max-turns 300 + --allowedTools "Read,Grep,Glob,Edit,Write,Bash(git:*),Bash(make:*),Bash(bash .rebase-tools/scripts/verify-backport.sh:*),Bash(bash .rebase-tools/scripts/kernel-build.sh:*),Bash(scripts/config:*),Bash(mkdir -p .rebase-scratch:*),Bash(diff:*),Bash(comm:*),Bash(sort:*),Bash(grep:*),Bash(sed:*),Bash(awk:*),Bash(cat:*),Bash(head:*),Bash(tail:*),Bash(wc:*),Bash(ls:*),Bash(cut:*),Bash(tee:*)" + + - name: Collect the result + id: collect + if: always() + env: + REPORT: ${{ runner.temp }}/backport-report.md + OLD: ${{ needs.prepare.outputs.old }} + OUTCOME: ${{ steps.claude.outcome }} + EXECUTION_FILE: ${{ steps.claude.outputs.execution_file }} + run: | + set -euo pipefail + out=$RUNNER_TEMP/out + mkdir -p "$out" + if [ -s "$REPORT" ]; then + cp "$REPORT" "$out/report.md" + else + # Only the API error messages and the final result text: neither + # carries tool output into a public log. + why= + f=${EXECUTION_FILE:-$RUNNER_TEMP/claude-execution-output.json} + if [ -s "$f" ]; then + why=$(jq -r ' + (if type == "array" then .[] else . end) + | if .type == "assistant" and (.error != null) then + "API error (\(.error)): " + ([.message.content[]? | .text? // empty] | join(" ")) + elif .type == "result" then + ([.result // empty] + (.errors // [] | map(tostring))) | join("\n") + else empty end' "$f" 2>/dev/null | head -c 2000 || true) + fi + { + printf 'Status: failed\n\nClaude left no report (backport step: %s).\n' "$OUTCOME" + if [ -n "$why" ]; then + # shellcheck disable=SC2016 # a literal Markdown fence + printf '\n```\n%s\n```\n' "$why" + else + printf 'The run log has the cause.\n' + fi + } >"$out/report.md" + [ -z "$why" ] || echo "::error::Claude ended without a result: ${why//$'\n'/ }" + fi + if git rev-parse --verify -q refs/heads/backport-result >/dev/null \ + && git merge-base --is-ancestor "$OLD" backport-result \ + && [ "$(git rev-parse backport-result)" != "$OLD" ]; then + git bundle create "$out/backport.bundle" refs/heads/backport-result "^$OLD" + echo "produced=true" >> "$GITHUB_OUTPUT" + echo "result=$(git rev-parse backport-result)" >> "$GITHUB_OUTPUT" + else + echo "produced=false" >> "$GITHUB_OUTPUT" + fi + + - name: Upload + if: always() + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: backport + path: ${{ runner.temp }}/out/ + if-no-files-found: error + retention-days: 30 + + verify: + needs: [prepare, backport] + if: needs.backport.outputs.produced == 'true' + runs-on: ubuntu-latest + timeout-minutes: 30 + outputs: + status: ${{ steps.check.outputs.status }} + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ needs.prepare.outputs.old }} + fetch-depth: 0 + filter: blob:none + persist-credentials: false + + - name: Trusted tools + run: | + set -euo pipefail + git fetch --no-tags --filter=blob:none origin "$GITHUB_SHA" "refs/pull/$PR/head" + mkdir "$RUNNER_TEMP/tools" + git archive "$GITHUB_SHA" .github/scripts | tar -x -C "$RUNNER_TEMP/tools" --strip-components=1 + + - name: Download the plan + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport-plan + path: ${{ runner.temp }}/in/plan + + - name: Download the backport + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport + path: ${{ runner.temp }}/in/backport + + - name: Check + id: check + env: + OLD: ${{ needs.prepare.outputs.old }} + RESULT: ${{ needs.backport.outputs.result }} + run: | + set -euo pipefail + git fetch "$RUNNER_TEMP/in/backport/backport.bundle" backport-result:backport-result + [ "$(git rev-parse backport-result)" = "$RESULT" ] || { echo "bundle does not hold $RESULT"; exit 1; } + # Capture the status: under bash -e a bare call would end the step + # before the verdict is recorded. + rc=0 + bash "$RUNNER_TEMP/tools/scripts/verify-backport.sh" "$OLD" "$RESULT" "$RUNNER_TEMP/in/plan/todo" >"$RUNNER_TEMP/verify.md" || rc=$? + case $rc in + 0) status=clean ;; + 1) status=differences ;; + *) status=broken ;; + esac + echo "status=$status" >> "$GITHUB_OUTPUT" + cat "$RUNNER_TEMP/verify.md" >> "$GITHUB_STEP_SUMMARY" + + - name: Upload + if: always() + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: backport-verify + path: ${{ runner.temp }}/verify.md + retention-days: 30 + + build: + needs: [prepare, backport] + if: needs.backport.outputs.produced == 'true' + runs-on: ubuntu-latest + timeout-minutes: 120 + strategy: + fail-fast: false + matrix: + arch: [x86_64, arm64] + name: build (${{ matrix.arch }}) + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ needs.prepare.outputs.old }} + fetch-depth: 0 + filter: blob:none + persist-credentials: false + + - name: Trusted tools + run: | + set -euo pipefail + git fetch --no-tags --filter=blob:none origin "$GITHUB_SHA" + mkdir "$RUNNER_TEMP/tools" + git archive "$GITHUB_SHA" .github/scripts | tar -x -C "$RUNNER_TEMP/tools" --strip-components=1 + + - name: Download + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport + path: ${{ runner.temp }}/in + + - name: Check out the result + env: + RESULT: ${{ needs.backport.outputs.result }} + run: | + set -euo pipefail + git fetch "$RUNNER_TEMP/in/backport.bundle" backport-result:backport-result + git checkout --detach "$RESULT" + + - name: Install toolchains + run: | + set -euo pipefail + sudo apt-get update + sudo apt-get install -y --no-install-recommends \ + build-essential bc bison flex libelf-dev libssl-dev python3 \ + gcc-aarch64-linux-gnu + + - name: Build + env: + ARCH: ${{ matrix.arch }} + run: bash "$RUNNER_TEMP/tools/scripts/kernel-build.sh" "$ARCH" "$RUNNER_TEMP/obj" + + publish: + needs: [prepare, backport, verify, build] + if: >- + always() + && needs.prepare.result == 'success' + && needs.prepare.outputs.state != 'ignored' + && needs.backport.result != 'cancelled' + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + actions: write + issues: write + pull-requests: write + steps: + - name: Harden runner + uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + ref: ${{ needs.prepare.outputs.old }} + fetch-depth: 0 + filter: blob:none + persist-credentials: false + + - name: Trusted tools + run: | + set -euo pipefail + git fetch --no-tags --filter=blob:none origin "$GITHUB_SHA" + mkdir "$RUNNER_TEMP/tools" + git archive "$GITHUB_SHA" .github/scripts | tar -x -C "$RUNNER_TEMP/tools" --strip-components=1 + + - name: Download the plan + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport-plan + path: ${{ runner.temp }}/in/plan + + - name: Download the backport + if: needs.prepare.outputs.state == 'run' + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport + path: ${{ runner.temp }}/in/backport + + - name: Download the checker report + if: needs.verify.result == 'success' + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: backport-verify + path: ${{ runner.temp }}/in/verify + + # Pushes and closes backport branches. Pull requests it opens trigger + # the usual checks, which ones opened with GITHUB_TOKEN would not. + - name: App token + id: app + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 + with: + client-id: ${{ vars.REBASE_APP_CLIENT_ID }} + private-key: ${{ secrets.REBASE_APP_PRIVATE_KEY }} + permission-contents: write + permission-pull-requests: write + + - name: Propose + env: + APP_TOKEN: ${{ steps.app.outputs.token }} + GH_TOKEN: ${{ github.token }} + STATE: ${{ needs.prepare.outputs.state }} + TITLE: ${{ needs.prepare.outputs.title }} + PRODUCED: ${{ needs.backport.outputs.produced }} + VERIFY: ${{ needs.verify.outputs.status }} + BUILD: ${{ needs.build.result }} + OLD: ${{ needs.prepare.outputs.old }} + RESULT: ${{ needs.backport.outputs.result }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + set -euo pipefail + R=$GITHUB_REPOSITORY + in=$RUNNER_TEMP/in + summary() { cat >> "$GITHUB_STEP_SUMMARY"; } + + # One status comment on the mainline pull request, kept up to date. + marker='' + status_comment() { # status_comment + local id + { echo "$marker"; echo "**Backport to \`$TARGET\`:** $(head -n1 "$1")"; tail -n +2 "$1"; echo; echo "[Run log]($RUN_URL)"; } >"$1.full" + id=$(gh api --paginate "repos/$R/issues/$PR/comments" \ + -q ".[] | select(.body | startswith(\"$marker\")) | .id" | head -n1) + if [ -n "$id" ]; then + gh api -X PATCH "repos/$R/issues/comments/$id" -F body=@"$1.full" >/dev/null + else + gh api -X POST "repos/$R/issues/$PR/comments" -F body=@"$1.full" >/dev/null + fi + } + # At most one of backport-waiting and backport-needs-attention. + set_label() { # set_label