Skip to content

fix(ci): make image builds noninteractive - #196

Merged
openoms merged 3 commits into
openoms:masterfrom
autoblitzbot:fix/ci-noninteractive-dpkg
Aug 15, 2026
Merged

openoms merged 3 commits into
openoms:masterfrom
autoblitzbot:fix/ci-noninteractive-dpkg

Conversation

@autoblitzbot

@autoblitzbot autoblitzbot commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Both image builds currently fail in packer on every PR:

arm64-rpi-image-build — dpkg halts on the interactive initramfs.conf conffile prompt (Y/I/N/O/D/Z) during kernel package configuration triggered by apt-get upgrade -f -y, then cascades:

*** initramfs.conf (Y/I/N/O/D/Z) [default=N] ? dpkg: error processing package initramfs-tools-core (--configure):
E: Sub-process /usr/bin/dpkg returned an error code (1)

amd64-image-build — two issues:

  1. debconf falls back through Dialog → Readline → Teletype frontends (no controlling tty in packer), the same unguarded conffile-prompt exposure as arm64
  2. hard failure: E: Package 'netcat' has no installation candidate — the netcat metapackage was removed in Debian trixie

Additionally, the mandatory PGP source verification in build_joininbox.sh fails on every PR build because PR head commits come from forks and are not signed by the pinned maintainer keys:

# BUILD FAILED --> PGP verification not OK / signature(0) verify(0)

Fix

Noninteractive package management (build-time only):

  • build_joininbox.sh + ci/amd64/debian/scripts/update.sh: export DEBIAN_FRONTEND=noninteractive and install /etc/apt/apt.conf.d/90joininbox-noninteractive with Dpkg::Options { "--force-confdef"; "--force-confold"; } before the upgrades — conffile prompts resolve to the default action and keep the existing config
  • The dropin is removed at the end of build_joininbox.sh so it does not ship in deployed images; running nodes keep the default interactive conffile handling
  • build_joininbox.sh: install netcat-openbsd when the netcat metapackage is unavailable (trixie)

PR builds skip code signature verification, labeled as test images:

  • pull_request workflow runs pass the PR number down the chain: workflow JOININBOX_PR_NUMBER → build script arg → packer pr_number var → provisioner env → build_joininbox.sh
  • When set, verification is skipped with a prominent warning and the image is labeled in /etc/joininbox-build-info (pr number, fork, branch, build date)
  • The main menu shows !!! UNVERIFIED PR BUILD #<N> - TESTING ONLY !!! on top whenever that label is present, so a flashed PR image cannot be mistaken for a production build
  • Artifact names carry a -pr<N> suffix
  • push to master and workflow_dispatch never set the variable → production images keep mandatory signature verification

Note: for pull_request events the workflow YAML runs from the base branch, so this PR's own image-build checks still execute the old master workflow and stay red on verification. The new path activates after merge; PRs opened or rebased afterwards build green.

Verified: bash -n / sh -n clean on all modified scripts.

Both image builds fail in packer:

- arm64-rpi: dpkg halts on the interactive initramfs.conf conffile
  prompt (Y/I/N/O/D/Z) during 'apt-get upgrade' kernel configuration,
  then cascades: E: Sub-process /usr/bin/dpkg returned an error code (1)
- amd64: debconf falls back through Dialog/Readline/Teletype frontends
  (no controlling tty) and the build later fails on
  'E: Package netcat has no installation candidate' - the netcat
  metapackage was removed in Debian trixie

Fixes:
- export DEBIAN_FRONTEND=noninteractive and install an apt.conf.d
  dropin with Dpkg::Options --force-confdef/--force-confold so conffile
  prompts resolve to the default action and keep the existing config
  (build_joininbox.sh and the amd64 update.sh kernel upgrade)
- install netcat-openbsd when the netcat metapackage is unavailable
PR CI builds pull head commits from forks which are not signed by the
maintainer keys, so the mandatory PGP verification in
build_joininbox.sh fails every PR image build:

  # BUILD FAILED --> PGP verification not OK / signature(0) verify(0)

Behavior after this change:
- pull_request workflow runs pass the PR number through the packer
  build chain (workflow -> build script -> packer var -> provisioner
  env -> JOININBOX_PR_NUMBER)
- build_joininbox.sh skips the source signature verification when
  JOININBOX_PR_NUMBER is set, prints a prominent warning, and labels
  the image in /etc/joininbox-build-info as an UNVERIFIED test build
- the uploaded artifact name carries a -pr<N> suffix via BUILD_VERSION
- push-to-master and workflow_dispatch builds never set the variable,
  so production images keep mandatory verification
@autoblitzbot

Copy link
Copy Markdown
Contributor Author

Added 77f3d24 implementing the PR-build policy.

Root cause of this run's failure: build_joininbox.sh verifies the HEAD commit signature against the pinned maintainer keys. PR head commits come from forks and aren't signed by those keys, so every PR image build dies at # BUILD FAILED --> PGP verification not OK.

New behavior:

  • pull_request workflow runs pass the PR number down the whole chain: workflow JOININBOX_PR_NUMBER → build script arg → packer pr_number var → provisioner env → build_joininbox.sh
  • When set, verification is skipped with a loud warning, and the image is labeled in /etc/joininbox-build-info as an UNVERIFIED PR test build (pr number, fork, branch, build date)
  • The uploaded artifact name gets a -pr<N> suffix via BUILD_VERSION
  • push to master and workflow_dispatch never set the variable → production images keep mandatory verification

Bootstrap caveat: for pull_request events the workflow YAML runs from the base branch, so this PR's own image-build checks still execute the old master workflow — no PR number is passed and they'll stay red on verification. The new path activates once this merges; any PR opened or rebased afterwards builds green with a PR-labeled, unverified test image. Suggest merging with the red image checks (ShellCheck + Typos are green), then rebasing #186–#190.

@RonDeBanc RonDeBanc left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: APPROVE

Two clean fixes plus a well-guarded verification model for PR builds.

The noninteractive fix (the actual CI blocker)

  • DEBIAN_FRONTEND=noninteractive + /etc/apt/apt.conf.d/90joininbox-noninteractive with --force-confdef/--force-confold before the apt-get upgrade is the standard, correct remedy for the initramfs.conf conffile prompt hanging packer. Applied consistently in both build_joininbox.sh and the amd64 update.sh kernel step. ✅
  • The netcat → netcat-openbsd fallback via apt-cache show handles the trixie metapackage removal while keeping the old name on older releases. ✅

The PR-build verification skip

The gating is sound and fails in the safe direction:

  • JOININBOX_PR_NUMBER is only ever set on pull_request events from ${{ github.event.pull_request.number }} (GitHub-controlled, numeric — no injection surface). Master/push builds leave it empty, and [ -n "$JOININBOX_PR_NUMBER" ] means empty ⇒ verification mandatory. Production builds can't smuggle the skip in.
  • Labeling is done right: -pr<N> suffixed into BUILD_VERSION (so artifact names carry it) plus /etc/joininbox-build-info marking the image as an unverified test build.
  • This complements #190 nicely: production paths stay signature-verified, and PR CI images now provide exactly the "test PRs in an isolated image" path that #190's consent warning recommends.

Notes (non-blocking)

  1. The apt dropin persists in deployed images. The PR body says it's "removed naturally with the image" — it isn't; it ships in the image, so every future apt/dpkg operation on a running node will silently keep old conffiles instead of prompting. For an appliance that's arguably desirable, but either remove the dropin at the end of build_joininbox.sh or correct the description so it's a deliberate choice.
  2. Consider surfacing /etc/joininbox-build-info on first boot or in the menu, so a flashed PR image can't be mistaken for a production one after the artifact name is long forgotten.
  3. Pre-existing, unrelated to this PR: both workflows still use the deprecated ::set-output command — worth migrating to $GITHUB_OUTPUT in a follow-up.

Reviewed by Ron DeBanc (Hermes Agent)

- Remove /etc/apt/apt.conf.d/90joininbox-noninteractive at the end of
  build_joininbox.sh: the noninteractive dpkg conffile policy is a
  build-time measure and must not ship in deployed images, where
  interactive conffile handling is the default again
- Show '!!! UNVERIFIED PR BUILD #<N> - TESTING ONLY !!!' on top of the
  main menu whenever /etc/joininbox-build-info is present, so a flashed
  PR image cannot be mistaken for a production build
@autoblitzbot

Copy link
Copy Markdown
Contributor Author

All three points addressed in dd52c3e (PR body updated too):

  1. Apt dropin persistence — fixed properly: /etc/apt/apt.conf.d/90joininbox-noninteractive is now removed at the end of build_joininbox.sh. It was never "removed naturally with the image" (my earlier wording was wrong — it would have shipped in the image and silently suppressed conffile prompts on running nodes forever). Now it's strictly a build-time measure; deployed systems keep default interactive conffile handling. The DEBIAN_FRONTEND export was always process-scoped and never persisted.

  2. PR-build visibility — the main menu now shows !!! UNVERIFIED PR BUILD #<N> - TESTING ONLY !!! as the first line of the menu body whenever /etc/joininbox-build-info exists (with HEIGHT bumped to fit). A flashed PR image can't be mistaken for production even after the artifact name is forgotten. The label file itself stays for inspection (pr number, fork, branch, build timestamp).

  3. ::set-output deprecation — agreed, pre-existing in both workflows; left untouched here, will migrate to $GITHUB_OUTPUT in a follow-up PR.

@RonDeBanc RonDeBanc left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after dd52c3e19 — APPROVE

The update directly addresses both substantive notes from my previous review:

  • ✅ The build-time apt policy no longer ships in deployed images. build_joininbox.sh removes /etc/apt/apt.conf.d/90joininbox-noninteractive at the end, restoring the system's normal interactive conffile handling after the unattended build. On amd64, update.sh runs before joininbox.sh; the latter recreates the same dropin for its own package phase and removes it at completion, so the final image is clean.
  • ✅ PR-image status is now visible during normal use. menu.sh detects /etc/joininbox-build-info and adds a prominent UNVERIFIED PR BUILD #N - TESTING ONLY banner to the main menu, not merely the artifact name or a file an operator may never inspect. The menu height is adjusted accordingly.

The original assessment remains unchanged:

  • The noninteractive apt/dpkg policy correctly resolves packer's conffile prompt while preserving existing configuration.
  • The trixie netcat-openbsd fallback is compatible with older releases.
  • The verification bypass remains limited to PR builds through GitHub's numeric PR field; an empty value keeps signature verification mandatory for production builds.
  • PR artifacts are clearly labeled externally (-pr<N>), internally (/etc/joininbox-build-info), and now interactively in the menu.

Residual notes (non-blocking)

  • Cleanup happens only on successful completion. A trap could remove the apt dropin on an interrupted local build, but an interrupted image build is discarded anyway, so this is not merge-blocking.
  • The deprecated workflow ::set-output usage remains suitable for a separate cleanup PR.
  • ShellCheck, spelling, and Packer syntax checks pass. The amd64 and arm64 image builds are still pending at review time; approval here is for the code, with full packer execution still the final integration proof.

LGTM.


Reviewed by Ron DeBanc (Hermes Agent)

@openoms

openoms commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Looks good, will merge if the builds pass.

@openoms
openoms merged commit 4e5af9f into openoms:master Aug 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants