Skip to content

Fix O4c container-universe transfers and CPU image fallbacks - #200

Merged
oshaughn merged 2 commits into
oshaughn:rift_O4cfrom
oshaughnessy-junior:codex/o4c-container-selector-fix
Sep 26, 2026
Merged

oshaughn merged 2 commits into
oshaughn:rift_O4cfrom
oshaughnessy-junior:codex/o4c-container-selector-fix

Conversation

@oshaughnessy-junior

@oshaughnessy-junior oshaughnessy-junior commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

GPU container-universe ILE and CALPILOT jobs on rift_O4c put full URLs inside container_image. HTCondor truncates that selector before match-time expansion, which can hold every worker. This ports the basename-plus-URL-transfer fix from #165 and pins MY.TransferInput to avoid a spurious basename input.

CPU ILE and CALPILOT now use a literal fallback image with no GPU-dependent transfer expression. CPU ILE keeps --gpu --force-xpy --vectorized for the NoLoop likelihood path; these executable options are independent of requesting a physical GPU. CPU calibration reweighting resolves a manifest to its fallback instead of trying to launch the YAML as an image, matching #164's calibration fix. GPU families with local/CVMFS entries fail during generation with actionable guidance.

Validation outside GitHub CI

  • Initial combined container-manifest, CIP startup, and DAG postprocessing gate: 41 passed. After the independent review correction, the expanded container suite passes 33 tests, including eight condor_submit -dry-run cases covering list/string/empty-string ILE inputs and CPU/GPU ILE/CALPILOT ads.
  • Against the unfixed base, the new effective-ad cases give 3 failed, 1 passed.
  • A live CPU control using the merged selector was held because the GPU-capability transfer expression could not expand. The fixed CPU fallback and GPU-selector jobs both exited 0. These small container probes validate image deployment, not scientific ILE inside the probe images.
  • Separate real BBH ILE smoke evaluations on a GPU and on the CPU NoLoop path both exited 0 with finite result rows and parseable sample XML. These are operational smoke tests, not convergence validation.
  • git diff --check passes.

Independent adversarial review found that string-valued transfer_files concatenated the image selector directly onto a filename. The follow-up commit adds the missing comma and tests that the selector is a distinct transfer item. Independent re-review passed at aeb426d7d1df93fa46ec2d07a440c4ede87c4eb4, with ten additional real HTCondor dry-run cases covering CPU/GPU jobs and list, single-file string, multiple-file string, empty-string, and empty-list inputs. No remaining blocker was found in this PR delta.

Scope

This fixes image delivery and CPU fallback selection without changing likelihood code or normalization. Separate adversarial findings remain: shared selectors do not enforce capability upper bounds, and the optional runtime wrapper added in #164 can select by the first physical GPU instead of the assigned GPU. O4c does not implement that wrapper. The deployment checks above do not validate those behaviors.

@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift-upstream September 26, 2026 18:01 — with GitHub Actions Active

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent automated review completed at the recorded exact commit. Detailed findings were withheld from public output by the private-context egress policy and require private human declassification.

@oshaughnessy-junior
oshaughnessy-junior deployed to private-review-dispatch-rift-upstream September 26, 2026 18:18 — with GitHub Actions Active
@oshaughn
oshaughn merged commit fe3ffdd into oshaughn:rift_O4c Sep 26, 2026
5 checks passed
@oshaughn
oshaughn deployed to private-review-dispatch-rift-upstream September 26, 2026 18:19 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
private-review-dispatch-rift-upstream — aeb426d7 Deployed Sep 26, 2026 by oshaughn via Dispatch exact upstream RIFT PR generation #117
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants