Skip to content

task image: remove git history after HEAD so the upstream fix is not on disk - #46

Merged
ArjunS07 merged 3 commits into
survey/phase1-template-shadow-conftestfrom
fix/prune-git-history
Oct 8, 2026
Merged

ArjunS07 merged 3 commits into
survey/phase1-template-shadow-conftestfrom
fix/prune-git-history

Conversation

@ArjunS07

@ArjunS07 ArjunS07 commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Purpose

Task images keep the upstream history after the base commit. lock_repo_to_current_commit deletes branches but never prunes, so git fsck --unreachable shows the fix commit (networkx 8138: 2573bd3b, numpy 21464: cfbbde8ae7). This PR removes that history as the last step of the task image build, and fails the build if any remains.

How it works

src/datasmith/harbor_adapter/
├── adapter.py                        # copies scrub_git.sh next to entrypoint.sh
└── template/environment/
    ├── Dockerfile                    # runs scrub_git.sh last, after the baseline measurement
    └── scrub_git.sh                  # new
src/datasmith/docker/templates/docker_build_run.sh   # ref check now fails the build
tests/docker/test_scrub_git.py        # new

Changes

  1. scrub_git.sh in the task image.
    Effect: it deletes refs that are not ancestors of HEAD, expires reflogs, and runs gc --prune=now. It keeps ancestor tags, so git describe (and versioneer) gives the same version. It pins core.abbrev, so the short hash keeps its length. The build fails if HEAD, tree, describe or status change, or if any later commit or ref remains.
    Before: 144 to 29,479 commits newer than HEAD in every image checked (59 of 59).
  2. Ref check in docker_build_run.sh.
    Effect: a ref ahead of HEAD now makes lock_repo_to_current_commit return 1.
    Before: exit 0 inside the check loop ended the whole build script with success, so the check could never fail. Later steps in the script were skipped.
  3. The base image build still keeps unreachable objects. ASV may resolve raw SHAs at that stage. The task image layer hides them from the container.

Verification

  • pytest tests/docker/test_scrub_git.py: 2 passed. The test makes a repo with a later fix commit and tag, runs the script, and checks that HEAD and describe are equal, the later tag is gone, and the fix object is gone.
  • scrub_git.sh on real images (docker run --network none): networkx 8138, numpy 21464, and dask 11754 all print scrub_git: ok. numpy 21464 keeps v1.23.0.dev0-1185-g7987557291.
  • tests/docker and the adapter tests: 7 failures, the same 7 as on 5c3b08e without this change.

Notes

  • Based on 5c3b08e, the Phase 1 template. Changing the template changes render_source_sha256, so images need a rebuild to pick this up. Existing images get the same scrub as a thin layer from scripts/scrub_image_git.py (formulacode-verified-rl #72).
  • The origin URL stays: lsv_init.py and run-tests.sh get the package name from it. Trials need no network for it to be harmless (separate PR).

…on disk; fix the ref check that never failed
…nftest' into fix/prune-git-history

# Conflicts:
#	src/datasmith/harbor_adapter/template/environment/Dockerfile
@ArjunS07
ArjunS07 merged commit 1060b16 into survey/phase1-template-shadow-conftest Oct 8, 2026
@ArjunS07
ArjunS07 deleted the fix/prune-git-history branch October 8, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant