Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/actions/setup-acceptance-documents/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,15 @@ provider. After platform package preparation, require successful `kpsewhich
soul.sty` resolution and an actual provider file; retain its path and SHA-256.
A package inventory or cached archive cannot replace that runtime lookup.

External callers select only converter mode and must supply `source-root`,
the pinned `source-sha` and an absolute `python-executable` from their
already synced locked virtual environment. Resolve the checkout as the
`docxology/template` Git root with that exact HEAD. The provider entrypoint,
manifest and build inputs come from a private export of the pinned commit.
Verify the caller's on-disk lock and project metadata against that export.
Key external caches on the Template source SHA and keep the local cache and
native same-run store rules in effect for in-repository callers.

The Linux mirror guard uses the action-owned `ubuntu_apt_mirrors.py` policy.
Admit only its exact original three-row ASCII serialization or the corresponding
extended form: Azure or a reviewed standard commercial EC2 regional Ubuntu host,
Expand Down
16 changes: 16 additions & 0 deletions .github/actions/setup-acceptance-documents/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,22 @@ and hosted cost must be measured after genuine execution.

See [AGENTS.md](AGENTS.md) for ownership and cleanup requirements.

## External converter callers

Pin both the action and the `docxology/template` checkout to full commit SHAs.
Check out Template into a separate directory with credentials persistence
disabled, then run `uv sync --locked --project .ci-template-sdk` with the
selected Python. Set `mode: converter`, `source-root: .ci-template-sdk`,
`source-sha` to the exact checkout SHA, and `python-executable` to
`${{ github.workspace }}/.ci-template-sdk/.venv/bin/python`. The pinned checkout
must contain the provider entrypoint and manifest.

The action verifies Template's checkout root, origin, HEAD and lock inputs.
It loads provider code and pins from a private export of that commit and keys
the external archive cache on the same commit. The caller's locked Python
environment runs the provider checks on Linux and macOS. Call this action
before making any local changes to Template's lock or project metadata.

## Frozen Linux TeX and tagged-PDF capability

Native Linux preparation installs one upstream TeX Live 2025 final tree from
Expand Down
62 changes: 58 additions & 4 deletions .github/actions/setup-acceptance-documents/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ inputs:
description: Already locked Python for Source provider preparation, the Linux mirror guard and native LaTeX validator; does not sync or acquire Python.
required: false
default: .venv/bin/python
source-root:
description: Template Git checkout for an external converter caller, relative to the caller workspace or absolute.
default: ""
source-sha:
description: Full commit SHA pinned by the external caller when checking out Template.
default: ""
nix-store-snapshot:
description: Optional flat preparation DATA from this exact workflow run; never a substitute for normal provider verification.
required: false
Expand All @@ -26,14 +32,59 @@ runs:
env:
PREPARATION_MODE: ${{ inputs.mode }}
PREPARED_PYTHON: ${{ inputs.python-executable }}
SOURCE_ROOT: ${{ inputs.source-root }}
SOURCE_SHA: ${{ inputs.source-sha }}
NIX_STORE_SNAPSHOT: ${{ inputs.nix-store-snapshot }}
NIX_STORE_MANIFEST_SHA256: ${{ inputs.nix-store-manifest-sha256 }}
run: |
set -euo pipefail
case "$PREPARATION_MODE" in
native|converter) echo "mode=$PREPARATION_MODE" >> "$GITHUB_OUTPUT" ;;
native|converter) ;;
*) echo "document preparation mode must be native or converter" >&2; exit 1 ;;
esac
provider_root="$GITHUB_WORKSPACE"
if [ -n "$SOURCE_ROOT" ] || [ -n "$SOURCE_SHA" ]; then
if [ "$PREPARATION_MODE" != converter ] || [ -z "$SOURCE_ROOT" ] || [[ ! "$SOURCE_SHA" =~ ^[0-9a-f]{40}$ ]]; then
echo 'An explicit Template checkout requires converter mode, source-root and a full source-sha' >&2
exit 1
fi
if [ -n "$NIX_STORE_SNAPSHOT" ] || [ -n "$NIX_STORE_MANIFEST_SHA256" ]; then
echo 'External converter preparation cannot restore native store preparation DATA' >&2
exit 1
fi
checkout=$(cd -- "$SOURCE_ROOT" && pwd -P)
if [ "$(git -C "$checkout" rev-parse --show-toplevel)" != "$checkout" ] ||
[ "$(git -C "$checkout" rev-parse --verify HEAD)" != "$SOURCE_SHA" ]; then
echo 'Template checkout root or HEAD does not match the supplied source pin' >&2
exit 1
fi
case "$(git -C "$checkout" remote get-url origin)" in
https://github.com/docxology/template|https://github.com/docxology/template.git) ;;
*) echo 'External converter preparation requires the docxology/template checkout' >&2; exit 1 ;;
esac
if [[ "$PREPARED_PYTHON" != /* ]] || [ ! -f "$PREPARED_PYTHON" ] || [ ! -x "$PREPARED_PYTHON" ]; then
echo 'External converter preparation requires an absolute python-executable from the locked environment' >&2
exit 1
fi
if ! "$PREPARED_PYTHON" -I -B -c 'import sys; raise SystemExit(sys.prefix == sys.base_prefix)'; then
echo 'External converter preparation requires a prepared Python virtual environment' >&2
exit 1
fi
# Import the pinned files only. The caller can keep its worktree and environment intact.
provider_root=$(mktemp -d "$RUNNER_TEMP/template-document-source.XXXXXX")
git -C "$checkout" archive "$SOURCE_SHA" infrastructure pyproject.toml uv.lock | tar -x -C "$provider_root"
if [ ! -f "$provider_root/infrastructure/reporting/native_document_preparation.py" ] ||
[ ! -f "$provider_root/infrastructure/config/native-document-providers.json" ]; then
echo 'Pinned Template source is missing its document preparation entrypoint or provider manifest' >&2
exit 1
fi
cmp "$checkout/pyproject.toml" "$provider_root/pyproject.toml"
cmp "$checkout/uv.lock" "$provider_root/uv.lock"
echo "cache-source=external-$SOURCE_SHA" >> "$GITHUB_OUTPUT"
elif ! [ "$GITHUB_ACTION_PATH" -ef "$GITHUB_WORKSPACE/.github/actions/setup-acceptance-documents" ]; then
echo 'External converter preparation requires explicit Template source-root and source-sha inputs' >&2
exit 1
fi
if [ "$PREPARATION_MODE" = native ] || [ "${RUNNER_OS:-}" = Linux ]; then
if [ ! -f "$PREPARED_PYTHON" ] || [ ! -x "$PREPARED_PYTHON" ]; then
echo "native/Linux document preparation requires an already locked Python executable" >&2
Expand All @@ -45,11 +96,13 @@ runs:
[ -d "$NIX_STORE_SNAPSHOT" ] && [ ! -L "$NIX_STORE_SNAPSHOT" ]
[[ "$NIX_STORE_MANIFEST_SHA256" =~ ^[0-9a-f]{64}$ ]]
fi
echo "mode=$PREPARATION_MODE" >> "$GITHUB_OUTPUT"
echo "provider-root=$provider_root" >> "$GITHUB_OUTPUT"
- name: Restore exact document provider archives
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ runner.temp }}/template-acceptance-document-archives
key: acceptance-document-archives-v3-${{ runner.os }}-${{ runner.arch }}-${{ steps.preparation-mode.outputs.mode }}-${{ hashFiles('infrastructure/config/native-document-providers.json', 'infrastructure/reporting/native_document_*.py', '.github/actions/setup-acceptance-documents/action.yml', '.github/actions/setup-acceptance-documents/ubuntu_apt_mirrors.py') }}
key: acceptance-document-archives-v3-${{ runner.os }}-${{ runner.arch }}-${{ steps.preparation-mode.outputs.mode }}-${{ steps.preparation-mode.outputs.cache-source || hashFiles('infrastructure/config/native-document-providers.json', 'infrastructure/reporting/native_document_*.py', '.github/actions/setup-acceptance-documents/action.yml', '.github/actions/setup-acceptance-documents/ubuntu_apt_mirrors.py') }}
- name: Prepare reviewed Linux acquisition and Nix
if: runner.os == 'Linux'
shell: bash
Expand Down Expand Up @@ -184,6 +237,7 @@ runs:
env:
PREPARED_PYTHON: ${{ inputs.python-executable }}
PREPARATION_MODE: ${{ steps.preparation-mode.outputs.mode }}
PROVIDER_ROOT: ${{ steps.preparation-mode.outputs.provider-root }}
run: |
set -euo pipefail
preparation_python="$PREPARED_PYTHON"
Expand All @@ -198,13 +252,13 @@ runs:
destination="$preparation/bin"
echo "diagnostics-directory=$destination" >> "$GITHUB_OUTPUT"
echo "diagnostics-id=$(basename "$preparation")" >> "$GITHUB_OUTPUT"
arguments=(--source-root "$GITHUB_WORKSPACE" --archives "$archives" --destination "$destination")
arguments=(--source-root "$PROVIDER_ROOT" --archives "$archives" --destination "$destination")
if [ "$RUNNER_OS" = Linux ]; then
arguments+=(--nix-executable "$(command -v nix)" --nix-sudo)
fi
"$preparation_python" -I -B -c \
'import runpy, sys; sys.path.insert(0, sys.argv.pop(1)); runpy.run_module("infrastructure.reporting.native_document_preparation", run_name="__main__")' \
"$GITHUB_WORKSPACE" "${arguments[@]}"
"$PROVIDER_ROOT" "${arguments[@]}"
echo "$destination" >> "$GITHUB_PATH"
echo "TEMPLATE_DOCUMENT_PROVIDER_PROVENANCE=$destination/document-provider-provenance.json" >> "$GITHUB_ENV"
- name: Retain original failed provider command diagnostics
Expand Down
90 changes: 89 additions & 1 deletion tests/infra_tests/project/test_public_capabilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -537,6 +537,10 @@ def test_document_preparation_requires_locked_python_for_native_or_linux(
NIX_STORE_SNAPSHOT="",
NIX_STORE_MANIFEST_SHA256="",
RUNNER_OS=system,
GITHUB_ACTION_PATH=str(repo / ".github/actions/setup-acceptance-documents"),
GITHUB_WORKSPACE=str(repo),
SOURCE_ROOT="",
SOURCE_SHA="",
GITHUB_OUTPUT=str(output),
),
capture_output=True,
Expand All @@ -545,11 +549,95 @@ def test_document_preparation_requires_locked_python_for_native_or_linux(
check=False,
)
assert result.returncode != 0
assert "requires an already locked Python executable" in result.stderr
assert {path.name for path in tmp_path.iterdir()} == (
{"output", "prepared-python"} if nomination == "not-executable" else {"output"}
{"prepared-python"} if nomination == "not-executable" else set()
)


@pytest.mark.parametrize(
"fault, diagnostic",
[
("none", ""),
("implicit-source", "requires explicit Template source-root and source-sha"),
("wrong-sha", "does not match the supplied source pin"),
("default-python", "requires an absolute python-executable"),
("missing-entrypoint", "missing its document preparation entrypoint or provider manifest"),
("changed-lock", "uv.lock"),
],
)
def test_document_preparation_binds_external_converter_source(tmp_path: Path, fault: str, diagnostic: str) -> None:
repo = Path(__file__).resolve().parents[3]
action_dir = repo / ".github/actions/setup-acceptance-documents"
action = yaml.safe_load((action_dir / "action.yml").read_text(encoding="utf-8"))
workspace = tmp_path / "consumer"
checkout = workspace / "Template SDK"
entrypoint = Path("infrastructure/reporting/native_document_preparation.py")
manifest = Path("infrastructure/config/native-document-providers.json")
tracked = {manifest: "{}", Path("pyproject.toml"): "", Path("uv.lock"): "pinned lock data"}
if fault != "missing-entrypoint":
tracked[entrypoint] = "# pinned entrypoint"
for relative, content in tracked.items():
path = checkout / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")

def git(*arguments: str) -> str:
return subprocess.check_output(["git", "-C", str(checkout), *arguments], text=True, timeout=5).strip()

git("init", "--quiet")
git("remote", "add", "origin", "https://github.com/docxology/template")
git("add", ".")
git("-c", "user.name=Template test", "-c", "user.email=test@example.com", "commit", "--quiet", "-m", "pin")
source_sha = git("rev-parse", "HEAD")

caller_entrypoint = workspace / entrypoint
caller_entrypoint.parent.mkdir(parents=True)
caller_entrypoint.write_text("# caller code", encoding="utf-8")
(checkout / entrypoint).parent.mkdir(parents=True, exist_ok=True)
(checkout / entrypoint).write_text("# edited checkout code", encoding="utf-8")
if fault == "changed-lock":
(checkout / "uv.lock").write_text("different lock data", encoding="utf-8")

venv = checkout / ".venv"
subprocess.run([sys.executable, "-m", "venv", "--without-pip", str(venv)], timeout=15, check=True)
output = tmp_path / "output"
environment = dict(
os.environ,
PREPARATION_MODE="converter",
PREPARED_PYTHON=".venv/bin/python" if fault == "default-python" else str(venv / "bin/python"),
SOURCE_ROOT="" if fault == "implicit-source" else checkout.name,
SOURCE_SHA={"implicit-source": "", "wrong-sha": "0" * 40}.get(fault, source_sha),
NIX_STORE_SNAPSHOT="",
NIX_STORE_MANIFEST_SHA256="",
RUNNER_OS="Linux",
RUNNER_TEMP=str(tmp_path),
GITHUB_WORKSPACE=str(workspace),
GITHUB_ACTION_PATH=str(action_dir),
GITHUB_OUTPUT=str(output),
)
result = subprocess.run(
["bash", "-c", action["runs"]["steps"][0]["run"]],
cwd=workspace,
env=environment,
capture_output=True,
text=True,
timeout=15,
check=False,
)
if fault != "none":
assert result.returncode != 0
assert diagnostic in result.stdout + result.stderr
assert not output.exists()
return

assert result.returncode == 0, result.stderr
values = dict(line.split("=", 1) for line in output.read_text(encoding="utf-8").splitlines())
assert values["mode"] == "converter"
assert values["cache-source"] == f"external-{source_sha}"
assert (Path(values["provider-root"]) / entrypoint).read_text(encoding="utf-8") == "# pinned entrypoint"


@pytest.mark.parametrize("fault", ["length", "sha256"])
def test_document_font_archive_cache_refuses_corruption_before_use(tmp_path: Path, fault: str) -> None:
"""Run the real font preparation against corrupt local data, without substitutes."""
Expand Down