diff --git a/docs/release-management/spec.md b/docs/release-management/spec.md index 213a662cd..c4a39dbb8 100644 --- a/docs/release-management/spec.md +++ b/docs/release-management/spec.md @@ -411,7 +411,9 @@ loop before posting `+1`. previous release, no prohibited binaries, published-JVM-artefact compliance via `tools/maven-artifact-verify` — POM licence set, podling incubation disclaimer, companion `-sources.jar` / - `-javadoc.jar` with signatures and checksums — source-tree + `-javadoc.jar` with signatures and checksums, informational + observations (timestamp reproducibility signal, package/groupId + correspondence, companion content sanity) — source-tree integrity, version-string consistency, and — optional per `release-build.md § Reproducibility checks` — reproducibility: the source artefact rebuilt from the tag with `repro-archive build` at diff --git a/plugins/magpie-release-management/skills/verify-rc/jvm-artefacts.md b/plugins/magpie-release-management/skills/verify-rc/jvm-artefacts.md index 9cef4b147..dbd03911b 100644 --- a/plugins/magpie-release-management/skills/verify-rc/jvm-artefacts.md +++ b/plugins/magpie-release-management/skills/verify-rc/jvm-artefacts.md @@ -35,10 +35,33 @@ and [Maven Central's publishing requirements](https://central.sonatype.org/publi a companion with no `.asc` is already a finding and gets no line. A main jar declared by a staged POM but not staged locally is an observation (`ABSENT`), not a failure: in the common ASF workflow the jars are staged in the Nexus staging repository, which this step never reads - (read-only, and check 4 is a later PR on [#1173](https://github.com/apache/magpie/issues/1173)). + (read-only; the Nexus staging-repository check — check 4 of the issue — is the read-only `asf-nexus` adapter and + Step 6c, landing via [#1505](https://github.com/apache/magpie/pull/1505)). Classify an `ABSENT` jar against `release-build.md § JVM artefact checks` — when that file declares `jvm_companion_location: staged`, an absent jar is a `FAIL`. +The same tool run emits **informational observations** — checks 5–7 of [issue #1173](https://github.com/apache/magpie/issues/1173) — +which are signals for the reviewer and never change the step's verdict: + +5. **Timestamp reproducibility signal** — whether every file entry of a main jar shares one timestamp (consistent with + `project.build.outputTimestamp` being set) or varies across entries. Worded as "consistent / not consistent with a + reproducible configuration", never as "reproducible" — only Step 9's rebuild-and-compare can assert that. An empty or + single-entry jar reports `insufficient-data`, never a pass. +6. **Namespace and package/groupId correspondence** — whether the declared `groupId` sits under `org.apache.*` (informational even + for ASF top-level projects: published coordinates cannot be renamed retroactively, so there is no available remedy to gate on), + and the proportion of the jar's class entries under the package path derived from the groupId plus the package roots actually + found — a proportion and a list for the reviewer to judge, never a boolean. `META-INF/` entries, `module-info.class` and + multi-release overrides are excluded as legitimate divergences. Most useful for podlings, where it surfaces whether the + `org.apache.` rename has happened. +7. **Companion content sanity** — whether `-sources.jar` carries `.java` / `.scala` / `.kt` sources and no `.class` files, and + whether `-javadoc.jar` is non-empty. Placeholder companions are a Maven-Central-sanctioned pattern, reported as such and never + failed; no Javadoc-specific structure is asserted (Scala/Kotlin projects publish dokka/scaladoc output under the `-javadoc` + classifier). Classified jars (`-tests`, `-shaded`, …) are not part of the required set and are not inspected. + + A jar that cannot be opened at all — truncated, corrupt central directory, undecodable entry names — yields an `unreadable` + observation in each affected section and never takes the run down: check 3 never opens a jar, so a damaged jar with a valid + signature and checksum can pass the blocking checks while the observations report that its contents could not be read. + Emit the paste-ready recipe. Resolve every placeholder to a concrete value: `` is the framework root (`.apache-magpie` in an adopter repository), @@ -66,6 +89,7 @@ Return ONLY valid JSON with this structure: "tool_report": "", "pom_findings": [""], "companion_findings": [""], + "observations": [""], "paste_recipe": "" } ``` @@ -74,6 +98,10 @@ Return ONLY valid JSON with this structure: one line each naming the artefact and what is wrong; a passing check is not a finding, and an empty list means there is nothing to report. +`observations` carries the tool's informational observations (checks 5–7), one line each naming the jar and what was observed; +an empty list means the staged set carried none. They never change `status`: a jar whose timestamps vary, whose groupId sits +outside `org.apache.*`, or whose `-sources.jar` contains `.class` files still passes every blocking check. + `status` is the tool report's `status`, except that an `ABSENT` jar becomes `FAIL` when `release-build.md § JVM artefact checks` declares `jvm_companion_location: staged` (the RC was expected to stage it). diff --git a/tools/maven-artifact-verify/README.md b/tools/maven-artifact-verify/README.md index e3468e7e6..4cd411adc 100644 --- a/tools/maven-artifact-verify/README.md +++ b/tools/maven-artifact-verify/README.md @@ -26,9 +26,12 @@ Verifies **locally staged JVM release-candidate artefacts** — the `.pom` files, the main jars and their companion `-sources.jar` / `-javadoc.jar` — the way `release-verify-rc` verifies a staged source artefact today. Implements the blocking checks 1–3 proposed in -[apache/magpie#1173](https://github.com/apache/magpie/issues/1173); -the Nexus staging-repository check (check 4) and the informational -checks (5–7) are later PRs on that issue. +[apache/magpie#1173](https://github.com/apache/magpie/issues/1173) +and reports the issue's informational checks (5–7) as `observations` +in the same JSON report; the Nexus staging-repository check (check 4) +will be implemented by the read-only `tools/asf-nexus` adapter and +`release-verify-rc` Step 6c — pending merge as +[#1505](https://github.com/apache/magpie/pull/1505). Until this tool exists, `release-verify-rc` handles a jar in exactly one direction: as *contraband inside the source tree* (Step 6's @@ -77,10 +80,42 @@ blocking. the release key, which `release-verify-rc` Step 2 runs against the main artefacts, and the Step 6b recipe extends to the companions. +The same run also reports the issue's **informational checks 5–7** +under `observations` — signals for a human reviewer that never change +the `status`: + +5. **Timestamp reproducibility signal** — whether every file entry of + a main jar shares one timestamp (consistent with + `project.build.outputTimestamp` being set) or varies across + entries. Worded as "consistent / not consistent with a reproducible + configuration", never as "reproducible" — only a rebuild-and-compare + can assert that. An empty or single-entry jar reports + `insufficient-data`, never a pass. ZIP's MS-DOS entry times carry + 2-second granularity and no timezone; entries are compared as raw + values within one jar and never converted to absolute times. +6. **Namespace and package/groupId correspondence** — whether the + declared `groupId` sits under `org.apache.*` (informational even + for ASF top-level projects: published coordinates cannot be renamed + retroactively, so a gate would leave the RM no remedy), and the + proportion of the jar's class entries under the package path + derived from the groupId plus the package roots actually found — a + proportion and a list, never a boolean. `META-INF/` entries, + `module-info.class` and multi-release overrides are excluded as + legitimate divergences. Most useful for podlings, where it + surfaces whether the `org.apache.` rename has happened. +7. **Companion content sanity** — whether `-sources.jar` carries + `.java` / `.scala` / `.kt` sources and no `.class` files, and + whether `-javadoc.jar` is non-empty. Placeholder companions are a + Maven-Central-sanctioned pattern and are reported as such, never + failed; no Javadoc-specific structure is asserted (Scala/Kotlin + projects publish dokka/scaladoc output under the `-javadoc` + classifier). + The overall `status` is `FAIL` when any check fails, `WARN` when only `INHERITED-UNVERIFIED` results remain, `PASS` otherwise, and `SKIP` when the staged set contains no `.pom` and no `.jar` at all — a -non-JVM project's RC runs the tool and skips cleanly. +non-JVM project's RC runs the tool and skips cleanly. The +informational observations never change it. ## Prerequisites @@ -117,9 +152,14 @@ not fail correct releases: - `packaging=pom` modules have no jar and are exempt from check 3 (they still get checks 1 and 2). - Placeholder companion jars are a Maven-Central-sanctioned pattern; - check 3 only verifies presence, signatures and checksums, and never - opens a jar to judge its content (that is informational check 7, a - later PR). + check 3 verifies presence, signatures and checksums only, and the + check-7 observation reports a placeholder as the sanctioned pattern + it is, never a defect. Opening a jar reads the zip central + directory only (entry names and timestamps) — no entry content is + extracted. A jar that cannot be opened at all (truncated, corrupt + central directory, undecodable entry names) yields an `unreadable` + observation in each affected section — never a failure and never a + crash. - Classified jars (`-tests`, `-shaded`, `-linux-x86_64`, …) are neither mains nor companions: a jar whose classifier is not `sources`/`javadoc` and that no staged POM declares is reported in diff --git a/tools/maven-artifact-verify/src/maven_artifact_verify/__init__.py b/tools/maven-artifact-verify/src/maven_artifact_verify/__init__.py index f19ff219e..4cf4044a3 100644 --- a/tools/maven-artifact-verify/src/maven_artifact_verify/__init__.py +++ b/tools/maven-artifact-verify/src/maven_artifact_verify/__init__.py @@ -49,9 +49,35 @@ ``packaging=pom`` modules are exempt (no jar), classified jars (``-tests``, ``-shaded``, ...) are neither mains nor companions. +Informational observations (checks 5-7 of the same issue) are +reported alongside and **never** affect ``status`` - they are +signals for a human reviewer, not gates: + +5. **Timestamp reproducibility signal** - whether every file entry + of a main jar shares one timestamp (consistent with + ``project.build.outputTimestamp`` being set) or varies (not + consistent with one). The report never claims the jar is or is + not reproducible; an empty or single-entry jar reports + ``insufficient-data``. +6. **Namespace and package/groupId correspondence** - whether the + declared ``groupId`` sits under ``org.apache.*``, and how many of + the jar's class-file entries live under the package path derived + from the groupId, plus the package roots actually found. Reported + as a proportion and a root list, never a boolean verdict. +7. **Companion content sanity** - whether ``-sources.jar`` carries + ``.java`` / ``.scala`` / ``.kt`` sources and no ``.class`` files, + and whether ``-javadoc.jar`` is non-empty. Placeholder companions + are a Maven-Central-sanctioned pattern and are reported as such, + never failed. + +Opening a jar here reads the zip central directory only (entry +names and timestamps); no entry content is extracted. + The tool is stdlib-only and fully offline: it reads the staged -directory, never the network. Nexus staging-repository checks are out -of scope here (issue #1173, PR 2). +directory, never the network. The Nexus staging-repository check +(issue #1173, check 4) will be handled by the read-only +`tools/asf-nexus` adapter and `release-verify-rc` Step 6c — pending +merge as [#1505](https://github.com/apache/magpie/pull/1505). Output is a single JSON document on stdout, in the shape `release-verify-rc` Step 6b consumes. @@ -65,6 +91,7 @@ import re import sys import xml.etree.ElementTree as ET +import zipfile from pathlib import Path MAVEN_NS = "http://maven.apache.org/POM/4.0.0" @@ -495,6 +522,202 @@ def split_jar_name(name: str) -> tuple[str, str | None, str | None]: return m.group("stem"), m.group("version"), classifier +SOURCE_EXTENSIONS = (".java", ".scala", ".kt") + + +def timestamp_signal(jar: Path) -> dict: + """Check 5 - jar entry timestamp consistency (informational only). + + If every file entry of the jar shares one timestamp, the project + almost certainly set ``project.build.outputTimestamp``; if the + timestamps vary, it almost certainly did not. The signal is + deliberately worded to never assert reproducibility either way - + only a rebuild-and-compare (``release-verify-rc`` Step 9) can do + that. An empty or single-entry jar gives no signal and reports + ``insufficient-data``, never a pass. + + ZIP stores MS-DOS local times at 2-second granularity with no + timezone. Comparing entries *within one jar* needs neither a + tolerance nor a timezone assumption: the raw ``date_time`` tuples + are compared as-is and are never converted to absolute times. + """ + try: + with zipfile.ZipFile(jar) as archive: + times = [info.date_time for info in archive.infolist() if not info.is_dir()] + except (zipfile.BadZipFile, OSError, NotImplementedError, UnicodeDecodeError, ValueError) as exc: + return { + "jar": jar.name, + "signal": "unreadable", + "detail": f"not a readable zip archive: {exc}", + } + if len(times) <= 1: + return { + "jar": jar.name, + "signal": "insufficient-data", + "entries": len(times), + "detail": "an empty or single-entry jar gives no timestamp signal", + } + distinct = sorted(set(times)) + if len(distinct) == 1: + return { + "jar": jar.name, + "signal": "consistent", + "entries": len(times), + "distinct_timestamps": 1, + "detail": "every file entry shares one timestamp - consistent with a " + "reproducible configuration (project.build.outputTimestamp set); " + "this observation does not claim the jar is reproducible", + } + return { + "jar": jar.name, + "signal": "inconsistent", + "entries": len(times), + "distinct_timestamps": len(distinct), + "detail": "entry timestamps vary - not consistent with a reproducible " + "configuration (project.build.outputTimestamp likely unset); this " + "observation does not claim the jar is unreproducible", + } + + +def namespace_signal(jar: Path, pom: dict) -> dict: + """Check 6 - groupId namespace and package/groupId correspondence. + + Two observations, informational **even for ASF top-level + projects** (a released artefact can legitimately sit outside + ``org.apache.*`` for historical reasons, and package/groupId + divergence is frequently legitimate - shaded or relocated + dependencies, multi-release jars, intentional naming): (a) whether + the declared ``groupId`` sits under ``org.apache.*``, and (b) how + many of the jar's class-file entries live under the package path + derived from the groupId, plus the package roots actually found - + a proportion and a list for the reviewer to judge, never a boolean + verdict. Most useful for podlings, where it surfaces whether the + ``org.apache.`` rename has happened. + + ``META-INF/`` entries, ``module-info.class`` and + ``META-INF/versions//`` multi-release overrides are excluded + from both the proportion and the roots: they are legitimate + divergences, not signals. + """ + group_id = pom.get("group_id") or "" + expected_prefix = "/".join(part for part in group_id.split(".") if part) + try: + with zipfile.ZipFile(jar) as archive: + names = archive.namelist() + except (zipfile.BadZipFile, OSError, NotImplementedError, UnicodeDecodeError, ValueError) as exc: + return { + "jar": jar.name, + "group_id": group_id or None, + "under_org_apache": group_id == "org.apache" or group_id.startswith("org.apache."), + "signal": "unreadable", + "detail": f"not a readable zip archive: {exc}", + } + class_entries = [name for name in names if name.endswith(".class") and not name.startswith("META-INF/") and name != "module-info.class"] + observation: dict = { + "jar": jar.name, + "group_id": group_id or None, + "under_org_apache": group_id == "org.apache" or group_id.startswith("org.apache."), + } + if not class_entries: + observation["detail"] = "no class-file entries outside META-INF/ and module-info.class; no package/groupId correspondence to report" + return observation + expected = expected_prefix + "/" if expected_prefix else "" + matching = sum(1 for name in class_entries if name.startswith(expected)) if expected else 0 + roots = sorted({"/".join(name.split("/")[:3]) for name in class_entries}) + shown = roots[:10] + detail = ( + f"{matching}/{len(class_entries)} class entries under the package path '{expected_prefix}' derived from groupId '{group_id}'" + if expected + else f"POM declares no groupId; {len(class_entries)} class entries have no package path to compare against" + ) + detail += f"; package roots (first three segments): {', '.join(shown)}" + if len(roots) > len(shown): + detail += f" (first {len(shown)} of {len(roots)} distinct roots)" + observation.update( + { + "class_entries": len(class_entries), + "matching_entries": matching, + "package_roots": shown, + "detail": detail, + } + ) + return observation + + +def companion_content_signal(companion: Path, classifier: str) -> dict: + """Check 7 - companion jar content sanity (informational only). + + Whether ``-sources.jar`` carries ``.java`` / ``.scala`` / ``.kt`` + sources and no ``.class`` files, and whether ``-javadoc.jar`` is + non-empty - both observations only. Placeholder companions are + explicitly permitted by Maven Central and are reported as such, + never failed; the javadoc side never asserts a Javadoc-specific + internal structure (Scala/Kotlin projects publish scaladoc/dokka + output under the ``-javadoc`` classifier for Central compliance). + Classified jars other than the two companions (``-tests``, + ``-shaded``, ...) are not part of the required set and are not + inspected here. + """ + try: + with zipfile.ZipFile(companion) as archive: + names = archive.namelist() + except (zipfile.BadZipFile, OSError, NotImplementedError, UnicodeDecodeError, ValueError) as exc: + return { + "jar": companion.name, + "kind": classifier, + "signal": "unreadable", + "detail": f"not a readable zip archive: {exc}", + } + file_entries = [name for name in names if not name.endswith("/")] + content_entries = [name for name in file_entries if not name.startswith("META-INF/")] + if classifier == "sources": + if not content_entries: + return { + "jar": companion.name, + "kind": "sources", + "signal": "placeholder", + "detail": "empty or MANIFEST-only jar - placeholder companions are a Maven-Central-sanctioned pattern; observation only", + } + class_files = [name for name in content_entries if name.lower().endswith(".class")] + if class_files: + return { + "jar": companion.name, + "kind": "sources", + "signal": "contains-class-files", + "detail": f"{len(class_files)} .class entries inside a -sources.jar " + "(compiled code in the sources companion); observation only, never a failure", + } + sources = [name for name in content_entries if name.lower().endswith(SOURCE_EXTENSIONS)] + if sources: + return { + "jar": companion.name, + "kind": "sources", + "signal": "sources-present", + "detail": f"{len(sources)} .java/.scala/.kt source entries; no .class entries", + } + return { + "jar": companion.name, + "kind": "sources", + "signal": "no-sources-found", + "detail": "no .java/.scala/.kt and no .class entries inside the -sources.jar; observation only", + } + if not content_entries: + return { + "jar": companion.name, + "kind": "javadoc", + "signal": "placeholder", + "detail": "empty or MANIFEST-only jar - placeholder companions are a " + "Maven-Central-sanctioned pattern; observation only (content is not " + "structure-asserted: dokka/scaladoc output is equally valid)", + } + return { + "jar": companion.name, + "kind": "javadoc", + "signal": "content-present", + "detail": f"{len(content_entries)} non-META-INF entries; documentation layout is not judged", + } + + def verify_staged_dir(staged_dir: Path, digests: list[str], podling: bool) -> dict: # rglob, not glob: a staging directory in Maven-repository layout # (org/apache/foo/foo-core/1.0.0/...) is a JVM artefact set too — a @@ -512,6 +735,7 @@ def verify_staged_dir(staged_dir: Path, digests: list[str], podling: bool) -> di "jars": [], "unmatched_jars": [], "findings": [], + "observations": {"timestamp_signal": [], "namespace_signal": [], "companion_content": []}, } if not poms and not jars: @@ -545,7 +769,8 @@ def verify_staged_dir(staged_dir: Path, digests: list[str], podling: bool) -> di report["poms"].append(entry) # --- check 3: per main jar --- - main_jars = [] + main_jars: list[Path] = [] + main_jar_poms: dict[Path, dict] = {} for pom_path, data in parsed.items(): if "error" in data or data["packaging"] == "pom": continue @@ -565,6 +790,7 @@ def verify_staged_dir(staged_dir: Path, digests: list[str], podling: bool) -> di main = pom_path.parent / f"{data['artifact_id']}-{data['version']}.jar" if main.exists(): main_jars.append(main) + main_jar_poms[main] = data else: # The jar is published via the Nexus staging repository # in the common ASF workflow and is not staged locally @@ -608,6 +834,24 @@ def verify_staged_dir(staged_dir: Path, digests: list[str], podling: bool) -> di for main in main_jars: report["jars"].append(check_companions(main, digests, report["findings"])) + # --- informational observations (checks 5-7) --- + # These are signals for a human reviewer, never gates: the + # aggregation below reads only report["poms"] and report["jars"], + # so the observations are structurally excluded from the verdict — + # not ordered after it. A jar whose timestamps vary, whose groupId + # sits outside org.apache.*, or whose -sources.jar contains .class + # files still passes every blocking check. + observations = report["observations"] + for main in main_jars: + observations["timestamp_signal"].append(timestamp_signal(main)) + pom_data = main_jar_poms.get(main) + if pom_data is not None: + observations["namespace_signal"].append(namespace_signal(main, pom_data)) + for classifier in COMPANION_CLASSIFIERS: + companion = main.with_name(main.name[: -len(".jar")] + f"-{classifier}.jar") + if companion.exists(): + observations["companion_content"].append(companion_content_signal(companion, classifier)) + # --- aggregate --- statuses = [] for entry in report["poms"]: @@ -645,7 +889,9 @@ def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser( prog="maven-artifact-verify", description="Verify locally staged JVM release-candidate artefacts " - "(POM licence set, podling disclaimer, companion jars). " + "(POM licence set, podling disclaimer, companion jars) and report " + "informational observations (timestamp reproducibility signal, " + "package/groupId correspondence, companion content sanity). " "Stdlib-only and offline; prints one JSON report.", ) parser.add_argument("staged_dir", type=Path, help="directory holding the staged .pom / .jar artefacts") diff --git a/tools/maven-artifact-verify/tests/test_maven_artifact_verify.py b/tools/maven-artifact-verify/tests/test_maven_artifact_verify.py index 13acd82cc..81cd1a121 100644 --- a/tools/maven-artifact-verify/tests/test_maven_artifact_verify.py +++ b/tools/maven-artifact-verify/tests/test_maven_artifact_verify.py @@ -29,6 +29,8 @@ import zipfile from pathlib import Path +import pytest + import maven_artifact_verify as mav DISCLAIMER = ( @@ -738,3 +740,272 @@ def test_missing_directory_fails(tmp_path: Path) -> None: report = json.loads(buffer.getvalue()) assert code == 2 assert report["status"] == "FAIL" + + +# --- informational observations: checks 5-7 (never affect status) -------- + + +def write_timed_jar(directory: Path, name: str, entries: dict[str, tuple[int, int, int, int, int, int]]) -> Path: + """Write a jar whose entries carry explicit MS-DOS date_time stamps.""" + path = directory / name + with zipfile.ZipFile(path, "w") as zf: + for entry, stamp in entries.items(): + info = zipfile.ZipInfo(entry, date_time=stamp) + info.external_attr = 0o644 << 16 + zf.writestr(info, b"content") + return path + + +def signed_companion(jar: Path, digest: str = "sha512") -> None: + (jar.parent / f"{jar.name}.asc").write_bytes(b"sig") + write_checksum(jar, digest) + + +def observations(report: dict) -> dict: + return report["observations"] + + +def test_timestamp_consistent_signal(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_staged(tmp_path) + stamp = (2026, 1, 1, 12, 0, 0) + write_timed_jar(tmp_path, "foo-core-1.0.0.jar", {f"org/apache/foo/C{i}.class": stamp for i in range(3)}) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + signal = observations(report)["timestamp_signal"][0] + assert signal["signal"] == "consistent" + assert signal["entries"] == 3 and signal["distinct_timestamps"] == 1 + assert "does not claim the jar is reproducible" in signal["detail"] + + +def test_timestamp_inconsistent_signal_never_fails(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_staged(tmp_path) + write_timed_jar( + tmp_path, + "foo-core-1.0.0.jar", + {"org/apache/foo/A.class": (2026, 1, 1, 12, 0, 0), "org/apache/foo/B.class": (2026, 1, 2, 12, 0, 2)}, + ) + report = json.loads(mav_json(tmp_path, ())) + # Varying timestamps are an observation, never a failure: the + # blocking checks all pass and the status stays PASS. + assert report["status"] == "PASS" + signal = observations(report)["timestamp_signal"][0] + assert signal["signal"] == "inconsistent" + assert signal["distinct_timestamps"] == 2 + assert "does not claim the jar is unreproducible" in signal["detail"] + + +def test_timestamp_insufficient_data_for_single_entry_jar(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_staged(tmp_path) + write_timed_jar(tmp_path, "foo-core-1.0.0.jar", {"org/apache/foo/A.class": (2026, 1, 1, 12, 0, 0)}) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + signal = observations(report)["timestamp_signal"][0] + assert signal["signal"] == "insufficient-data" + assert signal["entries"] == 1 + + +def test_namespace_proportion_and_roots(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_staged(tmp_path) + write_timed_jar( + tmp_path, + "foo-core-1.0.0.jar", + { + "org/apache/foo/A.class": (2026, 1, 1, 0, 0, 0), + "org/apache/foo/impl/B.class": (2026, 1, 1, 0, 0, 0), + "com/example/shaded/C.class": (2026, 1, 1, 0, 0, 0), + "module-info.class": (2026, 1, 1, 0, 0, 0), + "META-INF/versions/9/D.class": (2026, 1, 1, 0, 0, 0), + "META-INF/MANIFEST.MF": (2026, 1, 1, 0, 0, 0), + }, + ) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + signal = observations(report)["namespace_signal"][0] + assert signal["under_org_apache"] is True + # module-info, META-INF/versions and META-INF itself are excluded: + # they are legitimate divergences, not signals. + assert signal["class_entries"] == 3 + assert signal["matching_entries"] == 2 + assert "2/3" in signal["detail"] + assert "org/apache/foo" in signal["package_roots"] + assert "com/example/shaded" in signal["package_roots"] + + +def test_group_id_outside_org_apache_is_observation_only(tmp_path: Path) -> None: + write_pom( + tmp_path, + "foo-core-1.0.0.pom", + pom_xml(group_id="com.github.foo", licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM), + ) + write_staged(tmp_path) + write_timed_jar(tmp_path, "foo-core-1.0.0.jar", {"com/github/foo/A.class": (2026, 1, 1, 0, 0, 0)}) + report = json.loads(mav_json(tmp_path, ())) + # A groupId outside org.apache.* is real policy, but deliberately + # informational: a published artefact's coordinates cannot be + # changed retroactively, so failing the RC would leave the RM no + # remedy. The status must stay PASS. + assert report["status"] == "PASS" + signal = observations(report)["namespace_signal"][0] + assert signal["under_org_apache"] is False + assert signal["matching_entries"] == 1 + + +def test_sources_jar_with_class_files_is_observed_not_failed(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_timed_jar(tmp_path, "foo-core-1.0.0.jar", {"org/apache/foo/A.class": (2026, 1, 1, 0, 0, 0)}) + sources = write_timed_jar( + tmp_path, + "foo-core-1.0.0-sources.jar", + {"org/apache/foo/A.java": (2026, 1, 1, 0, 0, 0), "org/apache/foo/B.class": (2026, 1, 1, 0, 0, 0)}, + ) + signed_companion(sources) + javadoc = write_timed_jar(tmp_path, "foo-core-1.0.0-javadoc.jar", {"index.html": (2026, 1, 1, 0, 0, 0)}) + signed_companion(javadoc) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + content = {entry["kind"]: entry for entry in observations(report)["companion_content"]} + assert content["sources"]["signal"] == "contains-class-files" + assert "never a failure" in content["sources"]["detail"] + assert content["javadoc"]["signal"] == "content-present" + + +def test_placeholder_companions_are_sanctioned(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_timed_jar(tmp_path, "foo-core-1.0.0.jar", {"org/apache/foo/A.class": (2026, 1, 1, 0, 0, 0)}) + sources = write_timed_jar(tmp_path, "foo-core-1.0.0-sources.jar", {"META-INF/MANIFEST.MF": (2026, 1, 1, 0, 0, 0)}) + signed_companion(sources) + javadoc = write_timed_jar(tmp_path, "foo-core-1.0.0-javadoc.jar", {"META-INF/MANIFEST.MF": (2026, 1, 1, 0, 0, 0)}) + signed_companion(javadoc) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + content = {entry["kind"]: entry for entry in observations(report)["companion_content"]} + assert content["sources"]["signal"] == "placeholder" + assert content["javadoc"]["signal"] == "placeholder" + assert "Maven-Central-sanctioned" in content["sources"]["detail"] + + +def test_scala_and_kotlin_sources_count_as_sources(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_timed_jar(tmp_path, "foo-core-1.0.0.jar", {"org/apache/foo/A.class": (2026, 1, 1, 0, 0, 0)}) + sources = write_timed_jar( + tmp_path, + "foo-core-1.0.0-sources.jar", + {"org/apache/foo/A.scala": (2026, 1, 1, 0, 0, 0), "org/apache/foo/B.kt": (2026, 1, 1, 0, 0, 0)}, + ) + signed_companion(sources) + javadoc = write_timed_jar(tmp_path, "foo-core-1.0.0-javadoc.jar", {"doc/index.html": (2026, 1, 1, 0, 0, 0)}) + signed_companion(javadoc) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + content = {entry["kind"]: entry for entry in observations(report)["companion_content"]} + # Scala/Kotlin projects publish their own source and doc formats; + # neither the sources nor the javadoc side may assert a + # Javadoc-specific layout. + assert content["sources"]["signal"] == "sources-present" + assert content["javadoc"]["signal"] == "content-present" + + +def test_absent_main_jar_yields_no_observations(tmp_path: Path) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + assert report["jars"][0]["companions"][0]["classification"] == "ABSENT" + assert observations(report)["timestamp_signal"] == [] + assert observations(report)["namespace_signal"] == [] + assert observations(report)["companion_content"] == [] + + +def test_unreadable_jars_are_observations_not_crashes(tmp_path: Path) -> None: + # A zero-byte or truncated jar with a matching .asc and checksum + # passes check 3 (bytes verified); the observations that open the + # jar must degrade to an observation instead of crashing the run - + # otherwise an informational check takes down the blocking report. + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + main = tmp_path / "foo-core-1.0.0.jar" + main.write_bytes(b"not a zip") + for classifier in ("sources", "javadoc"): + companion = tmp_path / f"foo-core-1.0.0-{classifier}.jar" + companion.write_bytes(b"not a zip") + signed_companion(companion) + report = json.loads(mav_json(tmp_path, ())) + assert report["status"] == "PASS" + assert report["findings"] == [] + observations = report["observations"] + assert observations["timestamp_signal"][0]["signal"] == "unreadable" + assert "not a readable zip archive" in observations["timestamp_signal"][0]["detail"] + assert observations["namespace_signal"][0]["signal"] == "unreadable" + assert {entry["signal"] for entry in observations["companion_content"]} == {"unreadable"} + + +# --- damaged jars beyond the plain b"not a zip" case ---------------------- + + +def _set_zip_flag(data: bytes, sig: bytes, off: int) -> bytes: + raw = bytearray(data) + i = raw.find(sig) + flags = int.from_bytes(raw[i + off : i + off + 2], "little") + raw[i + off : i + off + 2] = (flags | 0x800).to_bytes(2, "little") + return bytes(raw) + + +def write_damaged_jar(path: Path, variant: str) -> None: + """Write a jar damaged in the way `variant` names. + + - ``not-a-zip``: bytes no zip reader accepts (BadZipFile). + - ``invalid-utf8-name``: entry name bytes that are not valid UTF-8 + with the UTF-8 flag (bit 11) set in both headers - zipfile + decodes flagged names strictly, so this raises + ``UnicodeDecodeError`` (a ``ValueError`` subclass). + - ``high-version-bytes``: the central directory's one-byte "version + needed to extract" field patched to 12.9 - zipfile rejects it + while parsing the central directory with ``NotImplementedError``. + """ + import io + + if variant == "not-a-zip": + path.write_bytes(b"not a zip") + return + buf = io.BytesIO() + with zipfile.ZipFile(buf, "w") as zf: + zf.writestr("AAAAAAAAAA.class", b"x") + raw: bytearray = bytearray(buf.getvalue()) + if variant == "invalid-utf8-name": + raw = bytearray(_set_zip_flag(bytes(raw), b"PK\x03\x04", 6)) + raw = bytearray(_set_zip_flag(bytes(raw), b"PK\x01\x02", 8)) + bad = bytes(range(0xF0, 0x100)) # 16 bytes, none valid UTF-8 lead/continuation + assert len(bad) == 16 + raw = bytearray(bytes(raw).replace(b"AAAAAAAAAA.class", bad)) + elif variant == "high-version-bytes": + i = raw.find(b"PK\x01\x02") + raw[i + 6] = 129 # version 12.9; the next byte is the host system + else: + raise ValueError(f"unknown variant: {variant}") + path.write_bytes(bytes(raw)) + + +DAMAGED_VARIANTS = ["not-a-zip", "invalid-utf8-name", "high-version-bytes"] + + +@pytest.mark.parametrize("variant", DAMAGED_VARIANTS) +def test_damaged_jar_variants_emit_the_report(tmp_path: Path, variant: str) -> None: + write_pom(tmp_path, "foo-core-1.0.0.pom", pom_xml(licenses=APACHE_LICENSES, developers=DEVELOPERS, scm=SCM)) + write_damaged_jar(tmp_path / "foo-core-1.0.0.jar", variant) + for classifier in ("sources", "javadoc"): + companion = tmp_path / f"foo-core-1.0.0-{classifier}.jar" + write_damaged_jar(companion, variant) + signed_companion(companion) + report = json.loads(mav_json(tmp_path, ())) + # The invariant the observations promise: whatever the damage, the + # JSON report is emitted, check 3 still passes (bytes verified), and + # the blocking verdict is exactly what it would be without the + # observations. + assert report["status"] == "PASS" + assert report["findings"] == [] + observations = report["observations"] + assert observations["timestamp_signal"][0]["signal"] == "unreadable" + assert observations["namespace_signal"][0]["signal"] == "unreadable" + assert {entry["signal"] for entry in observations["companion_content"]} == {"unreadable"} diff --git a/tools/skill-evals/README.md b/tools/skill-evals/README.md index 803e71872..86a27d6be 100644 --- a/tools/skill-evals/README.md +++ b/tools/skill-evals/README.md @@ -76,7 +76,7 @@ Suites are currently implemented for: - **release-prepare** — 15 cases across 4 suites (step-0-preflight, step-1-plan, step-14-post, step-2-prep) - **release-promote** — 9 cases across 2 suites (step-0-preflight, step-2-emit-commands) - **release-rc-cut** — 16 cases across 4 suites (step-0-preflight, step-2-tag-build-sign, step-2b-reproducibility, step-3-staging) -- **release-verify-rc** — 22 cases across 8 suites (step-0-preflight, step-2-verify-signatures, step-3-verify-checksums, step-5-notice-license, step-6-binary-exclusion, step-6b-jvm-artefacts, step-8-version-consistency, step-9-reproducibility) +- **release-verify-rc** — 24 cases across 8 suites (step-0-preflight, step-2-verify-signatures, step-3-verify-checksums, step-5-notice-license, step-6-binary-exclusion, step-6b-jvm-artefacts, step-8-version-consistency, step-9-reproducibility) - **release-vote-draft** — 9 cases across 3 suites (step-0-preflight, step-2-vote-draft, step-3-planning-comment) - **release-vote-tally** — 9 cases across 3 suites (step-0-preflight, step-2-classify, step-3-tally) - **reviewer-routing** — 7 cases across 2 suites (step-0-preflight, step-score-and-propose) diff --git a/tools/skill-evals/evals/release-verify-rc/README.md b/tools/skill-evals/evals/release-verify-rc/README.md index 3b96b6fc9..90d69efc1 100644 --- a/tools/skill-evals/evals/release-verify-rc/README.md +++ b/tools/skill-evals/evals/release-verify-rc/README.md @@ -15,11 +15,11 @@ Behavioural eval suite for the | `step-3-verify-checksums` | Step 3 — Verify checksums | 2 | Checksum classification (PASS/MISMATCH/MISSING-DIGEST), deprecated md5 detection | | `step-5-notice-license` | Step 5 — NOTICE/LICENSE presence | 2 | File presence (PASS/WARN/FAIL), diff-lines count, diff summary | | `step-6-binary-exclusion` | Step 6 — Binary exclusion check | 2 | Prohibited-binary detection (PASS/FAIL), expected-binary classification | -| `step-6b-jvm-artefacts` | Step 6b — JVM artefact checks | 4 | Tool-report classification (PASS/WARN/FAIL), POM licence/`INHERITED-UNVERIFIED` handling, companion signature detection, `ABSENT` jar vs `jvm_companion_location` | +| `step-6b-jvm-artefacts` | Step 6b — JVM artefact checks | 6 | Tool-report classification (PASS/WARN/FAIL), POM licence/`INHERITED-UNVERIFIED` handling, companion signature detection, `ABSENT` jar vs `jvm_companion_location`, informational observations (timestamp signal, package/groupId correspondence, companion content) that never change the verdict | | `step-8-version-consistency` | Step 8 — Version string consistency | 2 | Exact version match across manifest files (PASS/FAIL) | | `step-9-reproducibility` | Step 9 — Reproducibility checks | 5 | `repro-archive compare` verdict → status (`identical` PASS, `differs` FAIL, `content-identical` WARN in RM-key mode / FAIL under automated signing), `mandatory` under `automated_release_signing: enabled` with `--skip-repro` ignored, `trusted_hardware_asserted` mirrors the flag, a project-specific convenience artefact whose rebuild differs (source identical, artefact `FAIL`, named in `binaries.differs`) | -Total: **22 cases** across 8 step suites. +Total: **24 cases** across 8 step suites. ## Run @@ -77,7 +77,12 @@ which are graded semantically. failed; an `ABSENT` jar is a `"FAIL"` only when `release-build.md` declares `jvm_companion_location: staged`, and an observation otherwise; classified jars (`-tests`, `-shaded`, …) are never - flagged in either direction. + flagged in either direction; the informational observations + (timestamp signal, package/groupId correspondence, companion + content) never change the status and never assert reproducibility + either way — an empty or single-entry jar is `insufficient-data`, + and a placeholder companion is the sanctioned pattern, not a + defect. - **Step 8**: `status` must be `"FAIL"` for any `match: false` or `extracted: null`; dev/snapshot suffixes are always `match: false`. - **Step 9**: `differs` and `tag-moved` are always `"FAIL"`; diff --git a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-5-observations-never-fail/expected.json b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-5-observations-never-fail/expected.json new file mode 100644 index 000000000..7c281225d --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-5-observations-never-fail/expected.json @@ -0,0 +1,13 @@ +{ + "step": "jvm-artefacts", + "status": "PASS", + "pom_findings": [], + "companion_findings": [], + "observations": [ + "foo-core-1.0.0.jar: entry timestamps vary (4 distinct across 118 entries) - not consistent with a reproducible configuration (project.build.outputTimestamp likely unset); observation does not claim the jar is unreproducible", + "foo-core-1.0.0.jar: groupId org.apache.foo is under org.apache.*; 45/47 class entries under the derived package path org/apache/foo; roots: com/example/relocated, org/apache/foo", + "foo-core-1.0.0-sources.jar: contains 3 .class entries (compiled code in the sources companion) - observation only, never a failure", + "foo-core-1.0.0-javadoc.jar: placeholder (empty or MANIFEST-only) - a Maven-Central-sanctioned pattern, not a defect" + ], + "paste_recipe": "uv run --project .apache-magpie/tools/maven-artifact-verify maven-artifact-verify \"dist/dev/foo/1.0.0-rc1\" --digests sha512 --podling" +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-5-observations-never-fail/report.md b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-5-observations-never-fail/report.md new file mode 100644 index 000000000..bf791c14b --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-5-observations-never-fail/report.md @@ -0,0 +1,64 @@ + + +release-build.md § Digest set: sha512. § JVM artefact checks: +`jvm_companion_location: nexus-staging`. The RC is a podling (a +`DISCLAIMER` file ships at the source artefact root). + +maven-artifact-verify JSON report (verbatim): + +```json +{ + "tool": "maven-artifact-verify", + "status": "PASS", + "artefact_dir": "dist/dev/foo/1.0.0-rc1", + "podling": true, + "digests": ["sha512"], + "poms": [ + { + "pom": "foo-core-1.0.0.pom", + "packaging": "jar", + "check1": {"licenses": "PASS", "developers": "PASS", "scm": "PASS"}, + "check2": {"disclaimer": "PASS", "disclaimer_detail": null} + } + ], + "jars": [ + {"jar": "foo-core-1.0.0.jar", "companions": [ + {"companion": "foo-core-1.0.0-sources.jar", "classification": "PASS", "detail": null}, + {"companion": "foo-core-1.0.0-javadoc.jar", "classification": "PASS", "detail": null} + ]} + ], + "unmatched_jars": [], + "findings": [], + "observations": { + "timestamp_signal": [ + {"jar": "foo-core-1.0.0.jar", "signal": "inconsistent", "entries": 118, "distinct_timestamps": 4, + "detail": "entry timestamps vary - not consistent with a reproducible configuration (project.build.outputTimestamp likely unset); this observation does not claim the jar is unreproducible"} + ], + "namespace_signal": [ + {"jar": "foo-core-1.0.0.jar", "group_id": "org.apache.foo", "under_org_apache": true, + "class_entries": 47, "matching_entries": 45, + "package_roots": ["com/example/relocated", "org/apache/foo"], + "detail": "45/47 class entries under the package path 'org/apache/foo' derived from groupId 'org.apache.foo'; package roots (first three segments): com/example/relocated, org/apache/foo"} + ], + "companion_content": [ + {"jar": "foo-core-1.0.0-sources.jar", "kind": "sources", "signal": "contains-class-files", + "detail": "3 .class entries inside a -sources.jar (compiled code in the sources companion); observation only, never a failure"}, + {"jar": "foo-core-1.0.0-javadoc.jar", "kind": "javadoc", "signal": "placeholder", + "detail": "empty or MANIFEST-only jar - placeholder companions are a Maven-Central-sanctioned pattern; observation only (content is not structure-asserted: dokka/scaladoc output is equally valid)"} + ] + } +} +``` + +Three things to classify here: + +- Every blocking check passes: the POM set is clean, both companions + are staged with `.asc` and checksums. The status is the tool's + status. +- The observations — varying entry timestamps, two relocated class + roots, `.class` files inside the sources companion, a placeholder + javadoc companion — are signals for the reviewer. None of them may + change the verdict: the placeholder is Maven-Central-sanctioned, + and the observations never assert reproducibility either way. +- The podling signal is present, so `--podling` stays in the recipe. diff --git a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-6-namespace-outside-org-apache/expected.json b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-6-namespace-outside-org-apache/expected.json new file mode 100644 index 000000000..01af941a5 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-6-namespace-outside-org-apache/expected.json @@ -0,0 +1,13 @@ +{ + "step": "jvm-artefacts", + "status": "PASS", + "pom_findings": [], + "companion_findings": [], + "observations": [ + "foo-core-2.1.0.jar: every file entry shares one timestamp (204 entries, 1 distinct) - consistent with a reproducible configuration (project.build.outputTimestamp set); observation does not claim the jar is reproducible", + "foo-core-2.1.0.jar: groupId com.github.foo is NOT under org.apache.* (informational even for ASF projects - published coordinates cannot be renamed retroactively); 24/24 class entries under the derived package path com/github/foo; roots: com/github/foo", + "foo-core-2.1.0-sources.jar: 31 .java/.scala/.kt source entries; no .class entries", + "foo-core-2.1.0-javadoc.jar: 58 non-META-INF entries; documentation layout is not judged" + ], + "paste_recipe": "uv run --project .apache-magpie/tools/maven-artifact-verify maven-artifact-verify \"dist/dev/foo/2.1.0-rc1\" --digests sha512" +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-6-namespace-outside-org-apache/report.md b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-6-namespace-outside-org-apache/report.md new file mode 100644 index 000000000..645677862 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/case-6-namespace-outside-org-apache/report.md @@ -0,0 +1,65 @@ + + +release-build.md § Digest set: sha512. § JVM artefact checks: +`jvm_companion_location: nexus-staging`. No `DISCLAIMER` in the source +artefact — the project graduated from the Incubator years ago. + +maven-artifact-verify JSON report (verbatim): + +```json +{ + "tool": "maven-artifact-verify", + "status": "PASS", + "artefact_dir": "dist/dev/foo/2.1.0-rc1", + "podling": false, + "digests": ["sha512"], + "poms": [ + { + "pom": "foo-core-2.1.0.pom", + "packaging": "jar", + "check1": {"licenses": "PASS", "developers": "PASS", "scm": "PASS"} + } + ], + "jars": [ + {"jar": "foo-core-2.1.0.jar", "companions": [ + {"companion": "foo-core-2.1.0-sources.jar", "classification": "PASS", "detail": null}, + {"companion": "foo-core-2.1.0-javadoc.jar", "classification": "PASS", "detail": null} + ]} + ], + "unmatched_jars": [], + "findings": [], + "observations": { + "timestamp_signal": [ + {"jar": "foo-core-2.1.0.jar", "signal": "consistent", "entries": 204, "distinct_timestamps": 1, + "detail": "every file entry shares one timestamp - consistent with a reproducible configuration (project.build.outputTimestamp set); this observation does not claim the jar is reproducible"} + ], + "namespace_signal": [ + {"jar": "foo-core-2.1.0.jar", "group_id": "com.github.foo", "under_org_apache": false, + "class_entries": 24, "matching_entries": 24, + "package_roots": ["com/github/foo"], + "detail": "24/24 class entries under the package path 'com/github/foo' derived from groupId 'com.github.foo'; package roots (first three segments): com/github/foo"} + ], + "companion_content": [ + {"jar": "foo-core-2.1.0-sources.jar", "kind": "sources", "signal": "sources-present", + "detail": "31 .java/.scala/.kt source entries; no .class entries"}, + {"jar": "foo-core-2.1.0-javadoc.jar", "kind": "javadoc", "signal": "content-present", + "detail": "58 non-META-INF entries; documentation layout is not judged"} + ] + } +} +``` + +Two things to classify here: + +- The groupId `com.github.foo` is not under `org.apache.*`: the + project entered the ASF with existing Maven coordinates and kept + them for downstream compatibility. That is real policy quoted in + the issue, but deliberately informational **even for ASF + projects** — a published artefact's coordinates cannot be changed + retroactively, so failing the RC would leave the RM no available + remedy. The status stays the tool's status. +- The consistent timestamp signal reads "consistent with a + reproducible configuration" — it must never be reworded into a + claim that the jar is reproducible; only Step 9's rebuild can + assert that. diff --git a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/grading-schema.json b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/grading-schema.json index defd45770..ff4499c21 100644 --- a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/grading-schema.json +++ b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/grading-schema.json @@ -1,3 +1,3 @@ { - "prose_fields": ["paste_recipe", "tool_report", "pom_findings", "companion_findings"] + "prose_fields": ["paste_recipe", "tool_report", "pom_findings", "companion_findings", "observations"] } diff --git a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/output-spec.md b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/output-spec.md index f3c3830bc..c99cfdab1 100644 --- a/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-verify-rc/step-6b-jvm-artefacts/fixtures/output-spec.md @@ -12,6 +12,7 @@ The model must return ONLY valid JSON matching this schema: "tool_report": "", "pom_findings": [""], "companion_findings": [""], + "observations": [""], "paste_recipe": "" } ``` @@ -29,6 +30,19 @@ Grading rules: not be failed. - `pom_findings` / `companion_findings` name the exact failing artefact and what is wrong; an empty list means no findings for that kind. +- `observations` carries the tool's informational observations + (checks 5–7 of issue #1173), one line each naming the jar and what + was observed. They never change `status`: an inconsistent-timestamp + jar, a groupId outside `org.apache.*`, a `-sources.jar` containing + `.class` files and a placeholder companion all leave the blocking + verdict untouched. The wording must not assert reproducibility + either way — "consistent / not consistent with a reproducible + configuration" — and an empty or single-entry jar is + `insufficient-data`, never a pass. A placeholder companion is + reported as the Maven-Central-sanctioned pattern it is, never a + defect. A jar that cannot be opened at all reports `unreadable` + in each affected observation — check 3 never opens a jar, so the + blocking verdict is unaffected. - `paste_recipe` must be a non-empty string invoking `maven-artifact-verify` on the staged directory, with `--digests` set from `jvm_digest_set` when `release-build.md § JVM artefact diff --git a/tools/spec-loop/specs/release-management-lifecycle.md b/tools/spec-loop/specs/release-management-lifecycle.md index 6f0f15abe..bda54e170 100644 --- a/tools/spec-loop/specs/release-management-lifecycle.md +++ b/tools/spec-loop/specs/release-management-lifecycle.md @@ -94,7 +94,9 @@ code lands. runs read-only RC pre-flight (signatures, checksums, RAT headers, NOTICE/LICENSE, prohibited binaries, published-JVM-artefact compliance via `tools/maven-artifact-verify` — POM licence set, - podling disclaimer, companion jars; version consistency, + podling disclaimer, companion jars, plus informational + reproducibility / namespace / companion-content observations; + version consistency, Step 6); `release-vote-draft` (`mode: Drafting`) drafts the `[VOTE]` email body and planning-issue comment after a PASS pre-flight, never sending or