You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a deterministic verifier for the downloaded preset manifest, release tag, README install references, issue metadata, and resulting catalog and documentation files. It reads archive contents without executing them and supports preset-scoped manifests in monorepos.
Defer validation-passed and the draft catalog PR request until the generated files pass consistency and ordering checks. Distinguish confirmed submission mismatches from blocked checks and agent-generated files that need repair.
Regenerate the compiled workflow lock file and add positive and negative regression cases, including a stale --from URL alongside a valid --dev command, archive unavailability, and preserved created_at on updates.
gh aw compile add-community-preset --no-check-update
Live submission workflow run (requires an issue labeled for maintainer automation)
AI Disclosure
I did not use AI assistance for this contribution
I did use AI assistance
AI disclosure: GitHub Copilot App, GPT-6 Sol (runtime-default reasoning effort), autonomously generated and tested the verifier, workflow updates, and regression tests at the contributors request. The contributor requested the commit and PR; no human line-by-line review or live workflow validation is claimed.
Compare published manifests and README release references with issue fields, then verify generated catalog and documentation before success labeling. Regenerate the workflow lock file and cover submission, blocked, and repair outcomes.
Refs github#4746
Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record the submission UTC date and require generated catalog timestamps to match it; keep update creation dates intact. Detect stale README release links whose scoped tag prefix matches the submitted release, including ZIP archive URLs.
Refs github#4746
Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review-round update (commit 6acc33db): the preset verifier now records the submission UTC date and checks generated catalog timestamps against it while preserving existing creation dates on updates. README release checks also recognize the submitted scoped tag prefix, so stale URLs cannot be masked by another accepted install command. Added regressions for both archive URL forms, stale new/update timestamps, and unrelated monorepo releases; the focused suite passed (143 tests), and the workflow lock was regenerated. The two review threads are left open for reviewer verification.
On behalf of @mnriem: GitHub Copilot App (GPT-6 Sol, runtime-default reasoning effort, autonomous) authored these code, workflow, and test changes and this review-round summary. No human line-by-line review or live workflow run is claimed.
Include homepage in the validated snapshot. Step 4 requires this field for new catalog entries, and every current community preset has it, but generated() only compares keys present in expected; consequently a generated entry with a missing or stale homepage still passes. Since the issue form has no separate homepage field, the deterministic value here is the submitted repository URL.
Avoid treating unrelated unscoped monorepo release URLs as stale while retaining checks for matching preset scopes and release assets. Require generated catalog homepage to equal the submitted repository URL for new and updated presets.
Refs github#4746
Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review-round update (commit 0007401e): unscoped monorepo README releases are now compared only when the submitted preset is identifiable by the same release asset (or by an unscoped archive tag from an unscoped submission); unrelated releases no longer fail validation. Generated catalog entries must also include homepage equal to the submitted repository URL, including updates. Added success and failure regressions for both cases, reran the focused suite (150 passed), and regenerated the workflow lock file. The review thread remains open for reviewer verification.
On behalf of @mnriem: GitHub Copilot App (GPT-6 Sol, runtime-default reasoning effort, autonomous) authored the verifier, workflow, and test changes in this round and this summary. No human line-by-line review or live workflow run is claimed.
Use the downloaded archive manifest count to apply bare-tag stale URL checks only when the archive contains one preset. Keep scoped tags and matching release assets checked, and document the monorepo exception with regression coverage.
Refs github#4746
Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review-round update (commit 33815e53): bare archive tags are now treated as belonging to the submitted preset only when the downloaded archive contains a single preset.yml. A multi-preset archive can therefore have another preset鈥檚 unscoped archive URL in its README without being marked stale; scoped tags and matching release assets remain checked. Added before/after regression coverage for the monorepo case and matching/stale single-preset archives, regenerated the workflow lock file, and ran the focused suite (153 passed). The review thread is left open for reviewer verification.
On behalf of @mnriem: GitHub Copilot App (GPT-6 Sol, runtime-default reasoning effort, autonomous) authored this verifier, workflow, and test update and this review-round summary. No human line-by-line review or live workflow run is claimed.
Escaped pipes in a valid human-readable preset name are still treated as table delimiters here. documentation_row() deliberately renders Data | Governance as Data \| Governance, but this split parses the name as Data \, so the generated phase can never pass for that submission. Split on unescaped delimiters and unescape the cell value; please add a regression case with a pipe in the preset name.
data = yaml.safe_load(stream.read().decode("utf-8"))
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
triage-can-waitVerdict: valid and in-scope but deprioritized; held behind the evidence gate
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mnriem
added
the
triage-can-wait
Description
Closes #4746.
validation-passedand the draft catalog PR request until the generated files pass consistency and ordering checks. Distinguish confirmed submission mismatches from blocked checks and agent-generated files that need repair.--fromURL alongside a valid--devcommand, archive unavailability, and preservedcreated_aton updates.Testing
.venv/bin/python -m pytest -q tests/test_community_preset_validation.py tests/test_github_workflows.py --tb=short(137 passed)gh aw compile add-community-preset --no-check-updateAI Disclosure
AI disclosure: GitHub Copilot App, GPT-6 Sol (runtime-default reasoning effort), autonomously generated and tested the verifier, workflow updates, and regression tests at the contributors request. The contributor requested the commit and PR; no human line-by-line review or live workflow validation is claimed.