Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ github-issue: 2185
spec-path: docs/issues/open/2185-2003-triage-advisory-external-link-check-findings/ISSUE.md
branch: "2185-2003-triage-advisory-external-link-check-findings"
related-pr: null
last-updated-utc: 2026-09-10 07:25
last-updated-utc: 2026-09-11 08:27
semantic-links:
skill-links:
- create-issue
Expand All @@ -17,6 +17,7 @@ semantic-links:
- docs/issues/open/2003-overhaul-guardrails-and-automation/EPIC.md
- docs/issues/open/2162-enforce-lychee-and-schedule-external-link-checks/ISSUE.md
- docs/issues/open/2185-2003-triage-advisory-external-link-check-findings/external-link-baseline.md
- docs/issues/open/2185-2003-triage-advisory-external-link-check-findings/agent-review-reports.md
- .github/workflows/external-link-check.yaml
- .github/lychee-online.toml
---
Expand Down Expand Up @@ -87,8 +88,8 @@ Status values: `TODO`, `IN_PROGRESS`, `BLOCKED`, `DONE`.
| --- | ----------- | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| T1 | DONE | Preserve and classify the baseline | `external-link-baseline.md` maps all 461 report errors to nine recurring categories and dispositions. |
| T2 | TODO | Repair clearly stale references | Small, reviewable repairs replace or remove only references confirmed stale, with replacement-target evidence. |
| T3 | IN_PROGRESS | Add justified narrow exclusions | Added the exact C1/C9 pull-request review-comment-anchor candidate; hosted boundary verification is pending. |
| T4 | TODO | Revalidate hosted signal | A manually dispatched run after each remediation slice demonstrates the expected change without hiding remaining failures. |
| T3 | DONE | Add justified narrow exclusions | The exact C1/C9 review-comment-anchor rule was verified on a merged upstream hosted run. |
| T4 | DONE | Revalidate hosted signal | Hosted run 34578523069 excluded the intended anchors while retaining visible unrelated failures and its report artifact. |
| T5 | TODO | Document operations and review completion | Triage procedure, residual risks, acceptance evidence, and independent review are updated from observed results. |

## Commit Points
Expand All @@ -109,15 +110,15 @@ A category that needs no repository change is recorded in issue-local evidence w
- [x] Folder-style spec drafted in `docs/issues/drafts/2003-triage-advisory-external-link-check-findings/ISSUE.md`
- [x] Spec reviewed and approved by user/maintainer
- [x] GitHub issue created and issue number added to this spec
- [ ] (Optional, recommended for complex issues) Spec-only PR merged into `develop` before implementation
- [x] (Optional, recommended for complex issues) Spec-only PR merged into `develop` before implementation
- [x] Baseline classification independently reviewed
- [ ] Implementation completed
- [ ] Automatic verification completed (`linter all`, relevant tests, and any pre-push checks)
- [ ] Manual verification scenarios executed and recorded (status + evidence)
- [ ] Acceptance criteria reviewed after implementation and updated with evidence
- [ ] Evidence-based implementation completion review recorded: issue-local retrospective created for material discoveries, or progress log states why none was needed
- [ ] Reviewer validated acceptance criteria and updated checkboxes
- [ ] Independent reviewer reports recorded in issue-local `agent-review-reports.md` when reviewers received this folder-style specification
- [x] Independent reviewer reports recorded in issue-local `agent-review-reports.md` when reviewers received this folder-style specification
- [ ] Committer verified spec progress is up to date before commit
- [ ] Issue closed and spec moved from `docs/issues/open/` to `docs/issues/closed/`

Expand All @@ -128,14 +129,16 @@ A category that needs no repository change is recorded in issue-local evidence w
- 2026-09-09 15:15 UTC - Copilot - Downloaded the retained report from [run 34347690674](https://github.com/torrust/torrust-tracker/actions/runs/34347690674), classified all 461 errors in `external-link-baseline.md`, and selected only GitHub pull-request review-comment anchors as the first proposed remediation slice. No production configuration or link was changed.
- 2026-09-10 07:09 UTC - Task Reviewer - Independently re-parsed the retained report and passed the corrected classification: $416+7+14+3+1+5+3+4+8=461$, with C1/C9 covering 424 exact GitHub pull-request review-comment-anchor failures. No broad exclusion is proposed; no configuration or link changed in this evidence-only slice.
- 2026-09-10 07:25 UTC - Copilot - Added the exact C1/C9 URL-pattern exclusion to `.github/lychee-online.toml`. A two-link Lychee boundary test excluded a pull-request review-comment anchor while retaining a GitHub issue-comment anchor as a visible error. Hosted rerun evidence remains pending.
- 2026-09-11 08:27 UTC - Copilot - After PR #2197 merged, [run 34577938048](https://github.com/torrust/torrust-tracker/actions/runs/34577938048) was cancelled while its Lychee step was still in progress. Its upload step retained an empty, unusable report artifact, so it provides no usable Lychee output. Replacement [run 34578523069](https://github.com/torrust/torrust-tracker/actions/runs/34578523069) completed on merged revision `427b0c93`: it visibly failed with 52 remaining errors, excluded 476 links, uploaded a 1,743-byte report artifact, and contained no matching C1/C9 `#discussion_r` URLs. Unrelated `404`, `403`, loopback, issue-comment, other-fragment, and rate-limit failures remained visible.
- 2026-09-11 08:32 UTC - Task Reviewer - Independently reviewed the replacement hosted run and report artifact. The evidence supports T3/T4, AC3-AC5, and M4; the issue remains open for the remaining C2-C8 work. See `agent-review-reports.md`.

## Acceptance Criteria

- [x] AC1: An issue-local baseline records the exact hosted run, revision, summary counts, and a disposition for every distinct failing URL or recurring failure pattern.
- [ ] AC2: Each repair changes only a verified stale reference and records why its replacement target is correct.
- [ ] AC3: Each added exclusion is online-only, narrowly scoped to a documented durable false-positive category, and does not suppress unrelated external-link failures.
- [ ] AC4: The advisory workflow remains scheduled/manual, visibly fails for remaining external-link failures, and continues to upload its Markdown report on failure.
- [ ] AC5: At least one hosted rerun after each remediation slice records the resulting counts and explains material differences from the prior run.
- [x] AC3: Each added exclusion is online-only, narrowly scoped to a documented durable false-positive category, and does not suppress unrelated external-link failures.
- [x] AC4: The advisory workflow remains scheduled/manual, visibly fails for remaining external-link failures, and continues to upload its Markdown report on failure.
- [x] AC5: At least one hosted rerun after each remediation slice records the resulting counts and explains material differences from the prior run.
- [ ] AC6: Documentation explains any permanent exclusion rationale and preserves the existing rerun-then-repair-or-narrow-exclusion triage policy.
- [ ] `linter all` exits with code `0`.
- [ ] Relevant tests pass or their documented non-applicability is reviewed.
Expand All @@ -161,7 +164,7 @@ Status values: `TODO`, `IN_PROGRESS`, `DONE`, `FAILED`, `BLOCKED`.
| M1 | Reproduce baseline | Manually dispatch `external-link-check.yaml` on `develop`; download `lychee-external-link-report`. | Report is available even when Lychee fails; baseline counts and categories can be reviewed. | DONE | [Run 34347690674](https://github.com/torrust/torrust-tracker/actions/runs/34347690674), revision `7abc30b2`, 461 errors, 0 timeouts. |
| M2 | Classify durable failures | Review all report entries and group by URL/pattern, response type, owning document, and proposed disposition. | Every baseline failure has a traceable disposition; no broad host-level suppression is proposed. | DONE | `external-link-baseline.md`; independent reconciliation passed on 2026-09-10. |
| M3 | Verify reference repairs | Check each changed target using the appropriate authoritative source, then run local validation. | Replacement reference is correct and offline local-link validation remains clean. | TODO | Focused commands and review evidence. |
| M4 | Verify exclusion boundaries | Dispatch the hosted workflow after adding a proposed exclusion. | The intended durable false-positive category is absent, while representative unrelated external failures remain visible and the report uploads. | TODO | Hosted run URL and artifact summary. |
| M4 | Verify exclusion boundaries | Dispatch the hosted workflow after adding a proposed exclusion. | The intended durable false-positive category is absent, while representative unrelated external failures remain visible and the report uploads. | DONE | [Run 34578523069](https://github.com/torrust/torrust-tracker/actions/runs/34578523069) on `427b0c93`: 476 excluded, 52 remaining errors, no C1/C9 matches, and a retained 1,743-byte report artifact. |
| M5 | Distinguish transient failures | Re-run a newly observed timeout, 403, or other potentially transient result once. | The record distinguishes a persistent failure from a transient response before an exclusion or repair decision. | TODO | Pair of hosted-run URLs and comparison. |

### Acceptance Verification
Expand All @@ -170,9 +173,9 @@ Status values: `TODO`, `IN_PROGRESS`, `DONE`, `FAILED`, `BLOCKED`.
| ----- | ---------------------- | ----------------------------------------------------------------- |
| AC1 | DONE | `external-link-baseline.md` from run 34347690674. |
| AC2 | TODO | Reviewed reference-repair commits and target evidence. |
| AC3 | TODO | Configuration diff and hosted rerun showing exclusion boundaries. |
| AC4 | TODO | Workflow source and hosted failure/report artifact. |
| AC5 | TODO | Hosted rerun comparison for each remediation slice. |
| AC3 | DONE | Exact online-only rule and run 34578523069 report evidence. |
| AC4 | DONE | Run 34578523069 failed visibly and uploaded its report artifact. |
| AC5 | DONE | Run 34578523069 records 476 exclusions and 52 remaining errors. |
| AC6 | TODO | Updated documentation and reviewer confirmation. |

## Risks and Trade-offs
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
semantic-links:
related-artifacts:
- docs/issues/open/2185-2003-triage-advisory-external-link-check-findings/ISSUE.md
- docs/issues/open/2185-2003-triage-advisory-external-link-check-findings/external-link-baseline.md
- .github/lychee-online.toml
- .github/workflows/external-link-check.yaml
---

# Agent Review Reports - Issue #2185

> Append one completed independent-review entry at a time. Do not modify, reorder, or remove
> earlier entries. A correction is a new entry that names the earlier conclusion.

## Reports

### 2026-09-11 08:32 UTC - Task Reviewer

- Invocation scope: Read-only review of the hosted C1/C9 verification evidence in `ISSUE.md` and `external-link-baseline.md`.
- Inputs: GitHub Actions runs [34577938048](https://github.com/torrust/torrust-tracker/actions/runs/34577938048) and [34578523069](https://github.com/torrust/torrust-tracker/actions/runs/34578523069), the downloaded replacement artifact, `.github/lychee-online.toml`, and `.github/workflows/external-link-check.yaml`.
- Evidence: Replacement run 34578523069 completed with a visible Lychee failure on `427b0c93b01f1264f7ef824097ac5669d899486c`; its `Upload Lychee Report` step succeeded. The unexpired 1,743-byte `lychee-external-link-report` artifact expires on 2026-09-25 and records 1,816 total checks, 1,288 successes, 26 redirects, 476 exclusions, 52 errors, and no timeouts. It contains no exact `#discussion_r` pull-request review-comment URLs, while unrelated `404`, `403`, loopback, GitHub issue-comment/pull-request-review fragments, other missing fragments, and a `429` remain visible.
- Findings:
- Resolved: The exact online-only C1/C9 exclusion is sufficiently narrow and its hosted boundary behavior is verified.
- Correction required: Cancelled run 34577938048 retained an empty, unusable report artifact; it did not provide usable Lychee output but should not be described as having no artifact.
- Pending: Keep issue #2185 open. T5, AC2, AC6, remaining quality/completion checkpoints, and C2-C8 triage remain incomplete.
- Verdict: REVIEW PASSED after the cancelled-run artifact wording is corrected.
- Follow-up actions:
- Record the cancelled-run correction in `ISSUE.md`.
- Commit and review the hosted-verification evidence without closing issue #2185.
- Continue with the next independently reviewable category after the evidence is merged.
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,14 @@ The 14 affected package README files are:

C1 and C9 are the first remediation boundary: the exact GitHub pull-request review-comment URL pattern is responsible for 424 of 461 reported errors. The online-only configuration excludes exactly that pattern. A two-link local boundary test excluded a matching pull-request review-comment anchor while retaining a non-matching GitHub issue-comment anchor as a visible error. A hosted workflow rerun must still retain unrelated `404`, `403`, local-example, and third-party diagnostics.

## Hosted Verification

[External Link Check run 34578523069](https://github.com/torrust/torrust-tracker/actions/runs/34578523069) ran the merged configuration on revision `427b0c93b01f1264f7ef824097ac5669d899486c`. The `Check External Links` step failed visibly and `Upload Lychee Report` succeeded; the retained `lychee-external-link-report` artifact is 1,743 bytes and expires on 2026-09-25.

The report recorded 1,816 total checks, 1,288 successful checks, 26 redirects, 476 excluded links, 52 errors, and no timeouts. No URL matching the exact `https://github.com/torrust/torrust-tracker/pull/<number>#discussion_r<id>` pattern remained in the report. The excluded count is greater than the 424 baseline occurrences because the merged `develop` revision contained additional matching review-comment links.

The report still contained visible `404`, `403`, local loopback, GitHub issue-comment and pull-request-review fragments, third-party missing-fragment, and rate-limited diagnostics. This proves the C1/C9 rule suppresses the intended dynamic review-comment-anchor pattern without hiding unrelated categories. The remaining 52 errors are the next triage input; their exact count is not directly comparable to the 461-error baseline because the checked document set changed between revisions.
Comment on lines +142 to +146

## Deferred Work

C3-C9 are intentionally deferred until C1 is independently reviewed and its exclusion boundary is validated. This prevents the first configuration change from mixing clearly uncheckable dynamic anchors with potentially stale or transient external URLs.