Skip to content

feat(dev-tools): add /qa skill — recorded video verification for GitHub issues#36535

Draft
hassandotcms wants to merge 1 commit into
mainfrom
qa-skill-video-verification
Draft

feat(dev-tools): add /qa skill — recorded video verification for GitHub issues#36535
hassandotcms wants to merge 1 commit into
mainfrom
qa-skill-video-verification

Conversation

@hassandotcms

@hassandotcms hassandotcms commented Jul 11, 2026

Copy link
Copy Markdown
Member

What

A Claude Code skill: /qa <issue-number | PR-number | url>

It QAs a merged fix end-to-end against a local dotCMS instance — the way a human QA engineer would — and delivers a short screen-recording as proof of verification, plus a terse QA brief and a draft comment you can post to the issue after reviewing both.

Why

  • Verifying a fix by reading the diff (or trusting the PR checklist) misses what actually happens in the running product.
  • Manual QA proof today is ad-hoc screenshots; a narrated single-take video showing each acceptance criterion pass in the real UI is easier to trust and to review.
  • Encoding the process as a shared skill makes QA runs repeatable and consistent across the team instead of living in one person's head.

How to use

/qa 36369        # issue number
/qa 36371        # PR number — resolves the linked issue
/qa <github-url> # either, as a URL

The skill verifies the fix is actually in the running build, runs the PR's own tests, drives the real UI on camera against the acceptance criteria, and hands you the video + brief. It never posts to GitHub on its own — you review the video and the draft comment first.

Sample QA Video Recordings of GH issues

qa-35908-block-editor-popovers.mp4
qa-36369-content-drive-upload-button-final.mp4

Adds a Claude Code skill that QAs a GitHub issue or PR end-to-end against
a local dotCMS instance and produces a single-take video as proof:

- resolves issue -> merged PR -> fix commit, and verifies the served
  bundle was built from a tree containing the fix before testing
- runs the PR's own tests first, then drives the real UI like a human
  QA engineer: narrated login, menu navigation, visible cursor, live
  acceptance-criteria checklist that flips only after each assertion
- integrity rules: one unedited take, pointer-transparent overlays,
  visually-true assertions (visibleArea vs offsetParent), declared
  seed data only, zero environment footprint (Basic auth, no lingering
  API tokens)
- reusable across dev machines: no hardcoded paths, container names
  discovered at runtime, fixed-port env via just dev-run-fixed
@github-actions

Copy link
Copy Markdown
Contributor

❌ Issue Linking Required

This PR could not be linked to an issue. All PRs must be linked to an issue for tracking purposes.

How to fix this:

Option 1: Add keyword to PR body (Recommended - auto-removes this comment)
Edit this PR description and add one of these lines:

  • This PR fixes #123 or Fixes: #123

  • This PR closes #123 or Closes: #123

  • This PR resolves #123 or Resolves: #123

  • Other supported keywords: fix, fixed, close, closed, resolve, resolved
    Option 2: Link via GitHub UI (Note: won't clear the failed check)

  1. Go to the PR → Development section (right sidebar)

  2. Click "Link issue" and select an existing issue

  3. Push a new commit or re-run the workflow to clear the failed check
    Option 3: Use branch naming
    Create a new branch with one of these patterns:

  • 123-feature-description (number at start)

  • issue-123-feature-description (issue-number at start)

  • feature-issue-123 (issue-number anywhere)

Why is this required?

Issue linking ensures proper tracking, documentation, and helps maintain project history. It connects your code changes to the problem they solve.---

This comment was automatically generated by the issue linking workflow

@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Claude finished @hassandotcms's task in 3m 23s —— View job


PR Review — /qa skill

Scope check done. This PR is entirely agent tooling: 5 markdown docs + one Playwright helper module (.claude/skills/qa/scripts/qa-helpers.mjs). No product Java/Angular runtime code, so the usual dotCMS convention/security/DB checks don't apply here. I focused on whether the helper actually works against the real dotAdmin DOM and whether the codified rules are internally consistent.

  • Read all 6 changed files
  • Verified selectors/recipes against the live codebase
  • Reviewed helper logic for bugs

Verified against the codebase (all correct)

  • just dev-run-fixed / just dev-wait-ready — exist (justfile:103,110)
  • Nav selectors — nav.collapsed (dot-navigation.component.html:3), [data-testid="nav-item-main"] / nav-item-label (dot-nav-item.component.html:13,18), a.dot-nav-sub__link (dot-sub-nav.component.html:19), [data-testid="dot-nav-header-toggle-button"] (dot-nav-header.component.html:11) all match.
  • Login selectors — the HTML declares data-testId (capital I) at dot-login.component.html:37,57,96, while the helper queries [data-testid=...] (lowercase). This is not a bug: HTML attribute names are lowercased at parse time and CSS attribute selectors match them ASCII-case-insensitively, so querySelector('[data-testid="userNameInput"]') resolves correctly. (Consistent with the author's [BUG] Content Drive: hide Upload button when items are selected #36369 validation run.)

New Issues

No blocking issues found.

Notes (🟡 non-blocking)

  • 🟡 scripts/qa-helpers.mjs:157dismissToasts() is exported but unused anywhere in the skill flow (everything uses suppressToasts()). Either wire it in for the pre-navigation case or drop it to avoid a dead helper drifting out of sync.
  • 🟡 scripts/qa-helpers.mjs:187 (suppressToasts) — the MutationObserver is stored on window.__qaToastKiller but never disconnect()ed. Harmless in practice since the context is torn down in finalize(), but worth a one-line note that it's intentionally left running for the session lifetime.
  • 🟡 SKILL.md STEP 1 — "Try as issue first; if it's actually a PR, gh errors." gh issue view <PR#> does error on a pure PR number, but if an issue and PR happen to share a number namespace edge case this is fine as written; just flagging that the fallback relies on gh's error, so a network/auth failure would be misread as "it's a PR." Minor robustness point, not a correctness bug.
  • 🟡 Consistency: the PR body notes the branch intentionally deviates from the issue-N- convention because no issue is linked yet, which is why the issue-linking check is failing. That's a process gate, not a code issue — retro-linking an issue (or adding Fixes #N) will clear it.

Strengths worth keeping

  • The integrity model is genuinely well thought out: visible() vs visibleArea() (attached-but-empty must fail), pointer-events: none on every overlay after the banner-swallowed-a-click incident, comma-selector double-match guard, and the paint-driven recorder truncation workaround in finalize() are all real, hard-won failure modes correctly codified.
  • Provenance chain (merge-base --is-ancestor + container build time + bundle mtime) before trusting the merge badge is the right instinct.
  • Zero-footprint auth (Basic auth, no lingering tokens) directly avoids the self-inflicted toast-pollution problem — good closed loop.

Overall: clean, self-contained tooling addition. No changes required from a code-correctness standpoint; the notes above are polish. The only actionable item outside the code is the failing issue-link check.

· qa-skill-video-verification

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant