Skip to content

Infra: Add community workflows, compliance automation, and dependency review - #258

Open
sfc-gh-swalia wants to merge 5 commits into
apache:mainfrom
sfc-gh-swalia:swalia-community-infra
Open

sfc-gh-swalia wants to merge 5 commits into
apache:mainfrom
sfc-gh-swalia:swalia-community-infra

Conversation

@sfc-gh-swalia

Copy link
Copy Markdown
Contributor

Summary

Adding community management workflows and compliance automation to bring GitHub infra in line with other ASF projects (modeled on Apache Iceberg's approach).

Compliance:

  • Dependency license scanning on every PR - it blocks ASF Category X licenses (GPL, LGPL, AGPL, SSPL, EUPL) before merge
  • Added Apache RAT integration for automated license header checks (prevents recurrence of the manual cleanup done in Add missing ASF header #242)
  • ASF GitHub Actions allowlist verification ( now catches workflows that silently fail on apache/* repos)
  • PR template updated with NOTICE/LICENSE obligation reminder for vendor code

Community management:

  • Auto-label new issues needs-triage
  • Path-based PR labeling with sync-labels (wildcard catch-all for converters/** + specific per-converter labels)
  • First-time contributor welcome bot with ICLA reminder and Slack link
  • Label state machine: awaiting-response flips to needs-review when author responds
  • Stale issue/PR management (90-day issues, 60-day PRs — only targets items labeled awaiting-response, exempts anything assigned or milestoned)

Governance:

  • PR title format enforcement (Area: Description) for clean squash-merge history
  • Dependabot for GitHub Actions (weekly version updates, grouped)

All workflows include the ASF license header and are guarded with if: github.repository_owner == 'apache' so they do not fire on contributor forks.

Note: The labels referenced by these workflows (needs-triage, awaiting-response, needs-review, stale, converter/*, etc.) do not exist yet — GitHub creates them automatically on first use.

Checklist

Compliance

  • ASF license headers are present on all new source files
  • No third-party dependencies are added without PMC/IPMC approval

@jbonofre
jbonofre self-requested a review July 23, 2026 19:38
@jbonofre

Copy link
Copy Markdown
Member

@sfc-gh-swalia Thanks for that! I will do the review. I saw some changes that are not required now and also to be clarified for release.

Comment thread .github/workflows/asf-allowlist-check.yml Outdated
@jbonofre

Copy link
Copy Markdown
Member

Can you please disclose any AI tool assistance?

@sfc-gh-swalia

Copy link
Copy Markdown
Contributor Author

Can you please disclose any AI tool assistance?

Hey @jbonofre - Yes - I used Claude Opus 4.6 assistance for this PR. Primarily used it for researching ASF compliance patterns across workflow infrastructure, drafting workflow YAML andd validating configurations. I reviewed and validated the code before committing.

I propose we make AI tool disclosure a standard part of our contribution process i.e. a simple optional note in the PR template and a brief section in CONTRIBUTING.md. This keeps us transparent and aligned. Happy to include that in
this PR if people are on board.

@sfc-gh-swalia

Copy link
Copy Markdown
Contributor Author

Can you please disclose any AI tool assistance?

@jbonofre #259 - I added this PR asking contributors to disclose AI usage

@jbonofre

Copy link
Copy Markdown
Member

@sfc-gh-swalia do you mind to rebase and resolve conflict? I think part of this PR is already implemented.

@sfc-gh-swalia

Copy link
Copy Markdown
Contributor Author

@sfc-gh-swalia do you mind to rebase and resolve conflict? I think part of this PR is already implemented.

Hey @jbonofre - Done! ready for your review.

@kayemkim kayemkim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for turning the rebase around within the hour. A header check that runs on every PR is what the dev@ thread was asking for, so I ran this branch through it locally.

dev/check-license does not run as written. RAT 0.16.1 rejects the first option:

ERROR: Unrecognized option: --input-exclude-file

--input-exclude-file, --input-exclude-std and --output-style arrived in RAT 0.17. Setting RAT_VERSION="0.18" is enough; with that one change every option in the script is accepted. No workflow has run on this PR's head, which is why the checks here did not show it.

With 0.18, on the pre-#416 tree the check flags 27 files: the 22 that #416 covered, plus five my grep there missed because it only looked at source extensions. Those five are still on main, so the check would go red on the first PR after this merges:

  • converters/sigma/README.md and converters/sigma/LIMITATIONS.md are the only Markdown files in the repo without the <!-- --> header.
  • converters/microsoft/tests/fixtures/sales_model.bim and converters/microsoft/tests/test.ipynb are JSON, so **/*.bim and **/*.ipynb belong in .rat-excludes next to **/*.json.
  • converters/microsoft/tests/fixtures/sales_model.tmdl: tom.py treats leading /// lines as the description of the object below, so excluding the fixture is safer than a header.

On the other workflows:

  • community-pr-helper.yml: pulls.list has no creator parameter, so the call returns the repo's PRs regardless of author (I checked against the live API), prs.length is always the page size, and the welcome never fires. issues.listForRepo({ creator, state: 'all' }) filtered on pull_request would work. In the message itself, CONTRIBUTING.md now says an ICLA is only required at committer election (#339), and the Slack invite points to a different workspace than README and CONTRIBUTING.
  • pr-title-check.yml: over the last 60 merged PR titles the pattern passes 3. Parentheses are outside the character class, so fix(dbt):, docs(core-spec): and every dependabot chore(deps): title fail. Allowing type(scope): would avoid rejecting most incoming PRs on day one.
  • labeler.yml: converters/ now also has sigma, microsoft, nvidia and ontology; the catch-all converter label still applies, so this is only the per-converter labels.
  • dependency-review.yml: the dependency graph is enabled on the repo, so this will run.

The stale and awaiting-response workflows look fine to me; both only act on a label nothing here applies automatically.

If it helps get the header check in before the release, license-check.yml plus dev/ is independent of the rest and could land on its own. LGTM for that part once the RAT version is bumped and the five files above are handled.

- Bumped RAT version 0.16.1 to 0.17 ( since 0.17 is the minimum with ```--input-exclude-file```
  and ```--output-style;```)
- Added missing entries to .rat-excludes: *.bim, *.ipynb, sales_model.tmdl
- Added ASF license headers to converters/sigma/README.md and LIMITATIONS.md
- Fixed CRLF line endings in ```dev/check-license``` (it was breaking on Linux runners)
  lock in LF via new .gitattributes rule
- Fixed community-pr-helper and replaceed pulls.list (no creator param) with
  search.issuesAndPullRequests; correct ICLA wording per apache#339; fix Slack link
  to apache-ossie workspace (was pointing to ossaboratories)
- Fixed pr-title-check regex to allow type(scope): format; parentheses were
  missing from the character class, rejecting fix(dbt): chore(deps): etc.
- Add converter/sigma, converter/microsoft, converter/nvidia, converter/ontology
  labels to labeler.yml

Tested locally: dev/check-license passes clean (zero flagged files), pr-title
regex validated against last 60 merged PR titles.
@sfc-gh-swalia

Copy link
Copy Markdown
Contributor Author

Thanks for turning the rebase around within the hour. A header check that runs on every PR is what the dev@ thread was asking for, so I ran this branch through it locally.

dev/check-license does not run as written. RAT 0.16.1 rejects the first option:

ERROR: Unrecognized option: --input-exclude-file

--input-exclude-file, --input-exclude-std and --output-style arrived in RAT 0.17. Setting RAT_VERSION="0.18" is enough; with that one change every option in the script is accepted. No workflow has run on this PR's head, which is why the checks here did not show it.

With 0.18, on the pre-#416 tree the check flags 27 files: the 22 that #416 covered, plus five my grep there missed because it only looked at source extensions. Those five are still on main, so the check would go red on the first PR after this merges:

  • converters/sigma/README.md and converters/sigma/LIMITATIONS.md are the only Markdown files in the repo without the <!-- --> header.
  • converters/microsoft/tests/fixtures/sales_model.bim and converters/microsoft/tests/test.ipynb are JSON, so **/*.bim and **/*.ipynb belong in .rat-excludes next to **/*.json.
  • converters/microsoft/tests/fixtures/sales_model.tmdl: tom.py treats leading /// lines as the description of the object below, so excluding the fixture is safer than a header.

On the other workflows:

  • community-pr-helper.yml: pulls.list has no creator parameter, so the call returns the repo's PRs regardless of author (I checked against the live API), prs.length is always the page size, and the welcome never fires. issues.listForRepo({ creator, state: 'all' }) filtered on pull_request would work. In the message itself, CONTRIBUTING.md now says an ICLA is only required at committer election (docs: clarify ICLA is only required at committer election, not for contributions #339), and the Slack invite points to a different workspace than README and CONTRIBUTING.
  • pr-title-check.yml: over the last 60 merged PR titles the pattern passes 3. Parentheses are outside the character class, so fix(dbt):, docs(core-spec): and every dependabot chore(deps): title fail. Allowing type(scope): would avoid rejecting most incoming PRs on day one.
  • labeler.yml: converters/ now also has sigma, microsoft, nvidia and ontology; the catch-all converter label still applies, so this is only the per-converter labels.
  • dependency-review.yml: the dependency graph is enabled on the repo, so this will run.

The stale and awaiting-response workflows look fine to me; both only act on a label nothing here applies automatically.

If it helps get the header check in before the release, license-check.yml plus dev/ is independent of the rest and could land on its own. LGTM for that part once the RAT version is bumped and the five files above are handled.

Hey @kayemkim — thanks for the thorough review.

I addressed all the issues you raised: RAT bumped to 0.17, the five missing files added to .rat-excludes plus ASF headers added to the two sigma Markdown files, the community-pr-helper API fixed to use search.issuesAndPullRequests with total_count, ICLA wording corrected to match what (#339) updated in CONTRIBUTING.md, Slack link updated to the apache-ossie workspace, the PR title regex extended to allow type(scope): format, and the four missing converters added to labeler.yml.

While testing dev/check-license locally I also caught that the script had CRLF line endings, which would have broken it on any Linux runner. Fixed and locked in via .gitattributes.

I couldn't test dependency-review and stale locally as those depend on GitHub's dependency graph infrastructure and real aged issues. I am hoping CI will validate those once this merges.

Let me know what you think.

@kayemkim

Copy link
Copy Markdown
Contributor

Re-ran on 0ad3a37 merged into current main (94a688c). dev/check-license with 0.17 passes clean, exit 0. On the pre-#416 tree the same script still flags the 22 files from #416 plus the two sigma Markdown files, so the new excludes do not hide anything that should carry a header. The revised title pattern passes 29 of the same 60 merged titles; the rest have no prefix at all (Fix ..., Add ...), which is the convention the check is there to introduce. The search query returns 5 for you and 1 for a single-PR author, so the first-time check works now. Everything from my earlier comment is addressed, so LGTM from me.

Two small notes, neither needs a change. The committed dev/check-license at 0561546 already had LF endings (zero CR bytes), so the CRLF was on the checkout side; the .gitattributes guard is fine to keep. And on opened the search index can lag a few seconds behind the new PR, so a second-time contributor might see total_count of 1 and get greeted again. Harmless.

@jbonofre

Copy link
Copy Markdown
Member

Thanks! Let me do a review.

Comment thread .github/workflows/stale.yml Outdated
Comment thread .github/workflows/stale.yml Outdated
Comment thread .github/workflows/auto-label-issues.yml Outdated
Comment thread .github/workflows/auto-label-prs.yml
Comment thread .github/workflows/dependency-review.yml Outdated
Comment thread .github/workflows/dependency-review.yml Outdated
- stale.yml: disabled auto-close for issues and PRs (days-before-*-close: -1)
- auto-label-issues.yml: bootstrap needs-triage label before applying
- auto-label-prs.yml: added issues:write, bootstrap labels from labeler.yml
- dependency-review.yml: added missing ASF Category X SPDX identifiers
- dependency-review.yml: documented fork PR comment limitation

@jbonofre jbonofre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are pretty close! Thanks for the updated!

I have a couple of points to address before merging, then we should be good. Thanks again!

Comment thread .github/workflows/auto-label-prs.yml Outdated
Comment thread .github/workflows/remove-awaiting-response.yml
- remove-awaiting-response.yml: addLabels call with .catch for fork PRs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants