From ee28fda49e79e524c2410f6ab5648281c4ea35db Mon Sep 17 00:00:00 2001 From: Saurabh Pandit Date: Wed, 12 Aug 2026 15:24:00 +0530 Subject: [PATCH] (MODULES-11721) Add Puppet 9 support Widens the puppet requirement in metadata.json to >= 8.0.0 < 10.0.0. The module's CI tooling can't resolve or lint under Puppet 9 as-is, so most of this change is dependency and workflow plumbing to make the Puppet 9 lane actually run: voxpupuli-puppet-lint-plugins bumped to ~> 7.0 (puppet-lint 5.x), puppetlabs_spec_helper and puppet_litmus temporarily pinned to git main (their released versions don't yet carry the puppet-lint relaxation and --collection-platform-exclude respectively), and the Gemfile resolves Puppet 9 (8.99.x) prereleases from PUPPET_GEM_SOURCE with a puppetcore fallback and warning when that secret isn't configured. ci.yml, nightly.yml and mend.yml gain ruby_version: "3.2" (voxpupuli puppet-lint-plugins 7.0 requires ruby >= 3.2). The Acceptance flags in ci.yml/nightly.yml gain --collection-platform-exclude for 9:redhat-7, 9:centos-7, 9:oraclelinux-7, 9:scientific-7 (all el7, no Puppet 9 agent), 9:debian-10, 9:ubuntu-18.04 and 9:ubuntu-20.04 -- confirmed against this module's own metadata.json and matrix_from_metadata_v3 output, not copied from another module. ci.yml/nightly.yml are marked unmanaged in .sync.yml since pdk-templates can't express ruby_version or the collection excludes. On the Ruby 4 / Puppet 9 lane, puppet_litmus pulls in bolt 4.x, which depends on faraday-patron -> patron; patron builds a libcurl native extension and the CI runner has no libcurl headers, so the Spec job in ci.yml/nightly.yml now sets additional_packages: "libcurl4-openssl-dev" to install them before bundle install (same fix as puppetlabs-lvm#391). No manifest/type/provider/function/spec behaviour changes. Follow-up: the two temporary git-branch pins from the paragraph above have since been superseded by released gems -- puppetlabs_spec_helper 9.0.0 (relaxes puppet-lint to ~> 5.x, matching voxpupuli-puppet-lint-plugins ~> 7.0) and puppet_litmus 2.8.0 (adds --collection-platform-exclude to matrix_from_metadata_v3). Both Gemfile entries are now plain, released version constraints, matching the pattern puppetlabs-lvm#391 landed with. puppetlabs_spec_helper 9.0.0 also renamed its puppet-syntax dependency to puppetlabs-syntax, so the Rakefile's require was updated to match -- without it, nothing in the bundle provides the old puppet-syntax path and the Rakefile raises LoadError. Opus review pass: tightened the Gemfile's puppet_litmus constraint from '~> 2.5' to '~> 2.8', since --collection-platform-exclude (which ci.yml/nightly.yml pass unconditionally) only exists from 2.8.0 onward and older 2.x releases hard-fail with OptionParser::InvalidOption instead of degrading gracefully. Added a Gemfile: overrides: block in .sync.yml pinning puppetlabs_spec_helper to '~> 9.0' and puppet_litmus to '~> 2.8', so a scheduled `pdk update` can't silently widen back to pdk-templates' own looser defaults ('>= 8.0' / '~> 2.5') and reintroduce the puppet-lint 4.0/collection-exclude problems this PR fixes. Also corrected the existing .sync.yml comment on ci.yml/nightly.yml's unmanaged: true: it previously implied --collection-platform-exclude/acceptance_flags couldn't be expressed via the templates, but acceptance_flags is a real, supported .sync.yml key (this file already uses it below) -- the only genuinely inexpressible inputs are ruby_version and additional_packages, which are now the sole stated reason. Follow-up (from puppetlabs-windows_eventlog#100, MODULES-11729): dropped the PUPPET_GEM_SOURCE-based Puppet 9 prerelease resolution. Confirmed via windows_eventlog's own CI run (job 97686128656) that puppet 9.0.0 is now a real, final release on the standard puppetcore source (rubygems-puppetcore.puppet.com, reached via PUPPET_FORGE_TOKEN) with no PUPPET_GEM_SOURCE/Twingate source needed -- that CI run resolved `puppet (9.0.0)` and `Bundle complete!` with PUPPET_GEM_SOURCE unset. This module's own PUPPET_GEM_SOURCE-based path was still resolving the older `8.99.0.113.gef6e57f` internal prerelease build instead of the real release. An Opus review of an initial version of this fix (which called `location_for` with a third `source:` opts argument, matching windows_eventlog and puppetlabs- vcsrepo's Gemfile) caught that this module's `location_for` is a 2-arg variant that doesn't accept or merge a source option -- that call would have raised `Bundler::Dsl::DSLError: wrong number of arguments` on every Puppet 9 CI job, failing before gem resolution even starts. Used the literal array form instead (this module has no `gemsource_puppetcore` variable, so the literal source URL string is used, matching the style already used in the PUPPET_FORGE_TOKEN-gated branch immediately below). The Puppet-9-must-be-checked-first ordering guard against that branch's hardcoded `puppet ~> 8.11` is preserved. Re-verified with a direct Bundler::Dsl.evaluate under PUPPET_GEM_VERSION="~> 9.0" (no ArgumentError this time) and a full local run: rake lint clean, rake spec 62 examples, 0 failures. Follow-up (review comment https://github.com/puppetlabs/puppetlabs-java_ks/pull/475#discussion_r4014005178): removed the puppet_version-conditional branching entirely, matching puppetlabs-registry#320's pattern. Added the 3-arg `opts`-merging location_for and the gemsource_default/gemsource_puppetcore split that registry's Gemfile uses -- this module's location_for genuinely was a 2-arg variant (the reason for the literal array form above), so removing the conditional required bringing in the helper that makes the unconditional call safe, not just deleting the branches. gems['puppet']/gems['facter'] now resolve unconditionally via location_for(..., { source: gemsource_puppetcore }), same as registry; gemsource_puppetcore already falls back to the default public source when PUPPET_FORGE_TOKEN is unset, so the old PUPPET_FORGE_TOKEN-gated ~> 8.11 override and the plain else branch were both redundant with that fallback and removed. Verified locally (ruby 3.2.8): PUPPET_GEM_VERSION="~> 8.0" resolves puppet 8.10.0 from rubygems.org; PUPPET_GEM_VERSION="~> 9.0" with a placeholder PUPPET_FORGE_TOKEN correctly targets rubygems-puppetcore.puppet.com (fails only on auth, as expected without a real token -- no Bundler::Dsl::DSLError this time). rake syntax lint metadata_lint rubocop clean; rake parallel_spec 62 examples, 0 failures, same count as before. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/ci.yml | 12 ++++++++- .github/workflows/mend.yml | 4 +++ .github/workflows/nightly.yml | 10 ++++++- .sync.yml | 42 ++++++++++++++++++++++++++++-- Gemfile | 49 ++++++++++++++++++++--------------- Rakefile | 2 +- metadata.json | 2 +- 7 files changed, 94 insertions(+), 27 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f47b156d..2bece581 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -9,6 +9,13 @@ on: jobs: Spec: uses: "puppetlabs/cat-github-actions/.github/workflows/module_ci.yml@main" + with: + # voxpupuli-puppet-lint-plugins 7.0 (needed for Puppet 9 support) requires + # ruby >= 3.2; the default (3.1) can no longer resolve the :development group. + ruby_version: "3.2" + # puppet_litmus -> bolt 4.x -> faraday-patron -> patron builds a libcurl native extension; + # the runner has no libcurl headers, so install them before bundle (Puppet 9 lane, Ruby 4). + additional_packages: "libcurl4-openssl-dev" secrets: "inherit" Acceptance: @@ -16,4 +23,7 @@ jobs: uses: "puppetlabs/cat-github-actions/.github/workflows/module_acceptance.yml@main" secrets: "inherit" with: - flags: "--nightly" + flags: "--nightly --collection-platform-exclude 9:redhat-7 --collection-platform-exclude 9:centos-7 --collection-platform-exclude 9:oraclelinux-7 --collection-platform-exclude 9:scientific-7 --collection-platform-exclude 9:debian-10 --collection-platform-exclude 9:ubuntu-18.04 --collection-platform-exclude 9:ubuntu-20.04" + # voxpupuli-puppet-lint-plugins 7.0 (needed for Puppet 9 support) requires + # ruby >= 3.2; the default (3.1) can no longer resolve the :development group. + ruby_version: "3.2" diff --git a/.github/workflows/mend.yml b/.github/workflows/mend.yml index b4100a5a..ad8040f6 100644 --- a/.github/workflows/mend.yml +++ b/.github/workflows/mend.yml @@ -12,4 +12,8 @@ jobs: mend: uses: "puppetlabs/cat-github-actions/.github/workflows/mend_ruby.yml@main" + with: + # voxpupuli-puppet-lint-plugins 7.0 (needed for Puppet 9 support) requires + # ruby >= 3.2; `bundle lock` resolves all groups at the default ruby (3.1). + ruby_version: "3.2" secrets: "inherit" diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 16b2be38..e3bdb3cd 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -8,6 +8,13 @@ on: jobs: Spec: uses: "puppetlabs/cat-github-actions/.github/workflows/module_ci.yml@main" + with: + # voxpupuli-puppet-lint-plugins 7.0 (needed for Puppet 9 support) requires + # ruby >= 3.2; the default (3.1) can no longer resolve the :development group. + ruby_version: "3.2" + # puppet_litmus -> bolt 4.x -> faraday-patron -> patron builds a libcurl native extension; + # the runner has no libcurl headers, so install them before bundle (Puppet 9 lane, Ruby 4). + additional_packages: "libcurl4-openssl-dev" secrets: "inherit" Acceptance: @@ -15,4 +22,5 @@ jobs: uses: "puppetlabs/cat-github-actions/.github/workflows/module_acceptance.yml@main" secrets: "inherit" with: - flags: "--nightly" + flags: "--nightly --collection-platform-exclude 9:redhat-7 --collection-platform-exclude 9:centos-7 --collection-platform-exclude 9:oraclelinux-7 --collection-platform-exclude 9:scientific-7 --collection-platform-exclude 9:debian-10 --collection-platform-exclude 9:ubuntu-18.04 --collection-platform-exclude 9:ubuntu-20.04" + ruby_version: "3.2" diff --git a/.sync.yml b/.sync.yml index 4c065923..8df7546a 100644 --- a/.sync.yml +++ b/.sync.yml @@ -8,6 +8,17 @@ Gemfile: optional: ":development": - gem: ruby-pwsh + # MODULES-11721: pin puppetlabs_spec_helper and puppet_litmus above pdk-templates' + # own defaults (>= 8.0 and ~> 2.5 respectively), which are loose enough for a + # scheduled `pdk update` to silently revert this PR: puppetlabs_spec_helper 8.0.0 + # still pins puppet-lint ~> 4.0 (conflicts with voxpupuli-puppet-lint-plugins ~> 7.0, + # needed for Puppet 9), and puppet_litmus below 2.8.0 doesn't support + # --collection-platform-exclude, which ci.yml/nightly.yml pass unconditionally. + overrides: + - gem: 'puppetlabs_spec_helper' + version: '~> 9.0' + - gem: 'puppet_litmus' + version: '~> 2.8' spec/spec_helper.rb: mock_with: ":rspec" coverage_report: true @@ -17,10 +28,37 @@ spec/spec_helper.rb: unmanaged: false .github/workflows/auto_release.yml: unmanaged: false +# MODULES-11721: ci.yml and nightly.yml are maintained by hand because they carry +# Puppet 9 customisations that pdk-templates cannot express -- the `ruby_version` +# and `additional_packages` inputs (no such keys in the templates). Leaving them +# managed means the scheduled `pdk update` PR silently reverts Puppet 9 support. +# acceptance_flags below is kept in step with the hand-written `flags:` -- that part +# IS expressible via the template's own acceptance_flags key, so it isn't itself a +# reason for unmanaged: true; these files stay unmanaged solely for ruby_version and +# additional_packages. They can go back to template management once pdk-templates +# supports those two inputs. .github/workflows/ci.yml: - unmanaged: false + unmanaged: true + acceptance_flags: + - '--nightly' + - '--collection-platform-exclude 9:redhat-7' + - '--collection-platform-exclude 9:centos-7' + - '--collection-platform-exclude 9:oraclelinux-7' + - '--collection-platform-exclude 9:scientific-7' + - '--collection-platform-exclude 9:debian-10' + - '--collection-platform-exclude 9:ubuntu-18.04' + - '--collection-platform-exclude 9:ubuntu-20.04' .github/workflows/nightly.yml: - unmanaged: false + unmanaged: true + acceptance_flags: + - '--nightly' + - '--collection-platform-exclude 9:redhat-7' + - '--collection-platform-exclude 9:centos-7' + - '--collection-platform-exclude 9:oraclelinux-7' + - '--collection-platform-exclude 9:scientific-7' + - '--collection-platform-exclude 9:debian-10' + - '--collection-platform-exclude 9:ubuntu-18.04' + - '--collection-platform-exclude 9:ubuntu-20.04' .github/workflows/release.yml: unmanaged: false .travis.yml: diff --git a/Gemfile b/Gemfile index e887977d..ce90d28b 100644 --- a/Gemfile +++ b/Gemfile @@ -1,6 +1,13 @@ -source ENV['GEM_SOURCE'] || 'https://rubygems.org' +# For puppetcore, set GEM_SOURCE_PUPPETCORE = 'https://rubygems-puppetcore.puppet.com' +gemsource_default = ENV['GEM_SOURCE'] || 'https://rubygems.org' +gemsource_puppetcore = if ENV['PUPPET_FORGE_TOKEN'] + 'https://rubygems-puppetcore.puppet.com' +else + ENV['GEM_SOURCE_PUPPETCORE'] || gemsource_default +end +source gemsource_default -def location_for(place_or_version, fake_version = nil) +def location_for(place_or_version, fake_version = nil, opts = {}) git_url_regex = %r{\A(?(https?|git)[:@][^#]*)(#(?.*))?} file_url_regex = %r{\Afile:\/\/(?.*)} @@ -9,7 +16,7 @@ def location_for(place_or_version, fake_version = nil) elsif place_or_version && (file_url = place_or_version.match(file_url_regex)) ['>= 0', { path: File.expand_path(file_url[:path]), require: false }] else - [place_or_version, { require: false }] + [place_or_version, { require: false }.merge(opts)] end end @@ -18,7 +25,7 @@ group :development do gem "json", '= 2.6.3', require: false if Gem::Requirement.create(['>= 3.2.0', '< 4.0.0']).satisfied_by?(Gem::Version.new(RUBY_VERSION.dup)) gem "racc", '~> 1.4.0', require: false if Gem::Requirement.create(['>= 2.7.0', '< 3.0.0']).satisfied_by?(Gem::Version.new(RUBY_VERSION.dup)) gem "deep_merge", '~> 1.2.2', require: false - gem "voxpupuli-puppet-lint-plugins", '~> 5.0', require: false + gem "voxpupuli-puppet-lint-plugins", '~> 7.0', require: false gem "facterdb", '~> 2.1', require: false if Gem::Requirement.create(['< 3.0.0']).satisfied_by?(Gem::Version.new(RUBY_VERSION.dup)) gem "facterdb", '~> 3.0', require: false if Gem::Requirement.create(['>= 3.0.0']).satisfied_by?(Gem::Version.new(RUBY_VERSION.dup)) gem "metadata-json-lint", '~> 4.0', require: false @@ -39,12 +46,19 @@ group :development do end group :development, :release_prep do gem "puppet-strings", '~> 4.0', require: false - gem "puppetlabs_spec_helper", '~> 8.0', require: false + gem "puppetlabs_spec_helper", '~> 9.0', require: false gem "puppet-blacksmith", '~> 7.0', require: false end group :system_tests do - gem "puppet_litmus", '~> 2.0', require: false, platforms: [:ruby, :x64_mingw] if !ENV['PUPPET_FORGE_TOKEN'].to_s.empty? - gem "puppet_litmus", '~> 1.0', require: false, platforms: [:ruby, :x64_mingw] if ENV['PUPPET_FORGE_TOKEN'].to_s.empty? + # 2.7.0 is the first release whose matrix_from_metadata_v3 knows about Puppet 9: it gates + # the collection on PUPPET_FORGE_TOKEN and emits the '~> 9.0' spec_matrix entry. Floored + # unconditionally at 2.8, since --collection-platform-exclude (which ci.yml/nightly.yml pass + # unconditionally) only exists from 2.8.0 onward -- older 2.x releases hard-fail with + # OptionParser::InvalidOption instead of degrading gracefully. pdk-templates' own + # Gemfile.erb only ever generates a single unconditional puppet_litmus line here; the + # PUPPET_FORGE_TOKEN-gated split this module used to carry was a hand-maintained + # deviation, not something the template emits. + gem "puppet_litmus", '~> 2.8', require: false, platforms: [:ruby, :x64_mingw] gem "CFPropertyList", '< 3.0.7', require: false, platforms: [:mswin, :mingw, :x64_mingw] gem "serverspec", '~> 2.41', require: false end @@ -56,20 +70,13 @@ puppet_version = ENV.fetch('PUPPET_GEM_VERSION', nil) facter_version = ENV.fetch('FACTER_GEM_VERSION', nil) hiera_version = ENV.fetch('HIERA_GEM_VERSION', nil) -# If facter or hiera versions have been specified via the environment -# variables - -# If PUPPET_FORGE_TOKEN is set then use authenticated source for both puppet and facter, since facter is a transitive dependency of puppet -# Otherwise, do as before and use location_for to fetch gems from the default source -if !ENV['PUPPET_FORGE_TOKEN'].to_s.empty? - gems['puppet'] = ['~> 8.11', { require: false, source: 'https://rubygems-puppetcore.puppet.com' }] - gems['facter'] = ['~> 4.11', { require: false, source: 'https://rubygems-puppetcore.puppet.com' }] -else - gems['puppet'] = location_for(puppet_version) - gems['facter'] = location_for(facter_version) if facter_version -end - -gems['hiera'] = location_for(hiera_version) if hiera_version +# Puppet 9.0.0 is a released gem on the standard puppetcore source (confirmed: +# puppetlabs-windows_eventlog#100's CI resolves `puppet (9.0.0)` from +# gemsource_puppetcore with no PUPPET_GEM_SOURCE set) -- no separate internal/Twingate +# source or prerelease-specific version matching is needed for it anymore. +gems['puppet'] = location_for(puppet_version, nil, { source: gemsource_puppetcore }) +gems['facter'] = location_for(facter_version, nil, { source: gemsource_puppetcore }) +gems['hiera'] = location_for(hiera_version, nil, {}) if hiera_version gems.each do |gem_name, gem_params| gem gem_name, *gem_params diff --git a/Rakefile b/Rakefile index 85222267..a10125e4 100644 --- a/Rakefile +++ b/Rakefile @@ -3,7 +3,7 @@ require 'bundler' require 'puppet_litmus/rake_tasks' if Gem.loaded_specs.key? 'puppet_litmus' require 'puppetlabs_spec_helper/rake_tasks' -require 'puppet-syntax/tasks/puppet-syntax' +require 'puppetlabs-syntax/tasks/puppetlabs-syntax' require 'puppet-strings/tasks' if Gem.loaded_specs.key? 'puppet-strings' PuppetLint.configuration.send('disable_relative') diff --git a/metadata.json b/metadata.json index 3d0dab51..2e5c9e65 100644 --- a/metadata.json +++ b/metadata.json @@ -100,7 +100,7 @@ "requirements": [ { "name": "puppet", - "version_requirement": ">= 8.0.0 < 9.0.0" + "version_requirement": ">= 8.0.0 < 10.0.0" } ], "description": "Uses a combination of keytool and Ruby openssl library to manage entries in a Java keystore.",