A few years ago, I set up my modules with a bundle-based test setup and modulesync and wrote a companion blog post. Since that was written, a lot of things have changed with puppet. One of those is the release last year of the Puppet Development Kit (PDK). The goal of the PDK is to simply development of puppet code and modules by reducing or eliminating all of the headaches of creating a mature ruby/bundler/puppet-lint/etc. setup. There is also a brand new tool called pdksync that combines the PDK with the power of modulesync. I was somewhat involved in the initial efforts toward the PDK, through my work on puppet-lint, but I have not actually used the PDK “in anger” yet, in part because of my previously working modulesync setup. This seems like a great opportunity to switch to PDK and pdksync, starting with the PDK.
Why PDK?
Before we begin, we should look at why we want to use the Puppet Development Kit. My current setup is best described as fragile. Its effectiveness varies based on what version of Ruby I use and the version of the gems I happen to download via bundler on any given day. I use CentOS 7, which is stuck on Ruby 2.0. Most of the gems in my setup require at least Ruby 2.1 or 2.2, so I have to resort to RVM to provide me with Ruby 2.3.1. Someday, I’ll need to update that to Ruby 2.4 for a gem and my setup will break until I fix it.
I am also downloading a bunch of gems that are not pinned and updated versions can bring in subtle bugs or cascading failures not related to changes in my code. Sometimes the gem is directly related to my work, like a puppet-lint version with a bug that I can downgrade and pin. Other times, it’s a very indirect dependency of a dependency of a dependency to puppet-lint, and pinning it only creates more problems for everything that depends on it. Of course, bundler also relies on rubygems.org and internet/mirror access, which sometimes go down when you need them most.
While these are surmountable issues, they always come up while I’m trying to get something done in puppet, and the minutes or hours required to fix the problem prevents me from making the changes I need, when I need.
The PDK resolves this by bundling its own version of Ruby and dependent gems. Puppet vets the setup, so I do not have to. Everything is on disk, so there’s no more required downloading of gems that can be pulled or unavailable because of network issues. This is a huge benefit to all users, whether they pay for Puppet Enterprise or use Puppet Opensource Edition for free. Less time spent worrying about dependency hell and more time getting straight to work. This is important, valuable work, but it’s not my expertise or an actual goal of my job, so I am very content to let someone else handle the setup so I can spend more time managing my systems with Puppet.
The PDK is an installable tool. Install once and you can use it with all your puppet modules and controlrepos. Upgrading the PDK is simple using your package manager. You can of course combine using the PDK on some modules and stick with the ruby/bundler setup on others. However, it will be more difficult (but not impossible) to switch between the PDK and native bundler on the same module – our CI systems will use native bundler after all – but some gem dependencies will no longer be pinned and we lose the guarantee of pinned gems that will work together.
We will see below that you can still modify the setup on each module/controlrepo to some extent, but when using the PDK, the full range of customization bundler offered is unavailable to you. I think most people will not find this to be a problem, but you should definitely read up on the PDK to make sure you understand what you gain and lose before converting to using it. If you change your mind later, switching back from PDK to a bundler-based setup is possible, but it may involve some work to find a working setup of pinned gem versions.
Installing the PDK
The very first thing I need to do is install the PDK. The following is written using PDK v1.5.0.0. The PDK is relatively new and gets frequent updates, so this may become out of date rapidly. If you run into any issues, check the version, read the release notes, and adjust accordingly.
The docs decribe how to install on various systems. I use EL7 so I will install the RPM. I also use Puppet Enterprise, not Puppet Opensource, so I have to add the Puppet repository first. rpm and yum can get me there, or I can use puppet apply:
# Manual
sudo rpm -Uvh https://yum.puppet.com/puppet5/puppet5-release-el-7.noarch.rpm
sudo yum install pdk -y
# puppet apply
cat > ~/pdk.pp < EOF
package { 'puppet5-release-el-7':
ensure => present,
provider => 'rpm',
source => 'https://yum.puppet.com/puppet5/puppet5-release-el-7.noarch.rpm',
}
-> package { 'pdk':
ensure => present,
}
EOF
sudo puppet apply ~/pdk.pp
I can now call the command pdk successfully. Be aware that it includes its own bundled ruby, so the first time you run it may take a little time to be loaded and cached, which is expected.
[rnelson0@build03 domain_join:master]$ pdk --help
NAME
pdk - Puppet Development Kit
USAGE
pdk command [options]
DESCRIPTION
The shortest path to better modules.
COMMANDS
build Builds a package from the module that can be published to the Puppet Forge.
bundle (Experimental) Command pass-through to bundler
convert Convert an existing module to be compatible with the PDK.
help show help
module Provide CLI-backwards compatibility to the puppet module tool.
new create a new module, etc.
test Run tests.
update Update a module that has been created by or converted for use by PDK.
validate Run static analysis tests.
OPTIONS
-d --debug Enable debug output.
-f --format= Specify desired output format. Valid
formats are 'junit', 'text'. You may also
specify a file to which the formatted
output is sent, for example:
'--format=junit:report.xml'. This option
may be specified multiple times if each
option specifies a distinct target file.
-h --help Show help for this command.
--version Show version of pdk.
If you sign up for the puppet-announce mailing list, you will be notified every time there’s a new PDK release. After reading the release notes for edge cases that may impact you, can easily upgrade to the latest version with your distro’s equivalent of yum update pdk. That is a lot easier than updating ruby/bundler setups.
Converting to the PDK
Next, my existing setup must be converted to PDK. I will walk through my efforts, but you can also review the PuppetConf 2017 video and slides about the PDK in addition to this Converting To PDK doc. I am working on my domain_join module first, starting from the release candidate for v0.5.2, if you want to recreate this effort. The module is part of my modulesync config and has 68 tests. It’s a reasonably mature module but not overly complex, perfect for testing without being too deep or shallow. I am also going to break the modulesync setup, which you can see here. Before beginning, I create a new branch on the module:
[rnelson0@build03 domain_join:master]$ git checkout -b pdk
Switched to a new branch 'pdk'
Because I use modulesync, this will not work out of the box for me, but there’s a very naive default pdk convert that can be used to update the config. It will inform you of the files that will be added/modified and prompts you to continue due to the potential for destruction. As noted, this concern is mitigated by using version control, and if you’ve read my blog before, you’re obviously using version control, right? If not, get that done first! (it’s beyond the scope of this article, but my git 101 article may help) Here’s what the naive attempt looks like:
[rnelson0@build03 domain_join:pdk]$ pdk convert
------------Files to be added-----------
.pdkignore
.project
spec/default_facts.yml
.gitlab-ci.yml
appveyor.yml
----------Files to be modified----------
metadata.json
spec/spec_helper.rb
.gitignore
.travis.yml
Rakefile
.rubocop.yml
.rspec
Gemfile
----------------------------------------
You can find a report of differences in convert_report.txt.
pdk (INFO): Module conversion is a potentially destructive action. Ensure that you have committed your module to a version control system or have a backup, and review the changes above before continuing.
Do you want to continue and make these changes to your module? Yes
------------Convert completed-----------
5 files added, 8 files modified.
[rnelson0@build03 domain_join:pdk±]$
The git diff is really, really lengthy, but you can find it here. A lot of it is simply re-arranging of stanzas in existing files (.gitignore, .rspec, .travis.yml, metadata.json) and .rubocopy.yml updates. The rest is mostly in three files: Rakefile, Gemfile and spec/spec_helper.rb. It also adds five files: .gitlab-ci.yml, .pdkignore, .project, appveyor.yml, and spec/default_facts.yml. Some info on the minor changes:
- spec/default_facts.yml: If you had default facts in your spec/spec_helper.rb file, you should move them here. The rest is mostly “housekeeping” but it removes my hiera config, which I will explore in a moment.
- .gitignore, .pdkignore: The former has been updated a bit, and the latter is the exact same thing.
- .gitlab-ci.yml, .travis.yml, appveyor.yml: Puppet provides some good defaults for a number of external systems. I am sticking with Travis CI right now, but it’s great to have defaults for other services if I branch out. The latter looks targeted at testing Windows systems, too, something that’s often problematic. These are all optional but do not hurt by being present.
- .project: Looks like some XML for use with the editor Eclipse.
- .rubocop.yml: I really don’t like rubocop but it’s included. I plan to disable rubocop as quickly as possible. However, this addresses one of my pain points – every version of rubocop changes the name of Cops, and it fails to run if it finds an unknown Cop name in its config. Since Puppet vets the config, I do not have to deal with finding all the new Cop information everytime rubocop updates. It’s not enough for me to love it, but it is significant pain reduction.
This leaves the big three files mentioned earlier, which are worthy of more detailed investigation.
Rakefile
This file is MUCH smaller now, I presume in thanks due to some pdk magic. The conversion removed a changelog task I created, so I need to get this back.
GitHubChangelogGenerator::RakeTask.new :changelog do |config|
version = (Blacksmith::Modulefile.new).version
config.future_release = "v#{version}"
config.header = "# Change log\n\nAll notable changes to this project will be documented in this file.\nEach new release typically also includes the latest modulesync defaults.\nThese should not impact the
config.exclude_labels = %w{duplicate question invalid wontfix modulesync}
end
I also have a number of puppet-lint checks I have disabled, like arrow_alignment, that I need to make sure are restored. After restoring my task and disabled checks, I will be okay with this new, slimmer default.
Gemfile
PDK includes its own version of ruby and bundler and is guaranteed to deliver a gemset with all the dependencies needed to work together. You can run pdk bundle exec gem list to see what it includes, if you are curious what those are. I will add the github_changelog_generator gem here soon, but otherwise as long as everything works, I have no need to poke at this file anymore.
spec/spec_helper.rb
Though the diff is fairly long for this file, there is nothing tricky here, it just connects the new default facts and some other common practices. It DOES remove the hiera configuration. There is a more modern version of my hiera_config that we need to add back in:
RSpec.configure do |c|
c.hiera_config = 'spec/fixtures/hiera/hiera.yaml'
end
The naive conversion is not that bad for my setup, but it does leave me with three changes to make to keep functional parity: add the github_changelog_generator gem, the :changelog rake task, and re-enable hiera lookups.
Updating the PDK Setup
Now that I’ve identified the non-default changes needed, I can do some updates. The PDK can convert and update modules using a template system. The template it used is listed at the bottom of metadata.json. You can find the templates online, or clone that directory and examine the moduleroot contents (the moduleroot_init directory is also used when you run pdk new):
[rnelson0@build03 domain_join:pdk±]$ git diff metadata.json
diff --git a/metadata.json b/metadata.json
index 7730b90..381aff7 100644
--- a/metadata.json
+++ b/metadata.json
@@ -27,5 +27,8 @@
"name": "puppet",
"version_requirement": ">=4.0.0"
}
- ]
+ ],
+ "pdk-version": "1.5.0",
+ "template-url": "file:///opt/puppetlabs/pdk/share/cache/pdk-templates.git",
+ "template-ref": "1.5.0-0-gd1b3eca"
}
The copies on disk are from the RPM, and are almost definitely out of date. The latest templates are on GitHub. I can re-run the conversion with pdk convert --template-url=https://github.com/puppetlabs/pdk-templates. The changes for me are pretty small but will be much larger the further away in time you are from the date of the RPM build. After running it, the template info will also be updated:
+ ],
+ "pdk-version": "1.5.0",
+ "template-url": "https://github.com/puppetlabs/pdk-templates",
+ "template-ref": "heads/master-0-g7b5f6d2"
We can look at the individual templates here or clone it locally. The first thing to note is that frequently the .erb templates are dynamic data, rather than static. The simplest change is in spec/spec_helper.rb, just adding a single stanza to the Rspec.configure section, which is also dynamic:
RSpec.configure do |c|
c.default_facts = default_facts
<%- if @configs['hiera_config'] -%>
c.hiera_config = "<%= @configs['hiera_config'] %>"
<%- end -%>
<%- if @configs['strict_level'] -%>
c.before :each do
# set to strictest setting for testing
# by default Puppet runs at warning level
Puppet.settings[:strict] = <%= @configs['strict_level'] %>
end
<%- end -%>
end
I highlighted the conditional that will populate the filename with the contents of configs['hiera_config']. The configs hash is populated by config_defaults.yml. The README has a lot of helpful information on the defaults. There’s just a few lines for the spec_helper.rbfile:
[rnelson0@build03 pdk-templates:master]$ tail -2 config_defaults.yml
spec/spec_helper.rb:
strict_level: ":warning"
I need to add to this hash, but I cannot add to the templates since they are upstream. Thankfully, there’s a built in way to account for this. The contents of the configs hash are combined with the same hash taken from the local.sync.yml file!
Note: if you’d like you CAN change the templates by forking puppetlabs/pdk-templates and passing in --template-url when you call pdk new, convert, or update. You are then on the hook for updating your templates over time, though.
To make use of the sync file, I just need to add it to the root of my module directory and add the custom config. It is additive, so only differences need to be present. Here is the hiera_config value required:
[rnelson0@build03 domain_join:pdk±]$ cat .sync.yml
spec/spec_helper.rb:
hiera_config: 'spec/fixtures/hiera.yaml'
With the use of the pdk update command, I can re-apply the templates in --noop mode and see the change:
[rnelson0@build03 domain_join:pdk±]$ pdk update --noop
pdk (INFO): Updating rnelson0-domain_join using the default template, from 1.5.0 to 1.5.0
----------Files to be modified----------
spec/spec_helper.rb
----------------------------------------
You can find a report of differences in update_report.txt.
[rnelson0@build03 domain_join:pdk±]$ cat update_report.txt
/* Report generated by PDK at 2018-05-29 20:08:11 +0000 */
--- spec/spec_helper.rb 2018-05-29 18:53:09.140882197 +0000
+++ spec/spec_helper.rb.pdknew 2018-05-29 20:08:11.819978562 +0000
@@ -28,6 +28,7 @@
RSpec.configure do |c|
c.default_facts = default_facts
+ c.hiera_config = spec/fixtures/hiera.yaml
c.before :each do
# set to strictest setting for testing
# by default Puppet runs at warning level
Now that we have proved out the process, I need to make a few more changes. To add the github_changelog_generator, I add an array entry under Gemfile: required: ':development'. To add the task, I use Rakefile: extras:for the rake task, one entry per line (you can also use multi-line content in yaml if you prefer). This is what the file looks like as well as the pending changes:
[rnelson0@build03 domain_join:pdk±]$ cat .sync.yml
spec/spec_helper.rb:
hiera_config: 'spec/fixtures/hiera.yaml'
Gemfile:
required:
':development':
- gem: github_changelog_generator
Rakefile:
default_disabled_lint_checks:
- 'arrow_alignment'
- 'class_inherits_from_params_class'
- 'class_parameter_defaults'
- 'documentation'
- 'single_quote_string_with_variables'
extras:
- "require 'github_changelog_generator/task'"
- 'GitHubChangelogGenerator::RakeTask.new :changelog do |config|'
- ' version = (Blacksmith::Modulefile.new).version'
- ' config.future_release = "v#{version}"'
- ' config.header = "# Change log\n\nAll notable changes to this project will be documented in this file.\nEach new release typically also includes the latest modulesync defaults.\nThese should not impact the functionality of the module."'
- ' config.exclude_labels = %w{duplicate question invalid wontfix modulesync}'
- 'end'
[rnelson0@build03 domain_join:pdk±]$ pdk update --noop
pdk (INFO): Updating rnelson0-domain_join using the template at https://github.com/puppetlabs/pdk-templates, from master@7b5f6d2 to 1.5.0
----------Files to be modified----------
spec/spec_helper.rb
Rakefile
Gemfile
----------------------------------------
You can find a report of differences in update_report.txt.
[rnelson0@build03 domain_join:pdk±]$ cat update_report.txt
/* Report generated by PDK at 2018-05-29 20:39:41 +0000 */
--- spec/spec_helper.rb 2018-05-29 20:33:16.488401096 +0000
+++ spec/spec_helper.rb.pdknew 2018-05-29 20:39:41.492124202 +0000
@@ -28,6 +28,7 @@
RSpec.configure do |c|
c.default_facts = default_facts
+ c.hiera_config = "spec/fixtures/hiera.yaml"
c.before :each do
# set to strictest setting for testing
# by default Puppet runs at warning level
--- Rakefile 2018-05-29 20:33:16.489401137 +0000
+++ Rakefile.pdknew 2018-05-29 20:39:41.492832995 +0000
@@ -3,4 +3,11 @@
require 'puppet_blacksmith/rake_tasks' if Bundler.rubygems.find_name('puppet-blacksmith').any?
PuppetLint.configuration.send('disable_relative')
+
+require 'github_changelog_generator/task'
+GitHubChangelogGenerator::RakeTask.new :changelog do |config|
+ version = (Blacksmith::Modulefile.new).version
+ config.future_release = "v#{version}"
+ config.header = "# Change log\n\nAll notable changes to this project will be documented in this file.\nEach new release typically also includes the latest modulesync defaults.\nThese should not impact the functionality of the module."
+ config.exclude_labels = %w{duplicate question invalid wontfix modulesync}
+end
--- Gemfile 2018-05-29 20:16:10.321541394 +0000
+++ Gemfile.pdknew 2018-05-29 20:39:41.494035036 +0000
@@ -34,6 +34,7 @@
gem "puppet-module-win-default-r#{minor_version}", require: false, platforms: [:mswin, :mingw, :x64_mingw]
gem "puppet-module-win-dev-r#{minor_version}", require: false, platforms: [:mswin, :mingw, :x64_mingw]
gem "puppet-blacksmith", '~> 3.4', require: false, platforms: [:ruby]
+ gem "github_changelog_generator", require: false
end
puppet_version = ENV['PUPPET_GEM_VERSION']
I now run it in yesop mode and my changes take. A quick check of rake targets confirms it. Note that all pdk bundle output is written to STDERR, not STDOUT.
[rnelson0@build03 domain_join:pdk±]$ pdk bundle exec rake -T 2>&1 | grep change
rake changelog # Generate a Change log from GitHub
I did not add the puppet-lint disable checks back in here. That is because the PDK does not use the Rakefile when running puppet-lint, it relies on the configuration file. I need to create .puppet-lint.rc at the top of the repo so that the settings are available to my CI system. That file looks like this:
[rnelson0@build03 domain_join:pdk±]$ cat .puppet-lint.rc
--no-arrow_alignment-check
--no-class_inherits_from_params_class-check
--no-documentation-check
--no-single_quote_string_with_variables-check
One difference between the Rake target and the config file is that an invalid check name in the config file can cause errors, whereas the Rake setting just doesn’t do anything. I removed the class_parameter_defaults check from the list because it is no longer a valid check.
There are a lot more things you might want to change, especially if you use CI other than Travis, but this should be enough for me to gain parity with my existing setup. Remember that you can poke at the templates online, find the default settings in config_defaults.yml, tweak in your own .sync.yml, re-run pdk update and everything should work out. If the templates cannot be wrangled as is, you can always open a ticket in the PDK project.
Make sure you commit your changes and push them up to version control, eventually to be merged into master.
First Test
Now I need to run my tests. Before I do that, I clean up everything not in git. Since I have developed in this directory, there are bundler files that don’t need to be there and may cause conflicts with the tests. Again, make sure you’ve committed changes first, or some of your uncommitted changes from the conversion will be removed:
[rnelson0@build03 domain_join:pdk±]$ git clean -ffdx
Removing .bundle/
Removing Gemfile.lock
Removing bin/
Removing convert_report.txt
Removing coverage/
Removing pkg/
Removing spec/defines/
Removing spec/fixtures/manifests/
Removing spec/fixtures/modules/
Removing spec/functions/
Removing spec/hosts/
Removing update_report.txt
Removing vendor/
PDK Tests
The first test is real simple, it’s my unit tests via pdk test unit:
[rnelson0@build03 domain_join:pdk±]$ pdk test unit
pdk (INFO): Using Ruby 2.4.4
pdk (INFO): Using Puppet 5.5.1
[✔] Preparing to run the unit tests.
[✔] Running unit tests.
Evaluated 68 tests in 2.321391522 seconds: 0 failures, 0 pending.
[✔] Cleaning up after running unit tests.
I also want to validate linting and syntax and whatnot with pdk validate:
[rnelson0@build03 domain_join:pdk]$ pdk validate
pdk (INFO): Running all available validators...
pdk (INFO): Using Ruby 2.4.4
pdk (INFO): Using Puppet 5.5.1
[✔] Checking metadata syntax (metadata.json tasks/*.json).
[✔] Checking module metadata style (metadata.json).
[✔] Checking Puppet manifest syntax (**/**.pp).
[✔] Checking Puppet manifest style (**/*.pp).
[✖] Checking Ruby code style (**/**.rb).
info: task-metadata-lint: ./: Target does not contain any files to validate (tasks/*.json).
convention: rubocop: spec/spec_helper_acceptance.rb:17:27: Style/HashSyntax: Use the new Ruby 1.9 hash syntax.
convention: rubocop: spec/spec_helper_acceptance.rb:17:49: Style/HashSyntax: Use the new Ruby 1.9 hash syntax.
<more rubocop results>
I have a ton of rubocop results, which I will address below. Everything else works fine, as expected.
CI Tests
The second is a little trickier. Currently, whatever CI system you use will use ruby/bundler to perform the same checks. That is planned to change (PDK-709 tracks the Travis CI setup) When I use Travis CI, it uses tests from .travis.yml. Here are the relevant portions:
bundler_args: --without system_tests
matrix:
fast_finish: true
include:
-
env: CHECK="syntax lint metadata_lint check:symlinks check:git_ignore check:dot_underscore check:test_file rubocop"
-
env: CHECK=parallel_spec
-
env: PUPPET_GEM_VERSION="~> 4.0" CHECK=parallel_spec
rvm: 2.1.9
There are two different checks that will run. The first is all the syntax and linting, the equivalent of pdk validate. The second and third are the unit tests, run against the latest Puppet 4 and Puppet 5 independently, and equivalent to pdk test unit. Here’s what happens when I run the unit tests first:
[rnelson0@build03 domain_join:pdk]$ pdk bundle exec rake parallel_spec
pdk (INFO): Using Ruby 2.4.4
pdk (INFO): Using Puppet 5.5.1
Cloning into 'spec/fixtures/modules/stdlib'...
2 processes for 2 specs, ~ 1 specs per process
No examples found.
Finished in 0.00032 seconds (files took 0.07684 seconds to load)
0 examples, 0 failures
domain_join
on redhat-6-x86_64
with defaults for all parameters
should not contain Package[samba-common-tools]
should contain Package[oddjob-mkhomedir]
should contain Package[krb5-workstation]
should contain Package[krb5-libs]
should contain Package[samba-common]
should contain Package[sssd-ad]
should contain Package[sssd-common]
should contain Package[sssd-tools]
should contain Package[ldb-tools]
should contain Class[domain_join]
should contain File[/etc/resolv.conf]
should contain File[/etc/krb5.conf]
should contain File[/etc/samba/smb.conf]
should contain File[/etc/sssd/sssd.conf]
should contain File[/usr/local/bin/domain-join]
should contain Exec[join the domain]
with manage_services false
should not contain Package[sssd]
should not contain File[/etc/sssd/sssd.conf]
should contain File[/etc/resolv.conf]
should contain File[/usr/local/bin/domain-join]
with manage_services and manage_resolver false
should not contain Package[sssd]
should not contain File[/etc/sssd/sssd.conf]
should not contain File[/etc/resolv.conf]
should contain File[/usr/local/bin/domain-join]
start script syntax
should contain File[/usr/local/bin/domain-join] with content =~ /sssd status/
with container
should contain File[/usr/local/bin/domain-join] with content =~ /net ads join/
should contain File[/usr/local/bin/domain-join] with content =~ /container_ou='container'/
with account and password
should contain File[/usr/local/bin/domain-join] with content =~ /register_account='service_account'/
should contain File[/usr/local/bin/domain-join] with content =~ /register_password='open_sesame'/
with join_domain disabled
should not contain Exec[join the domain]
with manage_dns disabled
should not contain File[/usr/local/bin/domain-join] with content =~ /net ads dns register/
should not contain File[/usr/local/bin/domain-join] with content =~ /update add /
with manage_dns and ptr enabled
should contain File[/usr/local/bin/domain-join] with content =~ /net ads dns register/
should contain File[/usr/local/bin/domain-join] with content =~ /update add .+ addr show fake_interface/
on redhat-7-x86_64
with defaults for all parameters
should contain Package[samba-common-tools]
should contain Package[oddjob-mkhomedir]
should contain Package[krb5-workstation]
should contain Package[krb5-libs]
should contain Package[samba-common]
should contain Package[sssd-ad]
should contain Package[sssd-common]
should contain Package[sssd-tools]
should contain Package[ldb-tools]
should contain Class[domain_join]
should contain File[/etc/resolv.conf]
should contain File[/etc/krb5.conf]
should contain File[/etc/samba/smb.conf]
should contain File[/etc/sssd/sssd.conf]
should contain File[/usr/local/bin/domain-join]
should contain Exec[join the domain]
with manage_services false
should not contain Package[sssd]
should not contain File[/etc/sssd/sssd.conf]
should contain File[/etc/resolv.conf]
should contain File[/usr/local/bin/domain-join]
with manage_services and manage_resolver false
should not contain Package[sssd]
should not contain File[/etc/sssd/sssd.conf]
should not contain File[/etc/resolv.conf]
should contain File[/usr/local/bin/domain-join]
start script syntax
should contain File[/usr/local/bin/domain-join] with content =~ /status sssd.service/
with container
should contain File[/usr/local/bin/domain-join] with content =~ /net ads join/
should contain File[/usr/local/bin/domain-join] with content =~ /container_ou='container'/
with account and password
should contain File[/usr/local/bin/domain-join] with content =~ /register_account='service_account'/
should contain File[/usr/local/bin/domain-join] with content =~ /register_password='open_sesame'/
with join_domain disabled
should not contain Exec[join the domain]
with manage_dns disabled
should not contain File[/usr/local/bin/domain-join] with content =~ /net ads dns register/
should not contain File[/usr/local/bin/domain-join] with content =~ /update add /
with manage_dns and ptr enabled
should contain File[/usr/local/bin/domain-join] with content =~ /net ads dns register/
should contain File[/usr/local/bin/domain-join] with content =~ /update add .+ addr show fake_interface/
1 deprecation warning total
Finished in 2.34 seconds (files took 2.17 seconds to load)
68 examples, 0 failures
68 examples, 0 failures
Took 5 seconds
I, [2018-05-29T21:51:29.950455 #16602] INFO -- : Creating symlink from spec/fixtures/modules/domain_join to /home/rnelson0/modules/domain_join
/opt/puppetlabs/pdk/share/cache/ruby/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core.rb:179:in `block in const_missing': uninitialized constant RSpec::Puppet (NameError)
from /opt/puppetlabs/pdk/share/cache/ruby/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core.rb:179:in `fetch'
from /opt/puppetlabs/pdk/share/cache/ruby/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core.rb:179:in `const_missing'
from /home/rnelson0/modules/domain_join/spec/classes/coverage_spec.rb:1:in `block in '
Deprecation Warnings:
puppetlabs_spec_helper: defaults `mock_with` to `:mocha`. See https://github.com/puppetlabs/puppetlabs_spec_helper#mock_with to choose a sensible value for you
If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.
Tests Failed
The text in bold indicates an error in spec/classes/coverage_spec.rb. The simple solution for me is to git rm it, rather than add in the right coverage gem again. It’s not particularly important to me, but if it is to you, you need to add it back to Gemfile and spec/spec_helper.rb. The important thing is that a second run does not have the error and completes successfully.
The second test is a series of rake targets and causes me some grief out of the gate:
[rnelson0@build03 domain_join:pdk]$ pdk bundle exec rake syntax lint metadata_lint check:symlinks check:git_ignore check:dot_underscore check:test_file rubocop
pdk (INFO): Using Ruby 2.4.4
pdk (INFO): Using Puppet 5.5.1
init.pp
---> syntax:manifests
---> syntax:templates
---> syntax:hiera:yaml
rake aborted!
.pp files present in tests folder; Move them to an examples folder following the new convention
/opt/puppetlabs/pdk/share/cache/ruby/2.4.0/gems/puppetlabs_spec_helper-2.7.0/lib/puppetlabs_spec_helper/rake_tasks.rb:231:in `block (2 levels) in '
/opt/puppetlabs/pdk/share/cache/ruby/2.4.0/gems/rake-12.3.1/exe/rake:27:in `'
/opt/puppetlabs/pdk/private/ruby/2.4.4/bin/bundle:23:in `load'
/opt/puppetlabs/pdk/private/ruby/2.4.4/bin/bundle:23:in `'
Tasks: TOP => check:test_file
(See full trace by running task with --trace)
The fix is easy – move the existing files to a new location as the convention has changed, or remove them entirely if they are not valuable – and then I can proceed without further fault:
[rnelson0@build03 domain_join:pdk]$ mkdir examples
[rnelson0@build03 domain_join:pdk]$ git mv -v tests/*pp examples/
‘tests/init.pp’ -> ‘examples/init.pp’
[rnelson0@build03 domain_join:pdk±]$ pdk bundle exec rake syntax lint metadata_lint check:symlinks check:git_ignore check:dot_underscore check:test_file rubocop
pdk (INFO): Using Ruby 2.4.4
pdk (INFO): Using Puppet 5.5.1
Running RuboCop...
Inspecting 0 files
0 files inspected, no offenses detected
---> syntax:manifests
---> syntax:templates
---> syntax:hiera:yaml
As I mentioned earlier, I’d like to disable RuboCop. I don’t see how right now. If I specify selected_profile: off in .sync.yml for rubocop, pdk update errors out applying the template (PDK-998). However it seems to pass just fine in that check, though the individual check fails badly (PDK-997). I’m content to let it go so long as it’s passing test and I don’t have to rewrite anything, but I will find SOME way to get rid of it if it starts causing me problems!
If you use gitlab-ci, appveyor, or some other system for testing, you will want to ensure those tests pass as well. Once done, commit everything to git again.
I am now ready to submit a pull request, and if you are following along, you may be, too. You can review and compare my pull request and the tests if you would like. You will of course notice that I merged it in spite of rubocop failures!
Summary
We have looked at what the PDK is, why we want to use it, how to install it, and how to convert a module to use it. Each module can be customized and we explored the .sync.yml file that controls customization. Once we finalized our conversion, we ran the same tests we had prior to the PDK to make sure they still work and verified the Travis CI tests, too. The next step is to find a replacement for modulesync, which allows use to push the same general configuration to multiple modules. Lucky for us, Puppet just released a potential replacement, pdksync, that I will evaluate soon.
rnelson0 