Sometimes it’s very difficult to see how hiera works and what values it might return. To help with troubleshooting, you can use the hiera cli to do lookups yourself. You obviously need to do this on a puppet master, or a node that’s configured to have all the hiera data in the same place the masters do. Here’s the hiera.yaml configuration:

# managed by puppet
---
:backends:
  - yaml
:logger: console
:hierarchy:
  - "clientcert/%{clientcert}"
  - "puppet_role/%{puppet_role}"
  - global

:yaml:
   :datadir: /etc/puppet/environments/%{environment}/hiera

Tier 1: global

Let’s look up the ntp::servers key from the global tier. One thing to note here is that the datadir includes the variable environment, so we will need to provide that tuple to the cli, like so:

Continue reading →

This weekend, I’ve been playing around with getting my puppet modules up to par with puppet v4. I’ve also recently covered a lot of ground with Rakefiles/Travis CI and excluding files from the build. Building on that, I learned about Blacksmith and how it can let you quickly push new versions of your modules to the forge. It’s really simple.

First, you need an auth file. This belongs to your user, not the module repo, so there’s no chance it will make it into your git repo. You should still be sure it’s not readable to anyone but you, though. The syntax is very simple, just drop this at ~/.puppetforge.yml and replace the defaults with your user/password:

---
url: https://forgeapi.puppetlabs.com
username: myuser
password: mypassword

Next, make sure your module’s Rakefile and Gemfile are set up properly.

$ cat Rakefile
# Relevant snippet
# These gems aren't always present, for instance
# on Travis with --without development
begin
  require 'puppet_blacksmith/rake_tasks'
rescue LoadError # rubocop:disable Lint/HandleExceptions
end

$ cat Gemfile
# Relevant snippet
group :development do
  gem "travis"
  gem "travis-lint"
  gem "puppet-blacksmith"
  gem "guard-rake"
end

Once you use bundler to install the gems locally, you will find some new rake tests available:

$ be rake -T | grep module
rake build                                   # Build puppet module package
rake clean                                   # Clean a built module package
rake module:bump                             # Bump module version to the next patch
rake module:bump:major                       # Bump module version to the next MAJOR version
rake module:bump:minor                       # Bump module version to the next MINOR version
rake module:bump:patch                       # Bump module version to the next PATCH version
rake module:bump_commit                      # Bump version and git commit
rake module:clean                            # Runs clean again
rake module:dependency[module_name,version]  # Set specific module dependency version
rake module:push                             # Push module to the Puppet Forge
rake module:release                          # Release the Puppet module, doing a clean, build, tag, push, bump_commit and git push
rake module:tag                              # Git tag with the current module versio

If your module is ready to go and your authentication works, you just need to do a rake module:release. This will run the following commands (useful if one errors out and you need to replay from there):

bundle exec rake module:clean
bundle exec rake build
bundle exec rake module:tag
bundle exec rake module:push
bundle exec rake module:bump_commit
git push origin

You can of course use the target module:release, or use the independent targets. For instance, I wanted to make sure my build wasn’t 21M again, so I did the bump_commit/clean/build steps first, then the tag/push/git push:

[rnelson0@build02 local_user:master]$ be rake module:bump_commit
Bumping version from 1.0.1 to 1.0.2
[rnelson0@build02 local_user:master]$ be rake build
[rnelson0@build02 local_user:master]$ ls -lh pkg
total 12K
drwxrwxr-x. 5 rnelson0 rnelson0 4.0K Nov 15 17:21 rnelson0-local_user-1.0.2
-rw-rw-r--. 1 rnelson0 rnelson0 7.1K Nov 15 17:21 rnelson0-local_user-1.0.2.tar.gz
[rnelson0@build02 local_user:master]$ be rake module:tag
[rnelson0@build02 local_user:master]$ be rake module:push
Uploading to Puppet Forge rnelson0/local_user
[rnelson0@build02 local_user:master]$ git push origin master
Counting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 317 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
To git@github.com:rnelson0/puppet-local_user
   87a1957..13a66bb  master -> master

For verification, I visit my module on the forge and the latest version now says 1.0.2. Perfect! I know how it works, in case something breaks in the future, and I’m confident that I can release the next version using the module:release target. Thank you to the people at maestrodev and the many  contributors for this awesome tool!

I ran into a funny problem updating my rnelson0/certs puppet module today. I tried to use puppet-blacksmith to create and push a new build to the Forge. It continually failed with a strange error:

[rnelson0@build02 certs:master]$ be rake module:release
Cleaning for module build
Uploading to Puppet Forge rnelson0/certs
rake aborted!
Errno::ECONNRESET: Connection reset by peer

Well crap, that doesn’t make any sense! Let’s do a build manually and take a look at it:

[rnelson0@build02 certs:master]$ puppet module build
Notice: Building /home/rnelson0/modules/certs for release
Module built: /home/rnelson0/modules/certs/pkg/rnelson0-certs-0.7.0.tar.gz
[rnelson0@build02 certs:master]$ ls -lh pkg
total 21M
drwxrwxr-x. 6 rnelson0 rnelson0 4.0K Nov 15 04:16 rnelson0-certs-0.7.0
-rw-rw-r--. 1 rnelson0 rnelson0  21M Nov 15 04:16 rnelson0-certs-0.7.0.tar.gz
[rnelson0@build02 certs:master]$ tar tzvf pkg/rnelson0-certs-0.7.0.tar.gz | more
drwxrwxr-x rnelson0/rnelson0 0 2015-11-15 04:16 rnelson0-certs-0.7.0/
drwxrwxr-x rnelson0/rnelson0 0 2015-11-15 04:16 rnelson0-certs-0.7.0/tests/
-rw-rw-r-- rnelson0/rnelson0 509 2015-10-24 19:27 rnelson0-certs-0.7.0/tests/init.pp
-rw-rw-r-- rnelson0/rnelson0 911468 2015-11-15 04:16 rnelson0-certs-0.7.0/checksums.json
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/manifests/
-rw-rw-r-- rnelson0/rnelson0    310 2015-10-24 17:27 rnelson0-certs-0.7.0/manifests/init.pp
-rw-rw-r-- rnelson0/rnelson0   2208 2015-10-24 17:27 rnelson0-certs-0.7.0/manifests/vhost.pp
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/spec/
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/spec/classes/
-rw-rw-r-- rnelson0/rnelson0    141 2015-10-24 19:27 rnelson0-certs-0.7.0/spec/classes/init_spec.rb
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/spec/defines/
-rw-rw-r-- rnelson0/rnelson0   1433 2015-10-24 19:27 rnelson0-certs-0.7.0/spec/defines/vhost_spec.rb
-rw-rw-r-- rnelson0/rnelson0     52 2015-10-24 17:27 rnelson0-certs-0.7.0/spec/spec_helper.rb
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/spec/fixtures/
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/spec/fixtures/manifests/
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/spec/fixtures/modules/
-rw-rw-r-- rnelson0/rnelson0   1409 2015-10-24 19:27 rnelson0-certs-0.7.0/Rakefile
-rw-rw-r-- rnelson0/rnelson0   2208 2015-11-15 02:58 rnelson0-certs-0.7.0/README.md
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/vendor/
drwxrwxr-x rnelson0/rnelson0      0 2015-11-15 04:16 rnelson0-certs-0.7.0/vendor/ruby/

Can you see the problem? The vendor directory used by bundler is part of the package. That’s not right! It’s causing the forge connection to timeout because of the size or transmission time. Plus, no-one’s going to download a 21M module. Thankfully, there’s an easy way to fix this. In addition to the .gitignore that we use to prevent files from being made part of the git repository, we also need a .pmtignore, but we only need to exclude the vendors path:

[rnelson0@build02 certs:master]$ cat > .pmtignore
vendor/
[rnelson0@build02 certs:master±]$ puppet module build
Notice: Building /home/rnelson0/modules/certs for release
Module built: /home/rnelson0/modules/certs/pkg/rnelson0-certs-0.7.0.tar.gz
[rnelson0@build02 certs:master±]$ ls -lh pkg
total 12K
drwxrwxr-x. 5 rnelson0 rnelson0 4.0K Nov 15 04:23 rnelson0-certs-0.7.0
-rw-rw-r--. 1 rnelson0 rnelson0 7.0K Nov 15 04:23 rnelson0-certs-0.7.0.tar.gz
[rnelson0@build02 certs:master±]$ tar tzvf pkg/rnelson0-certs-0.7.0.tar.gz
drwxrwxr-x rnelson0/rnelson0 0 2015-11-15 04:23 rnelson0-certs-0.7.0/
drwxrwxr-x rnelson0/rnelson0 0 2015-11-15 04:23 rnelson0-certs-0.7.0/tests/
-rw-rw-r-- rnelson0/rnelson0 509 2015-10-24 19:27 rnelson0-certs-0.7.0/tests/init.pp
-rw-rw-r-- rnelson0/rnelson0 631 2015-11-15 04:23 rnelson0-certs-0.7.0/checksums.json
-rw-rw-r-- rnelson0/rnelson0 10143 2015-11-15 03:42 rnelson0-certs-0.7.0/Gemfile.lock
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/manifests/
-rw-rw-r-- rnelson0/rnelson0   310 2015-10-24 17:27 rnelson0-certs-0.7.0/manifests/init.pp
-rw-rw-r-- rnelson0/rnelson0  2208 2015-10-24 17:27 rnelson0-certs-0.7.0/manifests/vhost.pp
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/spec/
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/spec/classes/
-rw-rw-r-- rnelson0/rnelson0   141 2015-10-24 19:27 rnelson0-certs-0.7.0/spec/classes/init_spec.rb
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/spec/defines/
-rw-rw-r-- rnelson0/rnelson0  1433 2015-10-24 19:27 rnelson0-certs-0.7.0/spec/defines/vhost_spec.rb
-rw-rw-r-- rnelson0/rnelson0    52 2015-10-24 17:27 rnelson0-certs-0.7.0/spec/spec_helper.rb
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/spec/fixtures/
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/spec/fixtures/manifests/
drwxrwxr-x rnelson0/rnelson0     0 2015-11-15 04:23 rnelson0-certs-0.7.0/spec/fixtures/modules/
-rw-rw-r-- rnelson0/rnelson0  1409 2015-10-24 19:27 rnelson0-certs-0.7.0/Rakefile
-rw-rw-r-- rnelson0/rnelson0  2208 2015-11-15 02:58 rnelson0-certs-0.7.0/README.md
-rw-rw-r-- rnelson0/rnelson0   861 2015-10-24 19:27 rnelson0-certs-0.7.0/Gemfile
-rw-rw-r-- rnelson0/rnelson0   506 2015-11-15 04:23 rnelson0-certs-0.7.0/metadata.json
[rnelson0@build02 certs:master±]$

I’m going to submit a PR against puppet-module-skeleton to get that file added for the future (PR95).