Puppet 3.7.3 Updates

I took the plunge this weekend and updated my home lab to Puppet v3.7.3. Before you begin, check out the release notes. Puppet v4.0.0 is on the horizon, so there are a lot of new features and options available to you, along with bug fixes, but also some deprecations. It’s a great time to test some of the new features (like the future parser) while you can still turn them off, but as long as you’re coming from the 3.6.x series, your existing config should just work.

I had one issue with the upgrade. I started getting errors that classes couldn’t be found, regardless of which class I included and whether it was via puppet agent or puppet apply. Turned out, I had a manually installed puppet gem. Whoops! It was version 3.6.1 and I simply removed it and everything started working. You really shouldn’t have that issue, unless you’ve been doing testing on your master like I’ve been doing. Don’t do that.

While I’ve only been using it for about 12 hours now, I have to say I love the changes in 3.7.0. The big performance improvement comes from persistent HTTPS connections. Previously, new connections were initiated for (I believe) just about every file transferred, including plugins (facts, types, providers). This is a huge performance increase for me and I still have pretty small manifests. If you have really large catalogs, especially ones that transfer a lot of files, you should be really happy with this. The best part is, you don’t have to enable this, it’s turned on for you.

If you have the time this holiday season, give Puppet 3.7.3 a shot in your lab. You should be pleasantly surprised and prepared to upgrade production after the holiday!

DZone Research’s Guide to Continuous Delivery

Josh Gray posted a Random Thoughts on DevOps article last month that included a reading list. I’ve read the two novels on there, but I had not read DZone Research’s Guide to Continuous Delivery (it’s free with an email, and you will get a few of them but they seem to respect unsubscribe requests). It’s a pretty short guide, something you can read on your lunch break. It does have some overlap with other DevOps guides, but if you’re unfamiliar with Continuous Integration and Continuous Delivery, as ideas or as implementations, it’s definitely worthwhile.

Even if you are familiar with CD, page 24 is awesome. It’s a CD “Maturity Checklist”. If this is all new to you, it takes the previous 23 pages and distills all the advice and generalities down to some pretty specific and relatable items. If you are already on your CD journey, now you can view where you are and where you need to go.

There’s also a nice glossary on page 35. As I’ve said before, it’s a real benefit to everyone when you’re speaking the same language.

I recommend taking a few moments to read over this brief guide. Josh also recommended their Guide to Enterprise Integration, which I haven’t read yet but I’ll take his word for.

Publishing Forge Modules

Last week, we looked at an advanced spec helper, puppetlabs_spec_helper, and generated some tests with it. We also looked at the rake targets available with the helper and you may have noticed the build target: “Build puppet module package”. Prior to that, we created a new certs module that is code only, no data, for use in distribution of certificate files for web servers. It seems like a good opportunity to see how this works so we can upload a module to the forge.

Forge Modules

The Puppet Forge is a central repository for shared modules, written by Puppet Labs or by the community. It’s the puppet analog to perl’s CPAN, python’s pip, etc. – tell puppet you want a module and it fetches it from the forge. As the modules are shared, rather than specific to a user’s installation, be sure to use sound fundamentals to create a portable module. Your role and profile modules, which likely reference umpteen other modules and the files and templates they contain, are not good candidates for the forge, but a utility module like the certs module is a good candidate.


Puppet Labs maintains a large document on the procedure to publish a module to the forge. While we can all read the directions, where’s the fun in that? Well, you should, but we’ll focus on the highlights for now. We’ll need a forge account, an updated metadata.json file, and an uploadable tarball to continue. Create your login and save the user/pass in your password vault. The username you specify should be the same as you use in the metadata’s ‘author’ key. If you already have an account, sign in now.

Next, review the metadata.json in the module. When using puppet module generate, the version is set t0 0.1.0, you may want to increase that now. The default license is Apache 2.0, this will work fine for most people. Make sure the source/issue_url/project_page values point to the correct location and your dependencies are defined. You may want to add an operatingsystem_support hash so users understand where the module has been tested – even on well maintained modules, this doesn’t mean a non-listed operating system will not work, but it ensures that users are not surprised if it fails. Here’s what the certs metadata looks like:

  "name": "rnelson0-certs",
  "version": "0.5.0",
  "author": "rnelson0",
  "summary": "SSL Certificate File Management",
  "license": "Apache 2.0",
  "source": "https://github.com/rnelson0/certs",
  "issues_url": "https://github.com/rnelson0/certs/issues",
  "project_page": "https://github.com/rnelson0/certs",
  "operatingsystem_support": [
    "operatingsystemrelease":[ "6.5" ]
  "dependencies": [
      "version_range": ">= 1.0.0",
      "name": "puppetlabs-stdlib"

There is at least one bug with puppet and broken/incomplete metadata so be sure to make sure the syntax is correct. Beware of semantic version numbers and the placement of commas. Unlike the Puppet DSL, trailing commas in a list are invalid. I am unaware of a programmatic method to determine valid metadata, but the fix at least looks like it will specify the metadata as the error’s source.

With everything in place, it’s time to build the module. Run rake build. If you receive some “cannot parse JSON” errors, go back to the metadata and very that every list item has a comma except the last and that all brackets and braces have a begin and end pair. Once the metadata is correct, you’ll see exactly noting on the command line, but an ls reveals some new files:

[rnelson0@build certs]$ rake build
[rnelson0@build certs]$ ls
manifests  metadata.json  pkg  Rakefile  README.md  spec  tests
[rnelson0@build certs]$ ls pkg
rnelson0-certs-0.5.0  rnelson0-certs-0.5.0.tar.gz

Hooray, our module is ready to upload! Every time you run rake build, it will create a new package or overwrite the existing one. If for any reason you need to remove the packages, use rake clean. Be aware that you cannot undo that action, so don’t clean out old versions that you haven’t uploaded to the forge yet! You do not want to track these files, as they’ll become part of your module and start to bloat the size. Add them to .gitignore:

[rnelson0@build certs]$ cat >> .gitignore

Don’t forget to commit, push, and merge your changes to your upstream now!

Unfortunately, there is no CLI interface for the forge or a way to PUT your package via a URI*. Download the package from your build server, then upload it to the forge.

That’s it. Your module is now available for everyone to download!


Today’s exercise was pretty simple, but powerful: take an existing module, update some metadata, and publish it to the forge. Your module is now available for everyone to download. You can also move your module from the git repo section of your Puppetfile to the forge section, and so can other people. Take some time to promote your module via social media and then the fun stuff begins – Pull Requests. You’re going to have to take care of this thing you just created. For a small utility module like rnelson0-certs, that’s not going to require much. Large modules, like puppetlabs-apache, will require a lot more effort. I think it’s worth it, though!

Apply to be a vExpert 2015 candidate

If you haven’t already, you should apply to be a vExpert 2015 candidate. There are three forms you can use:

You have until Dec 12th to apply (next Friday) and results will be announced February 5th. There will also be a Q2 application period (est. March 15th deadline based on 2014) if you miss this for some reason. If you’re part of the virtualization community and unsure if you should apply, then you should apply. This is not a technical certification or aware, it’s a community award. Writing blogs, participating in VMUGs, using Twitter – these are all activities that vExperts take part in.

And if you don’t apply, you might find me recommending you at the beginning of next week. Hop to it!

Beyond rspec-puppet: puppetlabs_spec_helper

We recently discussed test-driven development for puppet modules in the context of rspec-puppet. That’s a nice, simple introduction to testing, but doesn’t provide everything we need. Rspec-puppet is limited in the matchers available (notably there are no negation tests) and its inability to test dependencies (when a module includes another module), both of which will be necessary eventually. The next step is puppetlabs_spec_helper, a project by Puppet Labs that provides us with more full-fledged specification tests.


The biggest requirement for puppetlabs_spec_helper is a ruby version of 1.9 or higher. CentOS 6.5, however, only includes v1.8.7. There are numerous ways to upgrade ruby, most of which are horrible. We’ll look at using the Ruby Version Manager, or RVM, to upgrade to 1.9.3. This can be done with puppet via the maestrodev/rvm module. After adding the module to your master, create a class or modify an existing one to provide RVM and some puppet and rspec gems.

Continue reading

Pull Requests aren’t just for Code anymore

Pull requests (PRs) are an interface to discuss proposed changes to be integrated into a project. As a sysadmin, you might typically hear about developers using PRs to manage code in a public repository. Even if you don’t know how to code, you can still contribute with PRs to your favorite project.

As a frequent user of r10k, but someone unfamiliar with ruby, I can’t contribute very much to the inner workings of the program. However, as a user, I’m in a good position to provide feedback on the user experience. To that end, I forked the repository on github and created some branches to update the documentation to (hopefully!) improve it for other users. Afterward, I submitted PRs and worked with Adrien Thebo, the project maintainer, to fine tune the PRs till they were correctly implemented. The results of that PR are here and the other PRs are merged or still being edited.

As I’ve noted before, documentation matters. If you can’t or aren’t willing to contribute code on a project, improving the documentation is a great way to give back to the community. Give it a shot!

Toyota Production Systems (Lean) Terminology

I found a great article about the Toyota Production Systems (TPS) terminology. TPS is also known as Lean and is the basis of The Goal, The Phoenix Project and DevOps. I’ll be using the terminology a lot in the future, so take a moment to read up on the terms. A shared language helps ensure effective communication. We’ve already discussed Kanban, here are some other terms to focus on:

  • Andon
  • Kaizen – notice it’s for everyone, not specialists.
  • Nemawashi
  • Muda
  • Mura
  • Muri
  • Set-Up Time
  • Tataki Dai