Upgrading Puppet OpenSource Agents with puppet_agent and jlambert121/puppet

Last week, I discussed how to upgrade your Linux and Windows Puppet Enterprise agents with puppet_agent. Today, we will look at two different ways to upgrade your Puppet OpenSource agents, first with puppet_agent and then with jlambert121/puppet. There are many other modules that you can use to manage puppet, some of which offer similar functionality that you may be able to apply some of these lessons to.

Why are we looking at two different modules today? I use jlambert121’s module to manage my puppetserver already. Unfortunately, it is puppet 4 only and I still had some puppet 3 agents hanging around according to puppetboard. I also have some other modules that are puppet 4 only, like the latest version of puppetlabs/ntp. I could use jlambert121/puppet to upgrade my agents that are already on puppet 4, but need something to help get my puppet 3 agents over the hump – that’s where puppet_agent comes in. We will start with that.

Continue reading

Tracking Puppet agent status with Puppetboard

I manage about 15 VMs in my home lab with OpenSource Puppet. Unlike Puppet Enterprise, there’s no Console to help you determine when agents stop checking in or encountering any problems applying their catalog. With only 15 agents it’s not a huge problem, but it always rears its head at the worst time. I decided to deploy Puppetboard at home to get a similar view to PE’s Console. As is always the way, we’re going to deploy that using a puppet module, Vox Pupuli’s puppet/puppetboard. Now, I’ve got Puppetboard running, but it’s served via HTTP and communicates with puppetdb via HTTP, so there’s room to improve. I’m not sure I’ll get to that, since this is on a private network, but something to keep in mind if you’re implementing this in a more sensitive deployment.

Special thanks to Tim Meusel for his assistance with some of the settings!

The first step is to add the module and its dependencies to the Puppetfile and .fixtures.yml. You may have a few of these modules already, but I’ve bumped them to the latest version.

Continue reading

Learn about Puppet module releases – via email!

We’ve all found ourselves anxiously awaiting a new module release of our favorite puppet module on the forge. It might be coming down the pipe today, next month, or who knows when, but you’re going to need it as soon as it’s released. The forge fairly recently announced an RSS feed per module. You can see this in your RSS reader, if you use one, or get email notifications with a number of services, a few of which are mentioned in that blog. I have wanted to play with If This, Then That for a while, so I decided to see if I could figure out how to get emails through it.

You’ll need an IFTTT account, at least one puppet module you want to receive alerts on, and of course an email address to receive the alert. Once you sign up for IFTTT, go to My Applets and choose New Applet. On the next page, it’s not obvious but click the +this text and it brings you to a selection page. Click on Feed, then New Feed Item. Now switch over to your target module on the forge and find the rss link. The pattern’s pretty easy, just add /rss to the module’s link.

Now click on +that and select Email, then Send me an email. If you want, adjust the contents of the email – I found the default field setup okay – and then hit Create Action. Click on the slider to the right of Receive notifications when this Applet runs and hit Finish


If you want to change anything, click the gear icon, adjust what you need and hit Save. Repeat for all the modules you want to receive notifications upon update. Here’s what an email will look like:


You can also view, update, and add additional applets using the IFTTT app for your phone or tablet. This might not be very pretty or exciting, but since I just want to get alerts without firing up an RSS reader, it’s good enough for me. Now to start updating my Puppetfile. Enjoy!

Updating Windows Puppet Enterprise agent versions with puppet_agent

Yesterday, I tackled upgrading Linux Puppet Enterprise agents with puppet_agent and said I would give Windows agents a try next. It’s simultaneously extremely easy and really difficult to accomplish this! On the manifest side, it’s as simple as including the class in a profile for Windows agents. On the rspec side, we need to add a lot of custom facts and even mock up some functions. I also encountered an issue with version 1.3.1 that broke my rspec tests. Version 1.3.2 was released in February 2017 to address that.

As usual, let’s start with the rspec test updates. Here’s the starting tests and manifest for an empty profile::base::windows, where nothing is really managed in the base profile yet.

Continue reading

Updating Linux Puppet Enterprise agent versions with puppet_agent

Happy New Year, everyone! I know that when I last blogged before the holiday, I was getting settled in with Jenkins, but I really needed to relax during the break so I had to hold off on that. I promise, I’ll get back to that soon, but in the meantime I encountered an issue that needed fixed and required a little more than the docs provided.

I ran into an issue recently with Puppet Enterprise agents that weren’t able to purge some resource types properly due to the use of Anchors. This is an issue with Puppet, not the module in question, and it was fixed in Puppet 4.4.0 but affected nodes were running older versions. Of course, rather than upgrade the agents by hand, I decided to automate it. Puppet has an Upgrading PE agents: *nix page that describes how to do this in a few ways. The latter options require manual effort, but the first option, via the module puppet_agent, can do this automatically for us. I diverged from the instructions a bit because I do my classifying with hiera rather than the PE Classifier and because there’s a missing step! (I’m trying to get that document updated, as well)

The guide says to install the module on the master and add some classification. To do this in a traditional roles and profiles setup using hiera as the classifier and with a monolithic controlrepo, we need to add the module to Puppetfile and .fixtures.yml, then add either a profile for puppet_agent or add it to a base profile. I chose the latter. Let’s add the module first:

Continue reading

Scheduling regular Travis CI builds with Cron Jobs

If you have a project of any complexity that is using Travis CI for testing, you’ve inevitably run into an issue where you make a minor change – maybe even just to a markdown file – and the tests that were passing before completely bomb. If this hasn’t happened to you, just wait, it will! This typically happens because some dependency, such as a newer rubygem you depend on indirectly, is no longer compatible with your test environment. As an example, I updated my .travis.yml in a puppet module project, opened PR16, and suddenly all the previously-green tests went red. The error NoMethodError: undefined method 'last_comment' for #<Rake::Application:0x000000015849f8> was observed because the dependent gem rake was updated from v11 to v12, conflicting with the pinned version of rspec. Until this problem is fixed, every PR, no matter how simple or complex, is going to fail its test.

This puts a lot of burden on both the person submitting the PR, who just wants their simple change to be approved, and the project maintainer, who wants their build status to work. Neither is satisfied until the problem is tracked down and remediated, often in a separate PR. By that time, the contributor may not have the interest to update their PR and the maintainer has to decide if they want to fix it up or discard it. No-one is happy. All because some upstream dependency changed and we weren’t aware of it until the moment a PR was submitted, even though it may have happened days or months ago.

Continue reading

Installing Jenkins and RVM

Update: Contrary to the module readme as of 12/1/2016, this WILL result in a VM running Jenkins 2, rather than version 1.

It’s time to get started installing Jenkins for use with Puppet. I’ll be installing this in my home lab on a vm called jenkins (so inventive, I know) as an all-in-one server for simplicity. I don’t know Jenkins real well so I don’t know what I should be doing for masters and build servers, but my home lab needs are small anyway. I gave it 2 cores and 1 GB of RAM, which I think is on the low end for Jenkins. My initial exploration led me to believe I’ll need to install Jenkins as well as RVM, since my base OS image is CentOS 7 (Ruby 2.0.0) and the current Puppet 4 AIO build is against Ruby 2.1. I’m using Puppet and the rtyler/jenkins module to set up Jenkins, since I don’t know anything about its innards. Down below, you’ll see that I installed RVM by hand, after which it occurred to me that there’s a perfectly cromulent maestrodev/rvm module that I’ve used in the past – something I may change out later, but I already wrote my manifest so I’m going to share it anyway!

I used Jenkins jobs to experiment a lot with this setup, such as enabling RVM on a job and seeing what errors showed up in the log, but I’m going to hold off on showing that magic until the next article. I will still explain where portions of the manifest came from, just without the actual errors.

Before we go any further, make sure you have the jenkins module and its dependencies added to your Puppetfile, .fixtures.yml, and wherever else you might need to track it.

Continue reading

Getting started with Jenkins and Puppet

If you’ve followed my blog, you’ve seen that I enjoy using Travis CI to run my puppet rspec tests on the controlrepo or against component modules. When you create a PR, Travis starts a build and adds some links to the PR notes to the builds. When it’s complete, the PR is updated to let you know whether the build was successful or not. You can also configure your repo to prevent merges unless the build succeeds. The best part is that it “just works” – you never have to worry about upgrading Travis or patching it or really anything other than making sure you enabled it. It’s a pretty awesome system, especially since it is absolutely free for open source projects. I do love Travis!

But, Travis isn’t always available and isn’t always the answer. Travis only runs against PRs or (typically) merges into the default branch. It won’t run on a schedule, even if your code never changed, which can help test any dynamic dependencies you might have (whether you should have any is a different topic!). It runs on a limited subset of OSes that Travis supports. You have to use Github and public repos to hit the free tier – if your controlrepo is private, there’s a different travis site to use and there is no free plan, though there is a limited number of trial builds to get you started. After that, it can be pretty pricey, starting at 1 concurrent builds for $69/month. This is great for a business, but most of us can’t afford $840 a year for the home network. It’s also, by itself, somewhat limited in how you can integrate it. It can be part of a pipeline, but it just receives a change and sends a status back to Github, it won’t notify another system itself. You have to build that yourself.

There are a ton of other Continuous Integration and Continuous Delivery systems out there, though. They can be cheaper, have better integrations, run on your own hardware/cloud, and don’t have to require Github. Among the myriad options available, I chose to look at Jenkins, with the rtyler/jenkins puppet module. I can run this on my own VM in my home lab (hardware/cloud), it can integrate with GitHub private repos ($0) or BitBucket (doesn’t require Github). CloudBees also discussed a new Puppet Enterprise Pipeline plugin at PuppetConf 2016, which is really appealing to me (better integrations). I can also have it run builds on a schedule, so if dependencies change out from underneath me, I’m alerted when that happens, not when I run my first test a month after the underlying change.

I’m very new to Jenkins and I’m going to do a lot of things wrong while testing it, but I’m going to start blogging about my experiences as I go, right or wrong, and then try and do a wrapup article once I learn more and have a fully working solution. I’ve been inspired to do this by Julia Evans and her wonderful technical articles that are more about the exploration of technology. That goes against my normal style of figuring it all out first, but I so love to see the exploration of others and hope you will, too! As we go on this journey, please feel free to educate or correct me in the comments and on twitter, as always.

Puppet Tech Debt: Moving Rspec Tests

Now that we have shaved the yak of generate-puppetfile, it’s time to move my rspec tests to the top of the controlrepo, as discussed on Thursday. To do so, we need to move not just the spec tests, but also the files Rakefile, .rspec, .rubocop.yml, and create a facsimile metadata.json, and of course, generate a new .fixtures.yml.

Moving the files is pretty simple. In my controlrepo, I only have a profile class with tests, so I’m going to take those as-is. We do want to make sure that we start with a clean environment, we don’t want to copy a bunch of fixtures or other temp files. We could use git clean -ffdxn; if you don’t want to redownload your gems you can manually clean up the relevant directories. Then we can do the file shuffle:

$ git clean -ffdxn
Would remove .bundle/
Would remove Gemfile.lock
Would remove coverage/
Would remove dist/profile/spec/fixtures/manifests/
Would remove dist/profile/spec/fixtures/modules/
Would remove vendor/
$ rm -fR dist/profile/spec/fixtures/manifests/ dist/profile/spec/fixtures/modules/
$ git mv dist/profile/spec ./
$ git mv dist/profile/{Rakefile,.rspec,.rubocop.yml} ./

Continue reading

Release 0.10.0 of generate-puppetfile

As I discussed on Thursday, I am looking to re-architect the layout of my controlrepo’s rspec tests. Of course, there were yaks to shave, first. To that end, I’ve released version 0.10.0 of generate-puppetfile (rubygem, github project) with the added ability to run generate-puppetfile --create-fixtures at the top of your controlrepo and generate a .fixtures.yml that will set up symlinks for all of the modules contained in your controlrepo. This functionality is based off the existence of the file environment.conf in the current directory and it containing a modulepath stanza. All the paths in the modulepath are explored and symlink-type fixtures created for each module found. Previous functionality when run inside a module directory is still preserved, of course.

Because the project has not reached version 1.0.0 yet, I have renamed the option --fixtures to --create-fixtures. This is in preparation for a feature request for an --only-fixtures feature request.

Let’s look at how to use this new feature. Before we start, we must be at the top level of the controlrepo. Next, we need environment.conf to have a modulepath stanza. In my controlrepo, the file looks like this:

Continue reading