Last week, I discussed how to upgrade your Linux and Windows Puppet Enterprise agents with puppet_agent. Today, we will look at two different ways to upgrade your Puppet OpenSource agents, first with puppet_agent and then with jlambert121/puppet. There are many other modules that you can use to manage puppet, some of which offer similar functionality that you may be able to apply some of these lessons to.

Why are we looking at two different modules today? I use jlambert121’s module to manage my puppetserver already. Unfortunately, it is puppet 4 only and I still had some puppet 3 agents hanging around according to puppetboard. I also have some other modules that are puppet 4 only, like the latest version of puppetlabs/ntp. I could use jlambert121/puppet to upgrade my agents that are already on puppet 4, but need something to help get my puppet 3 agents over the hump – that’s where puppet_agent comes in. We will start with that.

Puppet 3 to Puppet 4 with puppet_agent

I’ve documented a lot of detail about puppet_agent in the Linux and Windows articles references above, so I’m going to skim over the setup of that module real quick. Add it to your Puppetfile/.fixtures.yml, add it to profile::base, make a spec test, and you’re mostly set:

# Puppetfile
mod 'puppetlabs/puppet_agent', '1.3.1'
mod 'puppetlabs/transition', '0.1.0'

# .fixtures.yml
    puppet_agent:
      repo: "puppetlabs/puppet_agent"
      ref: "1.3.1"
    transition:
      repo: "puppetlabs/transition"
      ref: "0.1.0"

# dist/profile/manifests/base.pp
class profile::base {
  # existing profile

  # Manage the puppet agent on nodes
  include puppet_agent
}

# spec/classes/base_spec.rb
# ... existing tests
      it { is_expected.to contain_class('puppet_agent') } 
    end
  end
end

I mentioned that I use jlambert121/puppet and the latest ntp. A feature of the first and the entirety of the second is incompatible with the simultaneous combination of puppet 3 and puppet_agent. To get around this, I downgraded ntp and disabled puppet agent management. Finally, we want to provide a package version to puppet_agent. You can make this the latest, 1.8.2 (or latest if you want to go all YOLO); I specifically chose an earlier version so we still have something to do in part 2. As this is done in a feature branch called puppet_agent, it does not affect running systems at all. Here are the additional tweaks:

# Puppetfile
mod 'puppetlabs/ntp', '4.2.0' 

# .fixtures.yml
    puppet_agent:
      repo: "puppetlabs/puppet_agent"
      ref: "1.3.1"
    transition:
      repo: "puppetlabs/transition"
      ref: "0.1.0"

# hiera/global.yaml
puppet::agent: false 
puppet_agent::package_version: '1.8.0'

Commit all this and publish your feature branch puppet_agent. Direct your puppet 3 agents to that branch for a single run and then reboot them if you can. puppet_agent does an excellent job of upgrading the agent but once in a while leaves the existing version 3 agent in memory running and a reboot is quicker and cleaner than manually cleaning that up. If a node is really vital and you can’t reboot it, I recommend either waiting until you can reboot it to do the upgrade, or make sure you killed ALL the existing puppet processes before restarting the agent. When an upgraded agent checks in to the master using the cached version 3 code but the on-disk version 4 configuration, things get wonky.

If you’re doing the upgrade run by hand, and you got everything right, this will work:

# puppet agent -t --environment puppet_agent; reboot

Repeat this process with all of your puppet 3 agents. Now you have a choice. You can bring puppetlabs/ntp back to the latest version and then merge that branch, continuing to use puppet_agent for ongoing updates just fine. If you’re using jlambert121/puppet or a module with similar functionality, you can also delete the puppet_agent branch and perform upgrades using the puppet module. Before you make your decision, let’s look at what that entails.

Puppet 4 Upgrades with jlambert121/puppet

This module allows you to specify a puppet-agent version to install. It defaults to latest, which is fine for my home lab but probably not what you want in a production environment. All you need to do is make sure that your desired nodes receive the puppet class in their catalog and that puppet::agent is true – the default value. Since I am managing my puppet master with this module, it’s already getting the latest version of the puppet agent via profile::puppet_master. This is not in my base profile, however, so I need to add it there and update a spec test (this assumes you are already using the module, of course):

# dist/profile/manifests/base.pp
class profile::base {
  # existing class
  include puppet
}

# spec/classes/base_spec.rb
it { is_expected.to contain_class('puppet') }

That is literally all you need! Of course, you may want to include the class in a different profile and bundle it with roles, set the agent version in hiera with puppet::puppet_version: '1.8.2-1' (it doesn’t accept a nice semver number like puppet_agent, it needs to match the package version string for your distro(s)), and maybe use some other tweaks in hiera to disable agent management on certain nodes (puppet::agent: false). You can merge this into your production environment and when a new version of puppet comes out, just update puppet::puppet_version and wait an hour and all your agents will be updated. There’s generally no need to restart nodes, as puppet 4 to puppet 4 updates usually don’t result in a cached version of puppet running.

There’s one last little tweak I performed. jlambert121/puppet configures the environment the agent will use in puppet.conf. When puppet::env is not set, it dynamically chooses the current environment. When you test an agent against a feature branch like puppet agent -t --environment feature_branch, that node will now be configured to perform all future runs against the environment feature_branch. That can be useful in some situations, but probably isn’t what you want most of the time. If you merge that branch into production and delete the branch, agents that were checking into it will error out until you run puppet agent -t --environment production on them, or otherwise get puppet.conf back to the right settings. A simple way around this is to specify production as the environment in hiera:

# hiera/global.yaml
puppet::env: production

Now, puppet agent -t --environment feature_branch won’t change the long-term environment agents use. And of course, hiera gives you the flexibility to change that to some other environment when you do want to change it for one or more agents.

A lot of other modules for managing puppet have similar options. You don’t need to convert to the module I’m using to get these benefits, you’ll just have to do a little research to figure out how your chosen module provides the functionality. Please let me know in the comments if you have a favorite module you use and how to perform upgrades with it!

Summary

Between puppetlabs/puppet_agent and jlambert121/puppet you have a few ways to upgrade your agents. You can use whichever one you decide is easier, or use both in sequence, as I did. Automating your agent upgrades helps you stay up to date so you have one less yak to shave when you see new puppet features announced. Take care of your fleet!