Recently, we looked at converting a module to be hiera friendly. Another task you may have to look forward to is tracking configuration files for different hosts providing the same service. You could have a config for each node, network, environment, etc., all of which need updated if some common element changes. Or, you could use a Puppet Template to have a single template that is populated by node-specific elements from node facts and puppet variables. Each node would receive a personalized copy and any common element changes would be reflected across the board immediately.
As an example, I run some mediawiki servers at work. Each one points to a different database but is otherwise very similar. The search engine is SphinxSearch and it relies on the Sphinx config file /etc/sphinx/sphinx.conf. The config includes the database connection information, which varies from device to device, and a number of other settings standardized across the wikis (minimum search term length, wildcards, and other search settings). Keeping the database connection information accurate across three wikis would normally require three config files. Let’s simplify that with a template.
Puppet templates are written in ERB, a templating language that is part of the Ruby standard library. ERB commands are interpolated into the config where needed and puppet feeds facts and variables to ERB, which determines what values to populate the config with. We have a few good sources of information on the templates: the Ruby docs, a Puppet Labs Using Puppet Templates article, or the Learning Puppet chapter on Templates. I’ll be picking out some highlights, reference them as needed as we work on our template.
As we near the end of my schedule puppet content, I’ve asked the Twitterverse for any other topics people want to see discussed. Jason Shiplett asked a great question: Why Puppet?
This is essentially a two-fold question. First, you must understand what Configuration Management (CM) is and why you need it. Second, of all the CM tools out there, why would you choose Puppet?
In spite of my telling Jason that the world doesn’t need another “Why CM?” post, here we go 🙂
Plenty of other people have done a great job explaining what Configuration Management is and why you need it. Chief among these is Information Technology Infrastructure Library, or ITIL, a framework for IT Service Management. In the Service Transition volume, Configuration Management is described. We can simplify the meaning to describing and managing the state of a configuration through a service’s lifecycle.
For the past few weeks, we have been working on packaging our own software and deploying it with puppet. Before that, we touched on refactoring modules to use hiera. In fact, I grandiosely referred to it as Hiera, R10K, and the end of manifests as we know them. I included a very simple example of how to refactor a per-node manifest into the role/profile pattern and use hiera to assign it to the node. Today, we’ll look at more features of hiera and how you would refactor an existing class to use hiera.
In a legacy implementation of puppet, you’ll likely find plenty of existing modules whose classes have static assignment or lots of conditionals to determine the necessary values to be applied. Even in a greenfield implementation of puppet, you may find yourself writing straight Puppet DSL for your classes before refactoring them to use hiera. Figuring out how to refactor efficiently isn’t always obvious.
First, let’s take a look at Gary Larizza’s When to Hiera and The Problem with Separating Data from Puppet Code articles. Gary covers the when and why much better than I could, so please, go read his article and then come back here. Gary also covers the common pre-hiera pattern and a few patterns that can be used when refactoring to hiera. There is another pattern that is documented indirectly by Gary (under Hiera data bindings in Puppet 3.x.x) and in the Hiera Complete Example at docs.puppetlab.com. I’m going to explain this pattern and document this directly, adding another Hiera pattern to Gary’s list.
Having created one or more build servers, the next logical step is to start building software. We touched on this briefly a few weeks ago, and with a proper development station, it’s time to expand on it.
If you’re a developer by trade, you can probably skim or skip this article. Remember, this series is aimed at vSphere Admins, not devs. I’d certainly appreciate your insights in the comments or on twitter, though!
Modifying software build processes for FPM
We’ve used FPM in the past to take a directory and turn it into a package. This works very well when /some/long/path belongs entirely to your application. What if your application drops a binary in /bin, a manpage in /usr/share/man/man5, a config file in /etc, or even just a few files in a directory that’s shared with other packages? Let’s take a look at an extension for mediawiki. This is very simple, we have a legacy Makefile and two useful targets, dev and prod:
The Puppet series so far has really focused on VM builds and just started to touch on software packaging. We need an appropriate place to do this work, and what better way to set that than via Puppet itself? Today, we’ll create some roles and profiles for a build server, which could be permanent and share amongst developers, spun up as needed for the team, or spun up per developer.
Build Profile and Role
The last few examples we have done with FPM were on our “production” servers. That’s less than ideal for a few reasons. You wouldn’t want to mess up the publicly available service while packaging, whether by overwriting a file, exhausting resources, or the brief outage when services restart. It is not a good idea to add compilers and development libraries to any server unnecessarily as it increases the attack surface (additional security risks, additional packages to patch, additional items for auditors to flag, etc). You also probably do not want your build servers in the same environment as your production servers (unless, as is the case in these examples, your “production” environment is your lab – so just pretend it’s a different environment). Let’s assume that we do not have a good reason to violate these best practices, so our goal is set up a dedicated build server. It will require all the software we have been using so far, and we will throw on a local user. If you have LDAP or another directory service in your lab, I would suggest using it, but this is a good example as sometimes the build network is restricted.
We have two profiles to create, then. The first is the local users. We’ll call this class ::profile::build_users, in case we create another grouping of users later. The second profile is for our build software, and we will call it ::profile::build. Here are the two class files, located at profile/manifests/users/build.pp and profile/manifests/build.pp, respectively.