Editor’s note: Please check out the much newer article Configuring Travis CI on a Puppet Module Repo for the new “best practices” around setting up rspec-puppet. You are encouraged to use the newer setup, though everything on this page will still work!

Over the course of the Puppet series, one thing I’ve ignored is testing. As vSphere admins, many of us are comfortable with programming but probably not as well versed in some practices as full-time developers. Today we’ll look at an introduction to some test-driven-development with puppet.

Test Driven Development

What is this Test Driven Development, or TDD, that everyone speaks so highly of? In essence, you write tests that fail before you write any code, then you write code to satisfy the tests. Each test typically looks at a specific unit of functionality of a program, such as whether a file is created or has contents, and are called “unit tests.” By testing a specific function, when you have a failure, you can typically narrow down the problem domain to a few lines of code. When all unit tests generate successes, your code works (in theory!). In addition, when you modify the code in the future, these unit tests help ensure that you haven’t broken something that was previously working, also known as a “regression.”

TDD depends, of course, on writing tests that both provide coverage of all your code and that map to the requirements of the program. If you forget to provide a test that covers a vital portion of your code, all your tests can be successful but leave you with a broken program. If you have not been practicing TDD on an existing program, you can still add tests. However, you will not have 100% test coverage (the percent of code that is covered by unit tests) initially, or possibly ever, as all of the existing code was written prior to the unit tests. To keep things simple today, we’ll start writing some new code.

Rspec-puppet

The tool we will use today is called rspec-puppet. Rspec is a testing tool for Ruby and rspec-puppet is an implementation created specifically to test puppet modules. Before we work on the module, we need to install rspec-puppet. Using puppet, this is very simple. Add these statements to your build profile or use it in a puppet apply. Make sure to apply it to your build server before continuing.

  package {'rspec-puppet':
    ensure   => present,
    provider => gem,
  }

You can find some more documentation and examples of using rspec-puppet on its github page. Rspec tests are generally of the format it { should test(‘something’) }, a very human-readable testing framework. Let’s create a module that we want to test.

Certs module

I have a particular need for a module to easily create two SSL-related files for webservers based on existing content. This module is called certs, and this article will step through generating it fully. If you have your own need for a module, go ahead and create it now, but you will have to map the article to your module on your own. Here’s the command to generate a module called certs by rnelson0 in the current path. You can hit enter to all the questions it asks.

puppet module generate rnelson0-certs --modulepath=./

Cd into certs. The framework for a puppet module already exists. Run git init and add an upstream with git remote add origin <git url>. Before we go any further, let’s look at what our module needs to provide – remember, we start with tests, not code! Here’s part of an instance of an apache::vhost used by one of my profile classes:

  apache::vhost {$cname:
    ...
    ssl_cert    => "/etc/ssl/certs/${cname}.crt",
    ssl_key     => "/etc/ssl/certs/${cname}.key",
    ...
  }

The cert and key files must be present on the local system. You can achieve this with a File resource for the crt that comes before a File resource for the key which comes before the apache::vhost (file -> file -> apache::vhost). That’s great when you have one profile that uses apache::vhost, but not so awesome when you have multiple such profiles, it’s a lot of repeated code which means a lot of room for errors. The goal is to simplify the code. Here’s how I envision my module working:

  certs::vhost {$cname: } ->
  apache::vhost {$cname:
    ...
    ssl_cert    => "/etc/ssl/certs/${cname}.crt",
    ssl_key     => "/etc/ssl/certs/${cname}.key",
    ...
  }

The module can put the files where we want them. Hrm, what if someone wants the modules someplace other than /etc/ssl/certs? We should probably allow that with a target_path parameter. We’ll also need to let the caller specify where the files are coming from. The files will have to come from somewhere else – we can’t add them to the certs module if we want to share the module with others – perhaps another puppet module or an NFS mount. We’ll call that parameter source_path and we’ll suggest a puppet module called othermodule (maybe site_certificates or domainX_certificates). Here’s what the modified statements might look like:

  certs::vhost {$cname:
    source_path => 'puppet:///othermodule',
    target_path => '/etc/ssl/certs',
  } ->
  apache::vhost {$cname:
    ...
    ssl_cert    => "/etc/ssl/certs/${cname}.crt",
    ssl_key     => "/etc/ssl/certs/${cname}.key",
    ...
  }

We’re not worried about how it happens yet, just defining what the module needs to do. You can’t test something unless you know what it is.

Creating a unit test

It’s almost time to create a unit test. If we look at the rspec-puppet home page, we’ll see that we need to run rspec-puppet-init. You should see something like this:

[rnelson0@build certs]$ rspec-puppet-init
 + spec/defines/
 + spec/functions/
 + spec/hosts/
 + spec/fixtures/
 + spec/fixtures/manifests/
 + spec/fixtures/modules/
 + spec/fixtures/modules/certs/
 + spec/fixtures/manifests/site.pp
 + spec/fixtures/modules/certs/manifests
!! spec/spec_helper.rb already exists and differs from template
!! Rakefile already exists and differs from template

Whoops, there are two errors. I haven’t explored Rakefile yet, but spec/spec_helper.rb will definitely cause us problems. Remove the file and re-run rspec-puppet-init and you’ll see a much different file:

[rnelson0@build certs]$ cat spec/spec_helper.rb
require 'puppetlabs_spec_helper/module_spec_helper'
[rnelson0@build certs]$ rm spec/spec_helper.rb
[rnelson0@build certs]$ rspec-puppet-init
 + spec/spec_helper.rb
!! Rakefile already exists and differs from template
[rnelson0@build certs]$ cat spec/spec_helper.rb
require 'rspec-puppet'

fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures'))

RSpec.configure do |c|
  c.module_path = File.join(fixture_path, 'modules')
  c.manifest_dir = File.join(fixture_path, 'manifests')
end

You can also follow a manual process to set up the spec files, but this gets you there a lot faster. You can now type rspec at the root of your module and the default test – that include <modulename> results in including the module in a catalog – will succeed.  There are a TON of ‘deprecated’ warnings that will show up. PUP-3594 explains where the messages come from, a workaround, and where possible fixes may come from. I’m just going to ignore the warnings until there is a permanent fix, as it doesn’t obscure the last line, which is what we really care about:

[rnelson0@build certs]$ rspec
.

Deprecation Warnings:

Filtering by an `:example_group` subhash is deprecated. Use the subhash to filter directly instead. Called from /usr/lib/ruby/gems/1.8/gems/rspec-puppet-1.0.1/lib/rspec-puppet/example.rb:12.
Filtering by an `:example_group` subhash is deprecated. Use the subhash to filter directly instead. Called from /usr/lib/ruby/gems/1.8/gems/rspec-puppet-1.0.1/lib/rspec-puppet/example.rb:16.
Filtering by an `:example_group` subhash is deprecated. Use the subhash to filter directly instead. Called from /usr/lib/ruby/gems/1.8/gems/rspec-puppet-1.0.1/lib/rspec-puppet/example.rb:20.
Too many uses of deprecated 'Filtering by an `:example_group` subhash'. Pass `--deprecation-out` or set `config.deprecation_stream` to a file for full output.

--------------------------------------------------------------------------------
RSpec::Puppet::ManifestMatchers::CreateGeneric implements a legacy RSpec matcher
protocol. For the current protocol you should expose the failure messages
via the `failure_message` and `failure_message_when_negated` methods.
(Used from /home/rnelson0/git/certs/spec/classes/init_spec.rb:5)
--------------------------------------------------------------------------------


If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.

5 deprecation warnings total

Finished in 0.03549 seconds (files took 0.38404 seconds to load)
1 example, 0 failures

With no failures, we know that we have working code according to our tests. That’s great, except we haven’t really defined a test. So, let’s look at creating one. When we call certs::vhost with the name “www.example.com” and a source_path of “puppet:///othermodule”, we expect it to create /etc/ssl/certs/www.example.com.{crt,key} files. That’s the first thing we want to test. The rspec-puppet tutorial will help us generate the test.

Certs::vhost is a define, so tests will go in spec/defines/<thing to be tested>_spec.rb, where the thing to be tested has the module namespace stripped off, unless it’s the top level class certs. Create the file now with a basic outline of a unit test:

require 'spec_helper'

describe 'certs::vhost' do
  # Your tests go in here
end

The next bits of code go between the describe and end lines, in the order they are introduced.

First, we define what values are passed to the define, namely the title of the define and the source_path. This is done with the use of let. The title can be set on its own, but source_path is part of the params hash. You’ll notice that tests can be read as English with only a few articles missing. We let the keys have certain values like this:

  let(:title) { 'www.example.com' }
  let(:params) { {
    :source_path => 'puppet:///othermodule'
  } }

It should contain two file resources with known path and source values. Everything relates to either puppet type definitions or those provided by other modules. We can create the tests using almost-English commands and methods:

  it do
    should contain_file('www.example.com.crt').with({
      :path   => '/etc/ssl/certs/www.example.com.crt',
      :source => 'puppet:///othermodule/www.example.com.crt'
    })
  end
  it do
    should contain_file('www.example.com.key').with({
      :path   => '/etc/ssl/certs/www.example.com.key',
      :source => 'puppet:///othermodule/www.example.com.key'
    })
  end

The could should be pretty explanatory. Keep track of your paren, curly brace, and do/end pairs. Save the spec file and run rspec from the top directory of the module:

[rnelson0@build certs]$ rspec
.FF

Failures:

  1) certs::vhost
     Failure/Error: })
     Puppet::Error:
       Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type certs::vhost at line 4 on node build.nelson.va
     # ./spec/defines/vhosts_spec.rb:13

  2) certs::vhost
     Failure/Error: })
     Puppet::Error:
       Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type certs::vhost at line 4 on node build.nelson.va
     # ./spec/defines/vhosts_spec.rb:19
...
3 examples, 2 failures

We expect this to fail, because we haven’t written any code for the actual module. It’s time to start on the module.

Certs::vhost definition

I’m going to skip over documentation of your module, except to say that you should have some, because it’s simply too lengthy for this article. You can see mine on the forge and plenty of modules, especially the Puppet Approved Modules, have much better documentation (and more rspec tests as well!).

Create manifests/vhost.pp and the skeleton of our define. Remember that we have already determined two parameters, one of which has a default value.

define certs::vhost (
  $source_path = undef,
  $target_path        = '/etc/ssl/certs',
) {

}

If we save our file and  run rspec now, we will see slightly more helpful failure messages:

  1) certs::vhost should contain File[www.example.com.crt] with source => "puppet:///othermodule/www.example.com.crt" and path => "/etc/ssl/certs/www.example.com.crt"
     Failure/Error: })
       expected that the catalogue would contain File[www.example.com.crt]
     # ./spec/defines/vhosts_spec.rb:13

  2) certs::vhost should contain File[www.example.com.key] with source => "puppet:///othermodule/www.example.com.key" and path => "/etc/ssl/certs/www.example.com.key"
     Failure/Error: })
       expected that the catalogue would contain File[www.example.com.key]
     # ./spec/defines/vhosts_spec.rb:19

By defining the tests first, we have described what the define is required to do. Specifically, create two File resources, “title.{crt,key}”, with source and path values of “source_path/title.{crt.key}” and “target_path/title.{crt,key}”, respectively. We are also testing that it’s a file, not a directory, so the ensure value should be “file”. That’s easy enough to define. Remember that the title attribute is also known as name within the define. If you use title, it can get confusing as each resource we use has its own title. Here’s the full content of manifests/vhost.pp:

define certs::vhost (
  $source_path = undef,
  $target_path        = '/etc/ssl/certs',
) {

  file {"$name.crt":
    ensure => file,
    source => "${source_path}/${name}.crt",
    path   => "${target_path}/${name}.crt",
  }
  file {"$name.key":
    ensure => file,
    source => "${source_path}/${name}.key",
    path   => "${target_path}/${name}.key",
  }
}

If we run rspec now, we have 3 successes and 0 failures. Congratulations, we’ve built a simple module using test-driven development!

Bonus Points

There are a lot more tests you should have for our simple module, both in rspec and in the module itself. You can see those tests at GitHub, but see if you can sleuth them out on your own before looking at the answers. I’m new to TDD as well, so I’m sure I missed a few tests. Drop me a comment, or open an issue/PR on GitHub, and let me know what you find!

Summary

We looked at Test-Driven Development today and how we can use it with puppet modules. We built a small module using TDD practices (tests first, then code!). The value of TDD may seem low at the moment, but as complexity rises, the value grows. We are also comfortable that if we tweak our module – even just to adjust formatting – we can run rspec and ensure we didn’t accidentally touch the syntax and break it. Tests can be added to our existing modules – including roles and profiles! – to start providing test coverage until our entire puppet codebase has full coverage. And perhaps most importantly, we can tell our developer coworkers that we understand TDD now!