Puppet rspec tests with Hiera data

Editor’s note: Please check out the much newer article Configuring Travis CI on a Puppet Module Repo for the new “best practices” around setting up rspec-puppet. You are encouraged to use the newer setup, though everything on this page will still work!

I’ve covered puppet unit tests with rspec and beyond before. What if you need to go even further and test data from hiera? There’s a way to do that with rspec as well, and it only requires a few extra lines to your spec config – plus the hiera data, of course.

Use hiera in your class

We’ve covered a number of ways to use hiera in your class. You can use hiera lookups (hiera(), hiera_hash(), etc.) and automatic parameter lookups with classes. We’ll look specifically at hiera_hash() and the create_resources() it is commonly paired with. You cannot simply test that create_resources() was called because you need to know the resulting title of the generated resource. Here’s a simple DHCP profile class I created:

class profile::dhcp {
  # DHCP service and host reservations
  include dhcp::server
  $dhcp_server_subnets = hiera_hash('dhcp_server_subnets', undef)
  if ($dhcp_server_subnets) {
    create_resources('dhcp::server::subnet', $dhcp_server_subnets)
  }

  $dhcp_server_hosts = hiera_hash('dhcp_server_hosts', undef)
  if ($dhcp_server_hosts) {
    create_resources('dhcp::server::host', $dhcp_server_hosts)
  }
}

This is based on ajjahn/dhcp which will use two resources, dhcp::server::subnet and dhcp::server::host to manage the dhcp scopes and reservations, respectively. I don’t want to encode the subnets and hosts here, that’s the whole point of using hiera to separate the data and code, so the class looks up the data from hiera and uses create_resources() to generate whatever’s there. Of course, if nothing is there, the default value of ‘undef’ is passed and nothing happens. Here’s what some hiera data may look like for these two resource types. Keep in mind, this is sample data and isn’t intended to actually work:

dhcp_server_subnets:
  '10.0.0.0':
    broadcast   : '10.0.0.255'
    netmask     : '255.255.255.0'
    routers     : '10.0.0.1'
    range_begin : '10.0.0.100'
    range_end   : '10.0.0.150'
    dns_servers :
      - '10.0.0.1'
dhcp_server_hosts:
  sample:
    address:   10.0.0.254
    hwaddress: 00:00:00:00:00:0a

The problem is that our hiera data only exists on the puppet master. You should be using a build server or your local host to develop and test your modules. You won’t be able to see the real hiera data from here – and you may not want to, as the real data could contain sensitive information, like passwords. If you are encouraging other people to work on your modules, you definitely don’t want that information embedded.

Creating the tests

As always, the first step is to create some tests that will fail. Let’s test for a valid dhcp::server::subnet and dhcp::server::host, as well as our other simple tests:

$ cat spec/classes/dhcp_spec.rb
require 'spec_helper'
describe 'profile::dhcp', :type => :class do
  let :facts do
  {
    :id                     => 'root',
    :kernel                 => 'Linux',
    :osfamily               => 'RedHat',
    :operatingsystem        => 'RedHat',
    :operatingsystemrelease => '6',
    :concat_basedir         => '/dne',
    :path                   => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
  }
  end

  context 'with defaults for all parameters' do
    it { is_expected.to create_class('profile::dhcp') }
    it { is_expected.to contain_package('dhcp') }
    it { is_expected.to contain_dhcp__server__subnet('10.0.0.0') }
    it { is_expected.to contain_dhcp__server__host('sample') }
  end
end

You’ll notice that we replace colons with underscores in the names of our created resources. Make sure your .fixtures.yml file includes ajjahn/dhcp and run rake spec.

Failures:

  1) profile::dhcp with defaults for all parameters should contain Dhcp::Server::Subnet[10.0.0.0]
     Failure/Error: it { is_expected.to contain_dhcp__server__subnet('10.0.0.0') }
       expected that the catalogue would contain Dhcp::Server::Subnet[10.0.0.0]
     # ./spec/classes/dhcp_spec.rb:18:in `block (3 levels) in <top (required)>'

  2) profile::dhcp with defaults for all parameters should contain Dhcp::Server::Host[sample]
     Failure/Error: it { is_expected.to contain_dhcp__server__host('sample') }
       expected that the catalogue would contain Dhcp::Server::Host[sample]
     # ./spec/classes/dhcp_spec.rb:19:in `block (3 levels) in <top (required)>'

Great, our tests “work”, in that they fail when they should. Now, let’s work on getting them to pass.

Add hiera_config to your spec_helper.rb

The rspec setup page describes (all the way at the bottom) the variable hiera_config that can be used to point to a hiera.yaml file. We need to add that to the rspec config in our spec/spec_helper.rb file. You don’t simply set the variable, you access the Rspec object’s configuration and join the paths together. Here’s an example of how this looks:

require 'puppetlabs_spec_helper/module_spec_helper'

RSpec.configure do |c|
  c.hiera_config = File.expand_path(File.join(__FILE__, '../fixtures/hiera.yaml'))
end

The __FILE__ will be one of the spec/classes/*rb files (or defines, or functions…), hence the ../ at the beginning of the path. This will result in rspec using spec/fixtures/hiera.yaml under the root of our module for it’s hiera configuration. Here’s the very simple file contents (make sure you get the indentation correct):

---
:backends:
  - yaml
:yaml:
  :datadir: './spec/fixtures/hieradata'
:hierarchy:
  - '%{::clientcert}'
  - 'default'

Unless you want to test node-specific hiera data, which would go in spec/fixtures/hieradata/%{::clientcert}.yaml, you can jam your example content into spec/fixtures/hieradata/default.yaml. This is what that file looks like:

---
dhcp_server_subnets:
  '10.0.0.0':
    broadcast   : '10.0.0.255'
    netmask     : '255.255.255.0'
    routers     : '10.0.0.1'
    range_begin : '10.0.0.100'
    range_end   : '10.0.0.150'
    dns_servers :
      - '10.0.0.1'
dhcp_server_hosts:
  sample:
    address:   10.0.0.254
    hwaddress: 00:00:00:00:00:0a

Now run rake spec (or rake spec_standalone) and you will see your tests start passing:

[rnelson0@build profile]$ rake spec_standalone
Finished in 4.59 seconds (files took 1.08 seconds to load)
13 examples, 0 failures

This matches our sample data from earlier, but it doesn’t have to. If the hiera data provided a user and password, you could reduce those to ‘u’ and ‘p’ rather than provide an actual complex password. All you need to do is ensure that the values provided won’t break create_resources(). If you mess up the indentation on broadcast so it’s at the top level, leaving 10.0.0.0 as an empty hash, you receive errors about it being null:

  4) profile::dhcp with defaults for all parameters
     Failure/Error: it { is_expected.to create_class('profile::dhcp') }
     Puppet::Error:
       undefined method `empty?' for nil:NilClass at /home/rnelson0/puppet/profi
le/spec/fixtures/modules/profile/manifests/dhcp.pp:16 on node build.nelson.va

Pesky .gitignore

We have now added some files to spec/fixtures which is in our .gitignore file. We still do not care about any files in that directory, except our hiera configuration and data, so add those files manually. For any files that .gitignore wants to ignore, you’ll get a warning that you must force git to take the files with the -f flag. Do so now.

[rnelson0@build profile]$ git add spec/fixtures/hieradata/default.yaml
The following paths are ignored by one of your .gitignore files:
spec/fixtures
Use -f if you really want to add them.
fatal: no files added
[rnelson0@build profile]$ git add spec/fixtures/hieradata/default.yaml -f

Once you start tracking these files, and modifications will be tracked as well, so you won’t need to force git to add them to future commits. Commit and push all your changes

Summary

Today we created some tests for a class that relies on hiera, created some sample hiera data, and configured rspec to look at the hiera data. Afterward, we made sure all our changes were commited, in some cases by forcing git to start tracking files, and pushed them upstream so they are preserved. In the future, you can simply add to your hiera data and continue writing tests.

9 thoughts on “Puppet rspec tests with Hiera data

  1. Pingback: Improved r10k deployment patterns | rnelson0
  2. Can’t thank you enough for this. I’m just starting to learn puppet module rspec testing, and we use create_resources heavily, I wouldn’t have been able to generate working tests without this!

  3. Pingback: Modern rspec-puppet practices | rnelson0
  4. I have tried the above sample rspec-puppet for my testing but still getting “Failure/Error: it { should contain_ld_pcd(‘fs_pcd’) }
    expected that the catalogue would contain ld_pcd[fs_pcd]” below is my spec_helper.rb

    require ‘puppetlabs_spec_helper/module_spec_helper’

    RSpec.configure do |c|
    c.hiera_config = File.expand_path(File.join(__FILE__, ‘../fixtures/hiera.yaml’))
    end

    • This spec helper and the resulting test result look correct. The manifest you are testing should contain a line similar to “ld_pcd{‘fs_pcd’: }” somewhere in it or any classes it includes, but it is not found. Either the test is looking for the wrong object type or title, or the manifest is not providing that object type or title.

      • My actual manifests is

        $a = hiera_hash(‘ld_env_data’)
        create_resources(‘ld::pcd’, $a)

        ld_env_data
        fs_pcd:
        lv_size340mb
        owner:root

        so below is the line for rspec-puppet test case

        it { should contain_ld_pcd(‘fs_pcd’) }

        And ld modules contains

        define ld::pcd()
        {

        ….
        }

  5. I think your hiera data is not correct, but it could be formatting in the comments. Check out this link for the proper indentation: https://gist.github.com/rnelson0/3643d1910da2c5e9a2f8

    If your data is not formatted properly, $a will either be empty or malformed. When empty, the create_resources() call will do nothing and rspec-puppet will chug along and not find the resource you expect (and probably throw a warning, if you were using puppet agent or apply). When malformed, it would probably cause compilation to fail and all your rspec tests would bomb out very early with a note about invalid params on the resource or something similar.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s