Rspec trick: Getting at output you can’t see

I was having a problem yesterday with a specific rspec test that was failing for my puppet tool generate-puppetfile, and I couldn’t understand why. I was expecting an exitcode of 0 but was receiving 1, so obviously I had an issue, but I couldn’t see the accompanying error message. When I attempted to run the command myself, it succeeded, so I was pretty sure the issue was with either argument handling by my program or more likely, the way I wrote my rspec test (spoiler:  it was in my rspec!). Here’s what the rspec test looked like to start with:

  context 'when creating fixtures' do
    let :args do
        'rnelson0/certs'
        '--create-fixtures'
    end

    its(:exitstatus) { is_expected.to eq(0) }
    it 'should create .fixtures.yml' do
      File.exists? './.fixtures.yml'
    end
  end

Continue reading

Puppet Tech Debt Day 3, excluding OS testing

When using rspec-puppet-facts, there’s one minor limitation: it tests all the supported operating systems, even if a class is designed for a specific OS or family. You can easily skip the rspec-puppet-facts in a specific test, though it kind of defeats the purpose for your general purpose operating systems (if your OS type isn’t in facterdb – currently a certainty with network devices – you have to go around it anyway). But what if you want to keep using the facts and just exclude one or two compatible OSes? We were bandying this about on the Puppet slack yesterday and came up with a solution. Thanks to Daniel Schaaff for determining the syntax for this pattern!

Here’s what the per-OS portion of a spec test file looks like once you add rspec-puppet-fact:

on_supported_os.each do |os, facts|
  context "on #{os}" do
    let(:facts) do
      facts
    end

    it { is_expected.to contain_class('profile::unattendedupgrades')}
    it { is_expected.to contain_class('profile::linux::apt_source_list')}
  end
end

Obviously you don’t expect, or want, apt-related resources to apply to non-Debian OSes. We can filter that out using the osfamily fact. This lets us keep the on_supported_os.each pattern in our spec tests but preserve the functionality we want. Here’s what that looks like:

on_supported_os.each do |os, facts|
  context "on #{os}" do
    let(:facts) do
      facts
    end

    if facts[:osfamily] == 'Debian' then
      it { is_expected.to contain_class('profile::unattendedupgrades')}
      it { is_expected.to contain_class('profile::linux::apt_source_list')}
    end
  end
end

You can apply this wherever you want in your tests. If the class profile::unattendedupgrades were to apply to all OSes, move it out of the if block. You can also limit by kernel or whether or not selinux is enabled – or by a custom fact you generated.

Update: I came up with this pattern for my linux-only classes, to future proof against adding Windows as a supported OS:

describe 'profile::access_request', :type => :class do
  on_supported_os.each do |os, facts|
    next unless facts[:kernel] == 'Linux'

Puppet Tech Debt Day 2: Adjusting Rspec Tests

Yesterday was our 14th anniversary, so I didn’t have time to write a blog post, but I did look into a tech debt issue: rspec tests. In addition to adding rspec-puppet-facts, I found a program called onceover that offers two concepts I want to look into.

First, there is a rake task to generate fixtures. I have a similar feature in generate-puppetfile, but it’s not as perfect as I’d like – it often requires some manual touch afterward. Onceover’s rake task does not have that issue. I hope to be able to grab the rake task without being forced to use the rest of it or affecting the existing test set up. Maybe I’ll be interested in the rest of it someday, but not right now, and it’s great when you’re not forced to make a forklift upgrade like that.

The second item is centralizing rspec-puppet tests in the controlrepo rather than inside each module itself. That will change the relevant portions of my controlrepo layout from:

.
├── dist
│   ├── profile
│   │   ├── files
│   │   ├── lib
│   │   ├── manifests
│   │   ├── metadata.json
│   │   ├── Rakefile
│   │   ├── spec
│   │   │   ├── classes
│   │   │   ├── fixtures
│   │   │   │   ├── hieradata
│   │   │   │   ├── hiera.yaml
│   │   │   └── spec_helper.rb
│   │   ├── templates
│   │   └── tests
│   └── role
│       ├── manifests
│       ├── metadata.json
│       ├── Rakefile
│       ├── README.md
│       ├── spec
│       └── tests
├── environment.conf
├── Gemfile
├── hiera
├── hiera.yaml
├── manifests
│   └── site.pp
├── Puppetfile
└── r10k_installation.pp

To:

.
├── dist
│   ├── profile
│   │   ├── files
│   │   ├── lib
│   │   ├── manifests
│   │   ├── metadata.json
│   │   ├── Rakefile
│   │   ├── templates
│   │   └── tests
│   └── role
│       ├── manifests
│       ├── metadata.json
│       ├── Rakefile
│       ├── README.md
│       └── tests
├── environment.conf
├── Gemfile
├── hiera
├── hiera.yaml
├── manifests
│   └── site.pp
├── Puppetfile
├── r10k_installation.pp
└── spec
    ├── classes
    ├── fixtures
    ├── hieradata
    ├── hiera.yaml
    └── spec_helper.rb

I haven’t done this yet but did talk to some others who already do this and are benefiting from the simplified test setup. I’m looking forward to trying it out soon.

November Goal: Pay down Puppet Tech Debt Part 1

It is getting close to the time of the year when the pace of feature-driven change slows down – people want stability when they are on vacation and especially when they’re holding the pager and others are on vacation, and Lord help anyone who negatively affects a Black Friday sale. This is a great time to work on your technical debt. First, you need to identify where it lies!

I expect to spend most of this week identifying areas at work where there are pain points specifically related to tech debt and whether it is better to keep paying the interest or if it is time to pay the whole thing down. I have identified a few candidates related to Puppet already, mostly from lessons learned at PuppetConf.

  • Convert tests to use rspec-puppet-facts. A long list of custom facts in each spec test becomes untenable pretty quickly. Preliminary tests show that I need to chose whether tests are based on Windows or Linux, as mixing and matching in the same tests would break most of them, and I’m leaning toward Linux. This does mean that some tests will not use rspec-puppet-facts and will keep their own fact lists.
  • Convert params patterns to Data in Modules.
  • Try out octocatalog-diff – some unexpected string conversions have been painful before.
  • Get a BitBucket-Jenkins-Puppet workflow working and document. This looks promising, does anyone else have workflow guides I can follow?
  • Update my Puppet Workflow documentation. This isn’t paying down any actual tech debt, but I think it goes hand-in-hand with the above item and revisiting it should provide some clarity to what we do and maybe highlight some room for improvement.

I’m sure there will be more to come. I will try and blog about my progress throughout the vDM30in30 challenge.

PuppetConf Followup: Upgrading to Puppet 4

Last week, I gave a talk at PuppetConf 2016, “Enjoying the Journey from Puppet 3.x to 4.x,” and received some great feedback. One of the major points is that you wanted to hear more opinionated viewpoints than “it depends,” even when it depends! It can be difficult to fit that into a 45 minute talk – heck, I had a 45 minute talk at the airport about just one slide! – but thankfully, I have a blog where I can keep writing and no-one can stop me. Let’s take a look at my slides and go through some of the “it depends” points with some more strongly worded opinions.

Continue reading

PuppetConf 2016 Wrap-Up

Last week, I attended PuppetConf 2016. Spoiler alert: it was pretty awesome! Let’s take a look at what happened and provide some thoughts on what it means for the future of Puppet and IT in general. You can see all my live-tweets using this link, and storify links are in each section.

Contributor’s Summit

Storify

The day before the conference talks is the Contributor’s Summit. It is a combination of group brainstorming, a hackathon, and face time. It starts out with a few talks on where Puppet and the community is, and a non-keynote-spoilering rough idea of where things are going. After about two hours of talks, the summit breaks out into self-managed brainstorming and hacking. If you have a project or idea you are working, you are encouraged to step on stage and announce what you plan to work on and where you’re sitting. Others can then join you to contribute to what you’re working on. Or, you could hack away wherever you’re sitting and mingle with other attendees at will.

Continue reading

Puppet 3 End of Life 12/31/2016

I mentioned this at PuppetConf: Puppet 3 support ends 12/31/2016! Hopefully you weren’t surprised, but if you were, you have just over 60 days to get upgraded. My talk at PuppetConf was about the upgrade journey (video)so may help, and there was a whole track for Puppet 4 on the PuppetConf 2016 video list. Get thee to the upgrade-mobile, pronto!

Started the upgrade and having problems? Ask on the community slack. Need help doing the actual work? If you’re on PE, engage professional services; there are many consultants who will be willing to help you with FOSS.

Some of you have also asked about a reference for this EOS date:

Deploy your #Puppet Enterprise license key with Puppet

Since I manage my Puppet infrastructure with Puppet itself, I am for full automation. For Puppet Enterprise, that includes deploying the license key file from the puppet fileserver (profile/files/master/license.key served as puppet:///modules/profile/master/license.key). When upgrading to the latest Puppet Enterprise version, 2016.2.0, I encountered a change that was tricky to resolve – the puppet_enterprise::license class accepted a license_key parameter, which was marked as deprecated:

Warning: puppet_enterprise::license::license_key is deprecated and will be removed in the next
    PE version. Please use puppet_enterprise::license_key_path. If using the Node Manager, the class
    is located in the PE Infrastructure node group.

Easy, I’ll just use the parameter license_key_path instead! Except, it wants a location for a file on the master, and I’m trying to deploy a file to the master!

Continue reading

Upgrading to Puppet 4 at #PuppetConf 2016

As I did last year, I submitted a proposal for PuppetConf 2016 and it was accepted! As I did last year, I am requesting your help with it.

The talk,  Enjoying the Journey from Puppet 3.x to 4.x, will help attendees lay out a plan to get to Puppet 4. I will be sharing my experiences from POSS and PE upgrades,  including tools to assist with the migration and some pitfalls to avoid. There are many ways to perform these upgrades and my experiences are limited, so I’d like to hear about yours. If you are interested in sharing your experiences and grant me permission to share them in my talk, you can contact me on twitter/DM or by submitting a PR against my PuppetConf github repo. Let me know if you would like to be credited or keep it anonymous. Thanks!

Print the rspec-puppet catalog, courtesy of @willaerk

Sometimes, when you are writing an rspec-puppet test, you’re not sure exactly how the test should be written. You know that you want to test a resource with some extra attribute, but you may be describing the resource wrong, or using a bad regex to test the contents. Rspec-puppet will helpfully tell you when the code and the test do not match, but it does not always tell you where the mismatch is. In those cases, you want to see the whole catalog to see more detail on the resource attributes.

Let’s look at a very simple profile and its corresponding tests and test output:

Continue reading