Now that we have shaved the yak of generate-puppetfile, it’s time to move my rspec tests to the top of the controlrepo, as discussed on Thursday. To do so, we need to move not just the spec tests, but also the files Rakefile, .rspec, .rubocop.yml, and create a facsimile metadata.json, and of course, generate a new .fixtures.yml.

Moving the files is pretty simple. In my controlrepo, I only have a profile class with tests, so I’m going to take those as-is. We do want to make sure that we start with a clean environment, we don’t want to copy a bunch of fixtures or other temp files. We could use git clean -ffdxn; if you don’t want to redownload your gems you can manually clean up the relevant directories. Then we can do the file shuffle:

$ git clean -ffdxn
Would remove .bundle/
Would remove Gemfile.lock
Would remove coverage/
Would remove dist/profile/spec/fixtures/manifests/
Would remove dist/profile/spec/fixtures/modules/
Would remove vendor/
$ rm -fR dist/profile/spec/fixtures/manifests/ dist/profile/spec/fixtures/modules/
$ git mv dist/profile/spec ./
$ git mv dist/profile/{Rakefile,.rspec,.rubocop.yml} ./

Continue reading →

As I discussed on Thursday, I am looking to re-architect the layout of my controlrepo’s rspec tests. Of course, there were yaks to shave, first. To that end, I’ve released version 0.10.0 of generate-puppetfile (rubygem, github project) with the added ability to run generate-puppetfile --create-fixtures at the top of your controlrepo and generate a .fixtures.yml that will set up symlinks for all of the modules contained in your controlrepo. This functionality is based off the existence of the file environment.conf in the current directory and it containing a modulepath stanza. All the paths in the modulepath are explored and symlink-type fixtures created for each module found. Previous functionality when run inside a module directory is still preserved, of course.

Because the project has not reached version 1.0.0 yet, I have renamed the option --fixtures to --create-fixtures. This is in preparation for a feature request for an --only-fixtures feature request.

Let’s look at how to use this new feature. Before we start, we must be at the top level of the controlrepo. Next, we need environment.conf to have a modulepath stanza. In my controlrepo, the file looks like this:

Continue reading →

I was having a problem yesterday with a specific rspec test that was failing for my puppet tool generate-puppetfile, and I couldn’t understand why. I was expecting an exitcode of 0 but was receiving 1, so obviously I had an issue, but I couldn’t see the accompanying error message. When I attempted to run the command myself, it succeeded, so I was pretty sure the issue was with either argument handling by my program or more likely, the way I wrote my rspec test (spoiler:  it was in my rspec!). Here’s what the rspec test looked like to start with:

  context 'when creating fixtures' do
    let :args do
        'rnelson0/certs'
        '--create-fixtures'
    end

    its(:exitstatus) { is_expected.to eq(0) }
    it 'should create .fixtures.yml' do
      File.exists? './.fixtures.yml'
    end
  end

Continue reading →

Yesterday was our 14th anniversary, so I didn’t have time to write a blog post, but I did look into a tech debt issue: rspec tests. In addition to adding rspec-puppet-facts, I found a program called onceover that offers two concepts I want to look into.

First, there is a rake task to generate fixtures. I have a similar feature in generate-puppetfile, but it’s not as perfect as I’d like – it often requires some manual touch afterward. Onceover’s rake task does not have that issue. I hope to be able to grab the rake task without being forced to use the rest of it or affecting the existing test set up. Maybe I’ll be interested in the rest of it someday, but not right now, and it’s great when you’re not forced to make a forklift upgrade like that.

The second item is centralizing rspec-puppet tests in the controlrepo rather than inside each module itself. That will change the relevant portions of my controlrepo layout from:

.
├── dist
│   ├── profile
│   │   ├── files
│   │   ├── lib
│   │   ├── manifests
│   │   ├── metadata.json
│   │   ├── Rakefile
│   │   ├── spec
│   │   │   ├── classes
│   │   │   ├── fixtures
│   │   │   │   ├── hieradata
│   │   │   │   ├── hiera.yaml
│   │   │   └── spec_helper.rb
│   │   ├── templates
│   │   └── tests
│   └── role
│       ├── manifests
│       ├── metadata.json
│       ├── Rakefile
│       ├── README.md
│       ├── spec
│       └── tests
├── environment.conf
├── Gemfile
├── hiera
├── hiera.yaml
├── manifests
│   └── site.pp
├── Puppetfile
└── r10k_installation.pp

To:

.
├── dist
│   ├── profile
│   │   ├── files
│   │   ├── lib
│   │   ├── manifests
│   │   ├── metadata.json
│   │   ├── Rakefile
│   │   ├── templates
│   │   └── tests
│   └── role
│       ├── manifests
│       ├── metadata.json
│       ├── Rakefile
│       ├── README.md
│       └── tests
├── environment.conf
├── Gemfile
├── hiera
├── hiera.yaml
├── manifests
│   └── site.pp
├── Puppetfile
├── r10k_installation.pp
└── spec
    ├── classes
    ├── fixtures
    ├── hieradata
    ├── hiera.yaml
    └── spec_helper.rb

I haven’t done this yet but did talk to some others who already do this and are benefiting from the simplified test setup. I’m looking forward to trying it out soon.

It is getting close to the time of the year when the pace of feature-driven change slows down – people want stability when they are on vacation and especially when they’re holding the pager and others are on vacation, and Lord help anyone who negatively affects a Black Friday sale. This is a great time to work on your technical debt. First, you need to identify where it lies!

I expect to spend most of this week identifying areas at work where there are pain points specifically related to tech debt and whether it is better to keep paying the interest or if it is time to pay the whole thing down. I have identified a few candidates related to Puppet already, mostly from lessons learned at PuppetConf.

• Convert tests to use rspec-puppet-facts. A long list of custom facts in each spec test becomes untenable pretty quickly. Preliminary tests show that I need to chose whether tests are based on Windows or Linux, as mixing and matching in the same tests would break most of them, and I’m leaning toward Linux. This does mean that some tests will not use rspec-puppet-facts and will keep their own fact lists.
• Convert params patterns to Data in Modules.
• Try out octocatalog-diff – some unexpected string conversions have been painful before.
• Get a BitBucket-Jenkins-Puppet workflow working and document. This looks promising, does anyone else have workflow guides I can follow?
• Update my Puppet Workflow documentation. This isn’t paying down any actual tech debt, but I think it goes hand-in-hand with the above item and revisiting it should provide some clarity to what we do and maybe highlight some room for improvement.

I’m sure there will be more to come. I will try and blog about my progress throughout the vDM30in30 challenge.