Welcome to the Puppet and Git, class 205: Git Hooks. Since we finished up talking about workflows, let’s move on and explore what a git hook is and how you can use one to improve your workflow.

Git Hooks

What is a git hook? You can read some boring official documentation, but who does that? Instead here’s a short summary: A git hook is a program that is called when a git events are triggered. These programs are usually simple shell scripts and some common events people use them with are commits. We’ll look at commits and the event that fires before the commit is completed.

Pre-Commit Hook

If you’ve been programming for more than, say, an hour, you’ve undoubtedly experienced the bad mojo that results from missing a semi-colon or curly brace or other piece of syntactical junk. And you may have even committed such a piece of broken code and pushed it upstream just to watch the whole thing fall apart. If we can lint our code, we can determine whether the code meets the syntactic requirements of a language. It’s important to note that linting doesn’t verify that your code does what it says it will do, it JUST verifies that the code will parse or compile. This would prevent the wonderful pattern of commits that looks something like:

• Add feature A
• Forgot a semi-colon
• And a curly brace
• … and a bunch of other stuff
• OK really it works this time I promise
• #*$)^!

Yeah, we’ve all done it. But no-one wants to. So we want to lint code before a commit to find all that syntactic sugar before a commit. Git will let us do so with a pre-commit hook. When attempting a commit, any failures reported by the hook will abort the commit. Perfect! So… how do we do that?

First, we need a hook. I’ve shamelessly stolen a pre-commit hook from mattiasgeniar on github. This was designed to work with Puppet and lint both puppet files, and Ruby ERB templates. We’ll need to run gem install puppet-lint beforehand (if you’re running this on a node managed by puppet, you do NOT need gem install puppet). We’ll then create a new branch called githooks and place this file in hooks/pre-commit.sh. With a simple ln command, we can create a symlink between this file, which is managed as part of the repo, and .git/hooks/pre-commit, which is the actual hook but isn’t part of the repo. In this way other developers can “install” the hook with a simple line of code.

[root@puppet puppet-tutorial]# git checkout production
Switched to branch 'production'
[root@puppet puppet-tutorial]# git checkout -b githooks
Switched to a new branch 'githooks'
[root@puppet puppet-tutorial]# mkdir hooks
[root@puppet puppet-tutorial]# cat > hooks/pre-commit.sh
#!/bin/bash
# pre-commit git hook to check the validity of a puppet manifest
#
# Prerequisites:
#   gem install puppet-lint puppet
#
# Install:
#  /path/to/repo/.git/hooks/pre-comit
#
# Original:
#  blog: http://techblog.roethof.net/puppet/a-puppet-git-pre-commit-hook-is-always-easy-to-have/
#
# Authors:
#  Ronny Roethof
#  Mattias Geniar <m@ttias.be>

echo "### Checking puppet syntax, for science! ###"
# for file in `git diff --name-only --cached | grep -E '\.(pp|erb)'`
for file in `git diff --name-only --cached | grep -E '\.(pp)'`
do
    # Only check new/modified files
    if [[ -f $file ]]
    then
        puppet-lint \
            --no-80chars-check \
            --no-autoloader_layout-check \
            --no-nested_classes_or_defines-check \
            --with-filename $file

        # Set us up to bail if we receive any syntax errors
        if [[ $? -ne 0 ]]
        then
            syntax_is_bad=1
        else
            echo "$file looks good"
        fi
    fi
done
echo ""

echo "### Checking if puppet manifests are valid ###"
# validating the whole manifest takes too long. uncomment this
# if you want to test the whole shebang.
# for file in `find . -name "*.pp"`
# for file in `git diff --name-only --cached | grep -E '\.(pp|erb)'`
for file in `git diff --name-only --cached | grep -E '\.(pp)'`
do
    if [[ -f $file ]]
    then
        puppet parser validate $file
        if [[ $? -ne 0 ]]
        then
            echo "ERROR: puppet parser failed at: $file"
            syntax_is_bad=1
        else
            echo "OK: $file looks valid"
        fi
    fi
done
echo ""

echo "### Checking if ruby template syntax is valid ###"
for file in `git diff --name-only --cached | grep -E '\.(erb)'`
do
    if [[ -f $file ]]
    then
        erb -P -x -T '-' $file | ruby -c
        if [[ $? -ne 0 ]]
        then
            echo "ERROR: ruby template parser failed at: $file"
            syntax_is_bad=1
        else
            echo "OK: $file looks like a valid ruby template"
        fi
    fi
done
echo ""

if [[ $syntax_is_bad -eq 1 ]]
then
    echo "FATAL: Syntax is bad. See above errors"
    echo "Bailing"
    exit 1
else
    echo "Everything looks good."
fi
[root@puppet puppet-tutorial]# chmod +x hooks/pre-commit
[root@puppet puppet-tutorial]# ln -s ../../hooks/pre-commit.sh .git/hooks/pre-commit

Note: As usual, I’m leaving process up to the reader: Do you want to have a repo for hooks (or even use mattiasgeniar’s), or do you want to have hooks in your puppet/module repos? As long as your decision increases the chance of the pre-commit hook being used, you made the correct decision. I’ll touch on this again at the end.

Now, when we try to commit, the pre-commit hook will be run. You’ll notice that the hook pulls certain files out of the commit list and runs puppet lint and/or puppet parser validate against those files. The erb lint is only performed against *.erb files. Note that it is NOT feeding each file through the hook, it’s an external program that does whatever you tell it to. If you design your own it’s up to you to ensure the script does the right thing. To test this commit, we’ll edit manifests/site.pp and add some bogus Puppet DSL and commit. This should fail spectacularly when the linter gets to this file. Let’s see how it performs:

[root@puppet puppet-tutorial]# git add .
[root@puppet puppet-tutorial]# git status
# On branch production
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#       new file:   hooks/pre-commit.sh
#       modified:   manifests/site.pp
#
[root@puppet puppet-tutorial]# cat manifests/site.pp
node 'puppet.nelson.va' {
  include ::base
  notify { "Generated from our notify branch": }
}

node {
  include
}
[root@puppet puppet-tutorial]# git commit -m 'This should fail.'
### Checking puppet syntax, for science! ###
manifests/.site.pp.swp - ERROR: Syntax error (try running `puppet parser validate `) on line 1
manifests/site.pp - WARNING: double quoted string containing no variables on line 3
manifests/site.pp looks good

### Checking if puppet manifests are valid ###
Error: Could not parse for environment production: Could not match S▒WTrootpuppet~root/git/puppet-tutorial/manifests/site.pputf-8 at /root/git/puppet-tutorial/manifests/.site.pp.swp:1
PuTTYERROR: puppet parser failed at: manifests/.site.pp.swp
Error: Could not parse for environment production: Syntax error at '{'; expected '}' at /root/git/puppet-tutorial/manifests/site.pp:6
ERROR: puppet parser failed at: manifests/site.pp

### Checking if ruby template syntax is valid ###

FATAL: Syntax is bad. See above errors
Bailing
[root@puppet puppet-tutorial]#

Perfect! Clean up manifests/site.pp with the command git checkout HEAD manifests/site.pp (this checks out the version of the specified file from the latest commit, HEAD^ gets the previous one, HEAD~2 goes back two commits, etc.) and commit.

[root@puppet puppet-tutorial]# git checkout HEAD manifests/site.pp
[root@puppet puppet-tutorial]# cat manifests/site.pp
node 'puppet.nelson.va' {
  include ::base
  notify { "Generated from our notify branch": }
}
[root@puppet puppet-tutorial]# git commit -m 'Add git pre-commit hook'
### Checking puppet syntax, for science! ###

### Checking if puppet manifests are valid ###

### Checking if ruby template syntax is valid ###

Everything looks good.
[githooks 1e8a0c1] Add git pre-commit hook
 1 files changed, 86 insertions(+), 0 deletions(-)
 create mode 100755 hooks/pre-commit.sh

You’ve now got a pre-commit hook that’s here to stay. If you start working with the repo on another node, just run the ln command again to install the hook:

[root@puppet puppet-tutorial]# ln -s ../../hooks/pre-commit.sh .git/hooks/pre-commit

Long term, you’ll need to look at how to store and setup your hooks. You may have many puppet-related repos, and you neither want to store the same hooks in multiple libraries or install the hook manually in each repo clone. Find a tool that works for you.

Hooks For Everyone

One thing you’ll have to figure out on your own: how to get all your developers to use hooks. For the purpose of creating a general purpose tutorial, I’m going to commit my changes to the githooks feature branch:

[root@puppet puppet-tutorial]# git checkout production
Switched to branch 'production'
[root@puppet puppet-tutorial]# git merge githooks production
Fast-forwarding to: githooks
Already up-to-date with production
Merge made by octopus.
 hooks/pre-commit.sh |   86 +++++++++++++++++++++++++
 1 files changed, 258 insertions(+), 0 deletions(-)
 create mode 100755 hooks/pre-commit.sh
 create mode 100644 hooks/www-r10k.rb
[root@puppet puppet-tutorial]# git push origin production
Counting objects: 7, done.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 2.04 KiB, done.
Total 5 (delta 2), reused 0 (delta 0)
To https://rnelson0@github.com/rnelson0/puppet-tutorial
   b8a08d0..11cf47e  production -> production
[root@puppet puppet-tutorial]# git branch -D githooks
Deleted branch githooks (was 656c5e0).
[root@puppet puppet-tutorial]# git push origin :githooks
To https://rnelson0@github.com/rnelson0/puppet-tutorial
 - [deleted]         githooks

However, these hooks only exist in the puppet repo. The pre-commit hook is used where the edits and commits are made. You’ll need to have the same hook in your module repos – there’s just one now, but you’ll expand that number soon enough. And every developer will need the hook in place, on every machine they work on, as well as puppet-lint. If they only work on modules, they’ll need to clone the puppet repo as well just to get the hooks. Perhaps having the pre-commit hook in your repo makes sense, or maybe it’s a good idea to clone Mattias’s hook directly. Here’s a possible sequence for cloning and installing the hook in three modules that are already cloned:

[rn7284@puppet git]$ git clone https://github.com/mattiasgeniar/puppet-pre-commit-hook.git hook
[rn7284@puppet git]$ for module in `ls`; do (cd $module; ln -sf ../../../hook/pre-commit .git/hooks/pre-commit); done

You’ll note that the for loop adds the hook to the clone of mattiasgeniar’s repo which doesn’t use Puppet. Since you’re not commiting anything, you should be good. If you have other non-Puppet repos in the same dir, you can specify a list instead of the results of ls.

Make sure you figure this out before you implement Puppet in your production environment. Take into consideration your work’s social, technical, and political environment and create a policy that will succeed with your team members.

Post-Receive Coming Soon

Next week we’ll look into a post-receive hook, which fires when you push your changes to an upstream repo, such as Github or Stash, by the receiving repo. We’re almost done with Git tutorials, hang in there!

Additional Resources

Next week we’ll take a look at a post-receive hook. In the meantime, there’s still a lot of tricky problems with hooks, like making sure all your developers use them. While researching this article, I came upon some articles that address everything from the basics to these advanced problems for you and your collaborators.

• Puppet Conf 2013 Recorded Videos – a number of these talk about continuous integration pipelines involving Puppet + another tool, such as Jenkins
• What are git hooks?
• git-hooks: A tool to manage project, user, and global Git hooks for multiple git repositories.
• Why You Need a Git Pre-Commit Hook and Why Most Are Wrong
• Deploying Websites With a Tiny Git Hook
• Tips for using a git pre-commit hook
• Missing git hooks documentation