Today, let’s build on version 1.0 of our Jenkins job. We are running builds against every commit, but when someone opens a pull request, they don’t get automated builds or feedback. If the PR submitter even knows about Jenkins, and has network access and a login, they can look at it to find out how the tests went, but most people aren’t going to have that visibility (especially if your Jenkins server is private, as in this example setup). We need to make sure Jenkins is aware of the pull request and that it updates the PR with the status. Our end goal is for each PR to start a Jenkins build and update the PR with a successful check when done:

To get there, we will install and configure a new plugin and configure our job to use the plugin.

Github Pull Request Builder Plugin

Take a look at the GitHub pull request builder plugin (hereafter GHPRB). It does what it says on the tin, but I found the documentation a little clunky in places so we’ll review it (please note, they’re looking for a co-maintainer, if you’re interested in helping!) to make sure no steps are skipped. First, we need to add the plugin name, ghprb, to our plugins list. It has some dependencies, ssh-agent and bouncycastle-api (I didn’t come up with that name!), add them to profile::jenkins::plugins as we used during the install. After the next puppet run, they’ll be installed but not yet ready to go.

We need to configure the plugin. This was not intuitive to me in the plugin doc. Go to Manage Jenkins -> Configure System and scroll down to the new GitHub Pull Request Builder section. We need to add a credential. Click Create API Token…, enter your github user/pass, and click Create Token. If you entered the correct information, you’ll get a set of token and credentials IDs. You’ll need this for a modal dialog, so copy it into some other window. Up the page just a little bit, click on the Add button and add a Jenkins credential. The type is Secret Text. Enter the credentialsID here as well as some description, such as “Jenkins GHPRB”, then click Add. You can now click Test Credentials, enter the name of a github user and select Test basic connection to GitHub. Click Connect to API and it should say you’re connected. If that works, change the Repository owner/name to that of your controlrepo, check Test Permissions to a Repository, and click Check repo permissions. It should come back with Admin, Push, Pull. I’m not entirely up on the permissions of GitHub, but you can find all of your personal access tokens listed at https://github.com/settings/tokens, which is also where you can edit the permissions. Take a moment to check all your tokens while you’re here.

If you mess up or lose the token’s credentialID, you cannot get it back. Nor can you just create a new one via Jenkins. You will need to delete or rename the token on GitHub, as you cannot create two tokens with the same name and the plugin’s Create API Token… function always uses the same name. You can, of course, create a new one manually and copy the credentialID as a Jenkins credential.

Below the GitHub Auth section are a series of checkboxes and other settings. I checked Auto-manage webhooks and Use comments to report results when updating commit status fails and left the rest alone. This part is up to you. Make sure you hit Save when you are done. This completes the configuration of the plugin.

Modifying the Job

Next, we need to modify the job. Edit your existing job. We need to change some of the Git setup, enable some GHPRB plugin settings, and add a Post-Build action.

Under Source Code Management, click Advanced. The Refspec section should be blank; change that `+refs/pull/*:refs/remotes/origin/pr/*` so it looks at PR activity (this is a reference from GitHub rather than Git; if you’re using another service you’ll have to research the correct refspec). Under Branches to build -> Branch Specifier, enter ${sha1}. Hit the question mark icon for some details on what this is used for; the value is suggested by GHPRB. Click Apply.

Next, go to Build Triggers. Check the box GitHub Pull Request Builder. Choose the one credential available to you (interestingly this credential is available even if you skipped the plugin configuration above – though it won’t link to anything and just generates errors, hence my initial confusion). Check Use github hooks for build triggering, then click Advanced. Add a check to Build every pull request automatically without asking (Dangerous!). That’s all you NEED to do, but you may want to revisit this later with the help of the plugin docs to tailor the setup, specifically by restricting branches or admins. The one optional thing I configured was under the Trigger Setup button. I set the SUCCESS message to Success! and FAILURE message to Failure! You’ll get success/failure indicators but I like the additional text. Click Apply.

Finally, go to the Post-build Actions section. You should have none currently so click Add post-build action and select Set GitHub commit status (universal). You can leave it as-is, or click the question mark icons to see what the different options allow you to do. Click Save when you are done.

Create a PR

Go ahead and create a PR on your controlrepo now! All you need is a single commit (update the version in metadata.json for example) and then create the PR. After a moment or two, you should see an update to the PR indicating a build is starting!

If it didn’t build, that’s okay! SSH into the Jenkins node and take a look at /var/log/jenkins/jenkins.log. Here’s a successful run:

Jan 19, 2017 12:11:15 AM org.jenkinsci.plugins.ghprb.GhprbRootAction handleAction
INFO: Checking PR #57 for rnelson0/controlrepo
Jan 19, 2017 12:11:15 AM org.jenkinsci.plugins.ghprb.GhprbTrigger handlePR
INFO: Checking PR #57 for job test
Jan 19, 2017 12:11:15 AM org.jenkinsci.plugins.ghprb.GhprbPullRequest 
INFO: Created Pull Request #57 on rnelson0/controlrepo by rnelson0 (rnelson0@gmail.com) updated at: 1/19/17 12:11 AM SHA: a3553e3b079aeae104c6a745e0d09b41c5546d8b
Jan 19, 2017 12:11:15 AM org.jenkinsci.plugins.ghprb.GhprbPullRequest updatePR
INFO: Pull request #57 was updated on repo rnelson0/controlrepo but there aren't any new comments nor commits; that may mean that commit status was updated.

Unsuccessful runs are a little messier but tend to be pretty verbose, good enough to get you on the right track:

Jan 16, 2017 8:59:37 PM org.jenkinsci.plugins.ghprb.GhprbTrigger start
INFO: Starting the ghprb trigger for the test job; newInstance is true
getConfigPage -> /rvm_wrapper/config
Jan 16, 2017 9:03:00 PM org.jenkinsci.plugins.ghprb.GhprbGitHubAuth getBuilder
WARNING: credentialsId not set for context test, using anonymous connection
Jan 16, 2017 9:03:02 PM org.jenkinsci.plugins.ghprb.GhprbRepository initGhRepository
SEVERE: Could not retrieve GitHub repository named rnelson0/controlrepo (Do you have properly set 'GitHub project' field in job configuration?)
java.io.FileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3"}
        at org.kohsuke.github.Requester.handleApiError(Requester.java:666)
        # giant stack trace goes here

In this case, I had not completed the setup and created the API token, so any attempt to use the API resulted in authentication issues. That was the only error I encountered, hopefully other errors are as helpful.

Once you get the PR building, you’ll eventually receive success or failure. Here’s what success looks like:

If it fails (create a commit with bad metadata so it fails fast), it will look like this:

The Details link will point right to your Jenkins job. Be aware that anyone who can view the repository on GitHub can see that link, even if they can’t get to it, so make sure the URL is sanitary, so to speak 😉

Summary

We’ve now gone from builds on all commits to builds on PRs. One of those two is probably helpful for you. If you’re an individual and you have a private repo, this is a cost-effective replacement for Travis CI, as Travis CI doesn’t do free builds on private repos, and even if you can use Travis CI, this may enable some testing that you can’t do on Travis nodes for whatever reason.

We’ve now covered “legacy” jobs in Jenkins pretty well. While there’s certainly tweaking to be done here, they’re considered the past. The future of Jenkins lies in Pipeline as Code, so we’ll move on to that next. In the meantime, let me know in the comments or on twitter of any suggestions you might have for improving the use of legacy jobs. If I collect enough, I may look into a version 1.2 article. I will look into creating a gist of my Jenkins profile, hiera data, and an XML export of the job, for those who want to implement this. Thanks, and enjoy!

Update: Here’s the gist!