In last week’s article, we learned how to build a package with FPM, specifically an RPM. Today, we’ll look at creating a Yum repository to host your packages. The repo can be built with puppet, which can also distribute settings so all your managed nodes can use the repo. By adding the package to the repo, it becomes available to install, again via puppet. This is the first step on the road to the automated software packing and delivery that is vital for continuous integration.
A repo has a few components.
- Webserver – Content is served up over http.
- createrepo – A piece of software that manages the repo’s catalog.
- RPMs – What it’s serving up.
We don’t need to know how the pieces work, though. We’ll rely on palli/createrepo to manage the repo itself. We just make sure a webserver is available, the directories are there, and that there’s some content available.
Configure a host
I’m going to start moving faster now, because we’ve done this part a few times already. Please let me know on twitter if I’m going to fast and I can update the page and make sure future articles keep the same pace.
The first thing is to choose a node. Since our series so far relies on the hostname for a role, spin up a new VM called ‘yumrepo01’ or something similar. You don’t need to configure anything afterward, except possibly update Puppet (as I’m writing this, v3.7.0 was just released). Of course, you should run the agent and accept the cert so that the node can communicate with puppet before continuing.
Next up is the manifest. Start with the profile.
[rnelson0@puppet profile]$ cat manifests/yumrepo.pp
class profile::yumrepo {
include '::profile::apache'
apache::vhost {'yum.nelson.va':
docroot => '/var/www/html/puppetrepo',
}
}
Our existing ::profile::apache ensures the web server is installed and running and creates firewall rules to allow traffic in. The apache::vhost definition creates the virtual host for our web server. We will place our repo underneath /var/www/html/puppetrepo but you can choose anywhere you want. The next piece is the role.
[rnelson0@puppet role]$ cat manifests/yumrepo.pp
class role::yumrepo {
include profile::base # All roles should have the base profile
include profile::yumrepo
$repodirs = hiera('repodirs')
file { $repodirs :
ensure => 'directory',
}
create_resources('::createrepo', hiera_hash('yumrepos'), {require => File[$repodirs]} )
}
After adding the base and yumrepo commands, we need to create the directories for the repo itself. The vhost doesn’t actually create the specified directory, and there’s also a cache directory. We also create some ::createrepo defines. All of this content comes from hiera:
[rnelson0@puppet hiera-tutorial]$ cat puppet_role/yumrepo.yaml --- classes: - role::yumrepo repodirs: - '/var/www/html/puppetrepo' - '/var/cache/puppetrepo' yumrepos: 'el-6.5': repository_dir : '/var/www/html/puppetrepo/el-6.5' repo_cache_dir : '/var/cache/puppetrepo/el-6.5' suppress_cron_stdout : true
Our repo has the name el-6.5 (based on our CentOS 6.5 base image) and exists one level underneath our docroot and cache directory (there’s a good reason why). There’s a cron job that runs every minute, rebuilding the index. Unless you like getting new mail every minute, I suggest suppressing output. Here’s the cron job it will create:
# Puppet Name: update-createrepo-el-6.5 */1 * * * * /usr/bin/createrepo --cachedir /var/cache/puppetrepo/el-6.5 --changelog-limit 5 --update /var/www/html/puppetrepo/el-6.5 1>/dev/null
There are other options you can feed ::createrepo, but this will suffice to start. With the role and profile manifest and hiera yaml in place on the master, you can run the agent on yumrepo01 and everything should install properly. You can test access by visiting http://yum.nelson.va/ afterward. You should see the el-6.5/repodata directory structure, and the bottom dir will have a few files. Great success!
Populating the repo
This step is easy – copy files to the repo. Where packages go is up to you. You can toss them in the top level, you can create an RPMS or Packages directory; wherever you through it, createrepo will find it. We’ll put our files at the top level (/var/www/html/puppetrepo/el-6.5/ in this case). The crontab job runs once a minute, so after a brief wait it will be ready. You can again browse to the repo at http://yum.nelson.va/el-6.5/, using the correct hostname for your environment.
If you do use subdirectories, ::createrepo does not create or manage this subdirectory, so you would have to. Be aware that by doing so, you’ll be managing a directory that is managed by the define and you’ll have to ensure you don’t manage the same directory twice, once via ::createrepo and twice via a File resource. You could set manage_repo_dirs to false in your ::createrepo definition if you’d rather manage all the directories yourself.
Using the repo
Finally, we need a way to make use of the repo. A yumrepo resource type is native to puppet, so let’s look at the definition for our new repo:
yumrepo {'el-6.5':
descr => 'rnelson0 El 6.5 - x86_64',
baseurl => 'http://yum.nelson.va/el-6.5/',
enabled => 'true',
gpgcheck => 'false',
}
We are defining the repo el-6.5, giving it a description and the URL to use. Adding a repo will just create the repo file but it defaults to disabled, so we set enabled to true. The default for gpgcheck, which ensures signatures on RPMs are valid, is true. Since we haven’t signed our package, we’ll set that to false. I’ll leave configuration of gpg, or lack thereof, up to you. Let’s go back to server01 and use puppet apply to attach the repo. Afterward, search for helloworld.
[rnelson0@server01 ~]$ yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: bay.uchicago.edu
* extras: mirror.steadfast.net
* updates: mirror.anl.gov
repo id repo name status
base CentOS-6 - Base 6,367
extras CentOS-6 - Extras 15
puppetlabs-deps Puppet Labs Dependencies El 6 - x86_64 68
puppetlabs-products Puppet Labs Products El 6 - x86_64 422
updates CentOS-6 - Updates 1,467
repolist: 8,339
[rnelson0@server01 ~]$ sudo puppet apply
yumrepo {'el-6.5':
descr => 'rnelson0 El 6.5 - x86_64',
baseurl => 'http://yum.nelson.va/el-6.5/',
enabled => 'true',
gpgcheck => 'false',
}
Notice: Compiled catalog for server01.nelson.va in environment production in 0.11 seconds
Notice: /Stage[main]/Main/Yumrepo[el-6.5]/ensure: created
Notice: Finished catalog run in 0.07 seconds
[rnelson0@server01 ~]$ yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: bay.uchicago.edu
* extras: mirror.steadfast.net
* updates: mirror.anl.gov
el-6.5 | 2.9 kB 00:00
el-6.5/primary_db | 2.3 kB 00:00
repo id repo name status
base CentOS-6 - Base 6,367
el-6.5 rnelson0 El 6.5 - x86_64 1
extras CentOS-6 - Extras 15
puppetlabs-deps Puppet Labs Dependencies El 6 - x86_64 68
puppetlabs-products Puppet Labs Products El 6 - x86_64 422
updates CentOS-6 - Updates 1,467
repolist: 8,342
[rnelson0@server01 ~]$ yum search helloworld
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: bay.uchicago.edu
* extras: mirror.steadfast.net
* updates: mirror.anl.gov
========================================== N/S Matched: helloworld ===========================================
helloworld.x86_64 : no description given
Name and summary matches only, use "search all" for everything.
Go ahead and try and install it with yum (abbreviated output):
[rnelson0@server01 ~]$ sudo yum install helloworld ... Resolving Dependencies --> Running transaction check ---> Package helloworld.x86_64 0:1.0-1 will be installed --> Finished Dependency Resolution ... Is this ok [y/N]: y Downloading Packages: http://yum.nelson.va/el-6.5/helloworld-1.0-1.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 Forbidden"
If you check the permissions on the file, you might wonder why a world readable file isn’t available. Our CentOS template enforces selinux, so let’s take a look at the context:
[root@yumrepo01 el-6.5]# ls -laZ /var/www/html/puppetrepo/el-6.5/helloworld-1.0-1.x86_64.rpm -rw-rw-r--. rnelson0 rnelson0 unconfined_u:object_r:user_home_t:s0 /var/www/html/puppetrepo/el-6.5/helloworld-1.0-1.x86_64.rpm [root@yumrepo01 el-6.5]# ls -laZ /var/www/html/puppetrepo/el-6.5/repodata/repomd.xml -rw-r--r--. root root unconfined_u:object_r:httpd_sys_content_t:s0 /var/www/html/puppetrepo/el-6.5/repodata/repomd.xml
Here’s our issue: the new rpm is not in the httpd_sys_content_t context but is in the user_home_t context. This occurred because I scp’ed the file from server01 as rnelson0 to yumrepo01 as rnelson0, then on yumrepo01 moved the file to /var/www/html/puppetrepo as root. Let’s move the file back, then try copying it as root:
[root@yumrepo01 el-6.5]# mv /var/www/html/puppetrepo/el-6.5/helloworld-1.0-1.x86_64.rpm ~rnelson0/ [root@yumrepo01 el-6.5]# cp ~rnelson0/helloworld-1.0-1.x86_64.rpm /var/www/html/puppetrepo/el-6.5/ [root@yumrepo01 el-6.5]# ls -laZ /var/www/html/puppetrepo/el-6.5/helloworld-1.0-1.x86_64.rpm -rw-r--r--. root root unconfined_u:object_r:httpd_sys_content_t:s0 /var/www/html/puppetrepo/el-6.5/helloworld-1.0-1.x86_64.rpm
That’s better! You can also use the restorecon command to restore the default context on files, i.e. restorecon /var/www/html/puppetrepo/el-6.5/helloworld-1.0-1.x86_64.rpm. Now try and install it again:
[rnelson0@server01 ~]$ sudo yum install helloworld ... Is this ok [y/N]: y Downloading Packages: helloworld-1.0-1.x86_64.rpm | 1.7 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : helloworld-1.0-1.x86_64 1/1 Verifying : helloworld-1.0-1.x86_64 1/1 Installed: helloworld.x86_64 0:1.0-1 Complete! [rnelson0@server01 ~]$ rpm -qa | grep helloworld helloworld-1.0-1.x86_64 [rnelson0@server01 ~]$ rpm -ql helloworld /var/www/html/index.php
Summary
You now have a package, a repository that hosts it, and a client that can install the packages hosted by the repo. Go ahead and spin up server02-server10 and you should be able to easily define the repo and install the rpm. As we haven’t created any dependencies yet, the lack of httpd will prevent our app from working. If you install apache manually or via puppet, your hello world app will work. We’ll work on some more details of FPM, including dependencies, and integration of packages and repos with our puppet manifests next time.
rnelson0 
Awesome! Really like how neat the hiera + create_resources are. Going to refactor 🙂 And I really should have made suppress_cron_stdout deafult to true…
Thanks! Wouldn’t have been possible without your module!