In my exploration of Puppet, I’ve found a lot of oblique references to managing software deployments with it, but very little solid guides on how to do so. I need to tackle this for work, so I figured I should start at the top – creating a software deployment. To be clear, I’m speaking of internally developed software, or modifications to public software, not something you can find in your distribution’s packages and install with the puppet package resource type.
Creating Some Software
Going back even further, we need to create some software. I’d wager that most already have something laying around – perhaps a few scripts in a directory along with a Makefile that lets you run “make install” to put them in the final destination, or a tarball and config file that are “installed” by a script that untars the software and copies your customized config in place. If you don’t have something like that, let’s make something. How about a simple PHP application? It’s just a Hello World, nothing special, so you don’t need to know PHP for this.
Spin up a new VM, or requisition one of your existing dev VMs. I’m going to use server01 from the Puppet series. Make sure apache and php are installed, and if this node isn’t managed via our web server role, iptables may block connections so we will stop it:
[rnelson0@server01 ~]$ sudo puppet apply -e "package {['httpd', 'php']: ensure => present}"
Notice: Compiled catalog for server01.nelson.va in environment production in 0.33 seconds
Notice: /Stage[main]/Main/Package[php]/ensure: created
Notice: Finished catalog run in 20.70 seconds
[rnelson0@server01 ~]$ sudo service httpd restart
Stopping httpd: [FAILED]
Starting httpd: [ OK ]
[rnelson0@server01 ~]$ sudo service iptables stop
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
Next, we create the application. We’ll stick it in the documentroot of our apache installation, which on CentOS is /var/www/html. Here’s the file contents:
[rnelson0@server01 ~]$ sudo vi /var/www/html/index.php [rnelson0@server01 ~]$ cat /var/www/html/index.php <html> <head> <title>PHP Test</title> </head> <body> <?php echo '<p>Hello World</p>'; ?> </body> </html>
That’s it. If you view your web site (http://server01/) you should see a plain white page with the words “Hello world!” in black at the top. Congratulations, you’re a software developer!
Packaging
So you’ve got your software, now what can you do with it? As it is, not much. If you need another web server that wants to send salutations and greetings to the world, you’ll have to repeat the steps above. That’s not ideal, far from it. You could improve your manual effort by using tar to grab /var/www/html and it’s contents and scp it to the new server, or use rsync with scp to move the files in place. This is where your distribution’s packaging system comes into play. With CentOS, the target package system is rpm. There’s a guide to creating a .rpm. If you’re using a Debian-based system, there’s a guide for .deb too. If you can gaze upon either of those articles without your eyes crossing, you are a better person than I am.
Because these processes are so complicated, so fraught with danger, and so tedious, you probably won’t be surprised that in days past, most people gave up and stuck with their homegrown installation scripts. Or editing files in production. Or a million other horrible ways to manage your software. A packaging system gives you a lot of features that improve your software’s life. Two of the key features are versioning and dependencies. If you create a package, you can install v1.0.0, then install v1.0.1 and have a way to uninstall or rollback to v1.0.0 later. If it requires a webserver and php, as our software does, you can specify that so that someone installing your software will be prompted to add the missing packages. Except for the fact that the process is so horrible, who wouldn’t want to use a packaging system?
With that thought in mind, Jordan Sissel wrote FPM. What does that stand for? You can figure it out… FPM lets you create packages in a target architecture of your choosing with a single command rather than the hell that is those packaging guides above. As the software docs say, you just install, run rpm, and have your package. This is something that can either outright replace or be integrated with your current build scripts. Let’s install it with puppet. We only need ruby-devel and the gem package fpm per the site:
[rnelson0@server01 ~]$ sudo puppet apply -e "package {'ruby-devel': ensure => present}"
Notice: Compiled catalog for server03.nelson.va in environment production in 0.53 seconds
Notice: /Stage[main]/Main/Package[ruby-devel]/ensure: created
Notice: Finished catalog run in 2.25 seconds
[rnelson0@server01 ~]$ sudo puppet apply -e "package {'fpm': provider => gem, ensure => present}"
Notice: Compiled catalog for server03.nelson.va in environment production in 0.30 seconds
Error: Execution of '/usr/bin/gem install --no-rdoc --no-ri fpm' returned 1: ERROR: Error installing fpm:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb
creating Makefile
make
gcc -I. -I/usr/lib64/ruby/1.8/x86_64-linux -I/usr/lib64/ruby/1.8/x86_64-linux -I. -DJSON_GENERATOR -fPIC -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -fPIC -O3 -Wall -O0 -ggdb -c generator.c
make: gcc: Command not found
Yep, there’s a missing pre-req. I have submitted a PR to have gcc listed as well. Add it with puppet and then try to add the gem again:
[rnelson0@server01 ~]$ sudo puppet apply -e "package {'gcc': ensure => present}"
Notice: Compiled catalog for server03.nelson.va in environment production in 0.69 seconds
Notice: /Stage[main]/Main/Package[gcc]/ensure: created
Notice: Finished catalog run in 75.45 seconds
[rnelson0@server01 ~]$ sudo puppet apply -e "package {'fpm': provider => gem, ensure => present}"
Notice: Compiled catalog for server03.nelson.va in environment production in 0.67 seconds
Notice: /Stage[main]/Main/Package[fpm]/ensure: created
Notice: Finished catalog run in 46.63 seconds
You can handle the whole thing with this one liner:
[rnelson0@server01 ~]$ sudo puppet apply -e "package {['ruby-devel', 'gcc']: ensure => present} package {'fpm': provider => gem, ensure => present}"
FPM is now installed and ready to be used.
Generating Our First Package
The final step is to generate a package. We’ve got a directory /var/www/html that we want to turn into an rpm format package. FPM can do this with a few arguments:
- -s <type> – The source package type, in this case ‘dir’ for directory
- -t <type> – The destination package tpe, in this case ‘rpm’ for rpm
- -n <packagename> – The name of the package
- <source> – In our case, the directory that will be the source of the package
Of course, there are far more arguments and you should use fpm –help to get the full list. You’ll get a lot of mileage out of –exclude (-x, great way to exclude .git directories and the like), –version (-v), –iteration (rpm ‘release’, FreeBSD ‘PORTVERSION’, deb ‘debian_revision’), –epoch, –vendor, —maintainer (defaults to user@server), –description, –provides, and –depends. We’ll use the defaults for now:
[rnelson0@server01 ~]$ fpm -s dir -t rpm -n helloworld /var/www/html
no value for epoch is set, defaulting to nil {:level=>:warn}
no value for epoch is set, defaulting to nil {:level=>:warn}
Need executable 'rpmbuild' to convert dir to rpm {:level=>:error}
Uhoh, another missing dependency! I couldn’t find a good list of the full possible dependencies, and this probably isn’t listed because you only need the dependencies for what you’re building. If you only build debs, you don’t need rpmbuild, for instance. However it can be tricky because the package name doesn’t always correlate to the binary, though fpm only tells you the name of the missing binary. Let’s install rpmbuild, which is actually part of a package called rpm-build and try to generate the package again:
[rnelson0@server01 ~]$ sudo puppet apply -e "package {'rpm-build': ensure => present}" Notice: Compiled catalog for server03.nelson.va in environment production in 0.32 seconds
Notice: /Stage[main]/Main/Package[rpm-build]/ensure: created
Notice: Finished catalog run in 17.09 seconds
[rnelson0@server01 ~]$ fpm -s dir -t rpm -n helloworld /var/www/html
no value for epoch is set, defaulting to nil {:level=>:warn}
no value for epoch is set, defaulting to nil {:level=>:warn}
Created package {:path=>"helloworld-1.0-1.x86_64.rpm"}
It doesn’t get much simpler than that! Let’s examine the package to see what it provides. You can use rpm with some options to check out the metadata and the file contents:
[rnelson0@server01 ~]$ rpm -qpil helloworld-1.0-1.x86_64.rpm Name : helloworld Relocations: / Version : 1.0 Vendor: rnelson0@server03 Release : 1 Build Date: Fri 18 Jul 2014 08:35:51 PM GMT Install Date: (not installed) Build Host: server03.nelson.va Group : default Source RPM: helloworld-1.0-1.src.rpm Size : 112 License: unknown Signature : (none) Packager : <rnelson0@server03> URL : http://example.com/no-uri-given Summary : no description given Description : no description given /var/www/html/index.php
Summary
It doesn’t get much simpler than that. If you set aside an hour to learn this, you’ve probably got 40 minutes left to use wisely. You can now build an rpm in moments. With a little more time, you can start filling out the metadata and integrating it with your current software processes. Eventually, you can start distributing your apps via packages and no longer have to worry about any files or configuration that was done on development but skipped in the documentation, it’s all here! Stand up a brand new machine, install your software as an rpm, and start testing immediately. We’ll look into that in a future article.
rnelson0 
One thought on “Creating packages with FPM”