Puppet – the basics

So I was at this meetup and this guy comes over, “Hey dude, what to you think of puppet?”
“Puppet?”, I reply.
“Yeah, puppet, or are you a chef man?”
“No,no, I work in IT”, I try to explain. But the expression on his face changes to one of thinly veiled disgust, like a new dad changing his baby’s nappy for the first time,
“I’m talking about configuration management maaann”,
“Oh yes, well we use…” but he’s already wandered off to talk to someone cool who gets it.

Truth is, I’d been kind of using puppet for a few years (I just have a perverse impulse to act the fool)…however, this was on a version from the stone age and in all honesty, I didn’t really understand how it all hung together. Time for a knowledge upgrade. Let me share with you what I learnt.

First off let me say if you just want to know how to do it go to our wiki article. If you’re after an overview, stick with us here.

The puppet master

If, like my version of puppet, you’re from the stone age you may not know what puppet does. Well it’s a configuration management tool. Typical computing jargon but WTF does that mean? So when you build a server, there’s a bunch of stuff to do, like add users accounts, update files like resolv.conf, maybe add some more packages. That may not be too much of a problem if you have a handful of servers. but what if you have several hundred, oh and that resolv.conf file needs updating on them all. Of course you could knock up a script to do that but why bother when puppet can do all that stuff for you and more. That’s what puppet is for, keeping files in sync across servers, adding additional users on servers that need them, adding stuff to crontab, etc, etc, etc.

I decided to set up a mini test environment at home. I went for the latest community version of puppet from Puppetlabs. The install procedure (for Ubuntu based disties) is in the wiki article so I won’t repeat it here.

I went for the master/agent model whereby you have a puppet master then install agents on the other servers. The agents pull down their configs from the master. This is one key aspects of the puppet design, certificates are used to ensure the agents talk to the genuine master and the master talks to certified agents. One of the key tasks when adding agents is to get the puppet master to sign the agent certificate otherwise it ain’t going to work. Certificate issues are also one of the common reasons for puppet updates not working. See our puppet certificate troubleshooting article if you have these difficulties.

Environments

Puppet allows you to have more than one environment. It’s a really good idea to make use of this. Why? Because imagine you make an update to environment file but there’s a typo in it. Next thing you know it’s been rolled out to hundreds of production servers, things break, users and managers get cross and you’re unhappy. Stay happy, try it in a test environment first 🙂

It’s a straightforward task to create a test environment. Here’s how:

On the puppet master:

cd /etc/puppetlabs/code/environments
mkdir test

On the agent server:

vi /etc/puppetlabs/puppet/puppet.conf and point environment to test, something like this:

[main]
  certname = x220-laptop
  server = puppet
  environment = test
  runinterval = 1h

(Of course the actual locations of the puppet directories may vary by distribution but they’ll have a similar layout)

The manifest

OK, so now you have somewhere to make cockups without everyone moaning, lets take a look at the files that make things happen. The key file type is known as a manifest. What’s that you may ask? Well, it’s a file that puppet reads and takes instructions from. Here’s an example:

# Manage bleachbit utility
  class bleachbit {
  package { 'bleachbit':
  ensure => 'installed',
  }
}

This file is a class type of manifest with a name of bleachbit. It tells puppet to ensure the package bleachbit is installed on the agent server. But how does puppet know how to install the bleachbit package? Its uses a feature call facter to gather facts about the agent server. One of these facts is that it’s an Ubuntu based server so use apt to do the install.

So we’ve seen an example manifest file but where dose it go? Well, at the top level there’s a file called site.pp This is the top level manifest that points to other manifests. In my test environment it would go here:

/etc/puppetlabs/code/environments/test/manifests/site.pp

For production

/etc/puppetlabs/code/environments/production/manifests/site.pp

Here’s my site.pp from my test environment

node 'x220-laptop' {
  include bleachbit
}

Now, I missed a bit, I decided to set up bleachbit as what is known as a module in puppet. So I placed my bleachbit manifest previously shown in the following directory

/etc/puppetlabs/code/environments/test/modules/bleachbit/manifests

It’s now called init.pp (the base module manifest file is always called init.pp). site.pp knows where to look for modules so the include statement will pick it up. The advantage of using modules is that they can be called multiple times in different manifests.

For my production environment I decided I wanted to run bleachbit regularly via cron. My init.pp looks like this:

class bleachbit {
  include bleachbit::install
  include bleachbit::cronadd
}

That’s uses the include instruction too to call two additional files, install.pp (for installing the package) and cronadd.pp (for updating cron). here are those files:

class bleachbit::install {
  package { 'bleachbit':
  ensure => installed,
  notify => Class['bleachbit::cronadd'],
  }
}

class bleachbit::cronadd {
  cron { 'bleachbit':
  ensure => present,
  command => '/usr/bin/bleachbit -c',
  user => 'root',
  weekday => '*',
  minute => '10',
  hour => '11',
  subscribe => Package['bleachbit'],
  }
}

The notify and subscribe statements establish some dependencies between the two. site.pp for production looks like this:

node default {
}

node 'Microserver' {
  include bleachbit
  include users
  realize (Users::Virtual['tony'])
  realize (Users::Virtual['melissa'])
  realize (Users::Virtual['daniel'])
  realize (Users::Virtual['jamie'])
}

Defined resource types and virtual resources

But wait, I hear you say, what’s all that realize stuff? Well I thought I’d add some users and I’m using a puppet feature called virtual resources. In puppet, a defined resource is some code that can be called multiple times with different parameters, a bit like a bash function. A virtual resource is a resource that is ignored by puppet unless it is specifically called, i.e. it’s not added to the catalog. Usually you’re not allowed to declare the same resource more than one time but sometimes you get overlapping sets of resources which may be required by any number of classes. This is where virtual resources come in although it is only declared once, it can be realized multiple times. As an example, suppose you had a puppet module to install wordpress then a second module for a custom website. Both of these require the apache package. If you install these on different servers, there’s no issue. If you were to try and install them on the same server, puppet would complain with an error message along the lines of “duplicate declaration”. So instead of declaring you can use realize.

Another example is, imagine you have an application made up of several components. ComponentA is installed on server1. You create a manifest for ComponentA which includes a user definition the application has to run under. ComponentB is installed on server2 and includes the same user definition. This isn’t a problem because they’re different servers. But server3 needs both ComponentA and ComponentB. Now we have a problem because the user is declared twice and that’s not allowed. Virtual resources get around this because puppet understands that they’re virtual and doesn’t complain if they clash.

Back to my test example, here’s the init.pp manifest file for the users module:

class users {

  @users::virtual { 'tony':
    userfiles => true,
    uid => 1000,
    realname => 'Tony',
    pass => '$6$QzIYITBc$RCs75ujfU/9XgffYYBn45VEyk4Zr4jJadzd9URVa1PFAMbOOCRvfUPulLtBlYHAR/OYcasrAQrgpAN33/RFkf/',
  }
  @users::virtual { 'melissa':
    uid => 1003,
    realname => 'Melissa',
    pass => '$6$7ajWS2Fh$g0Apa3rwWzq0urAgvbYilw4XB09.cHibX0JGI6sE8XkMGvgrJIsrKcKWscnrBxHXNfITLtT2lL.eXQDua/xe40',
  }
  @users::virtual { 'daniel':
    uid => 1004,
    realname => 'Daniel',
    pass => '$6$DuIkulz3$ZD/y5vV.FjrRCGj97V5l6Hv/IWt7x8RxL66Y5H.Uw9KdYaF/IE8D3kAYgmhvAdCQBrEmWm13miD4zmSFGcrUG0',
  }
  @users::virtual { 'jamie':
    uid => 1005,
    realname => 'Jamie',
    pass => '$6$DuIkulz3$ZD/y5vV.FjrRCGj97V5l6Hv/IWt7x8RxL66Y5H.Uw9KdYaF/IE8D3kAYgmhvAdCQBrEmWm13miD4zmSFGcrUG0',
  }
}

Note the @ symbol notifying a virtual resource. virtual.pp looks like this:

define users::virtual ($uid,$realname,$pass,$userfiles = false) {

  user { $title:
    ensure => 'present',
    uid => $uid,
    gid => $title,
    shell => '/bin/bash',
    home => "/home/${title}",
    comment => $realname,
    password => $pass,
    managehome => true,
    require => Group[$title],
  }

  group { $title:
    gid => $uid,
  }

  # userfiles
  if $userfiles == false {
    # just create the directory
    file { "/home/${title}":
    ensure => 'directory',
    mode => '0755',
    owner => $title,
    group => $title,
    require => User["$title"]
    }
  } else {
    # copy in all the files in the subdirectory
    file { "/home/${title}":
    recurse => true,
    mode => '0755',
    owner => $title,
    group => $title,
    source => "puppet:///modules/users/${title}",
    require => User["$title"]
    }
  }

}

So for this manifest file, some parameters are passed from the init.pp, i.e. $uid,$realname,$pass . Also $userfiles is set to a default of false. However, for tony the init.pp overides this and sets it to true. The action performed by puppet is slightly different depending on if userfiles is true or false. If it’s true, a file (.bashrc) is copied from /etc/puppetlabs/code/environments/production/modules/users/files as well. As it happened I didn’t really need to use virtual resources but it was interesting experimenting with them (I was going to say fun but it took quite a bit of trial and error to get it to work so the fun part started to wear off…).

OK, don’t know about you but my brain is now in overload so I’ll leave it there. I did think of a new slogan for Puppetlabs, “Don’t be a muppet, use puppet”. I’ve emailed this to their marketing department. Still waiting to hear back….

Bye for now!


Leave a Reply