History

Puppet code reuse in complex environments

Introduction

Managing several identical or almost identical set of servers with a single puppetmaster is very easy. Things get progressively more difficult as heterogenuity of the environment increases. Complexity is increased by at least two factors:

  • Operating system differences
  • Multiple puppetmasters with site-specific code or files

Each of these is handled separately in the next subsections. It is assumed that puppet clients are at least on version 0.24.5 (Debian Lenny) and the server at least on 0.25.4 (Debian Lenny/backports).

Handling operating system differences

There are three basic strategies for handling operating system differences:

  • Using variables in node definitions
  • Adding conditionals to puppet classes and templates
  • Using variables in template and file names

Using variables in node definitions

Using variables in node definitions works well for differences one stumbles upon constantly. For example consider these two node definitions:

node 'server-linux' {
   $etc = "/etc"
   $admingroup = "root"
   include something
}

node 'server-freebsd' {
   $etc = "/usr/local/etc"
   $admingroup = "wheel"
   include something
}

These two variables, $etc and $admingroup are then referred to constantly in puppet code – especially in file resources:

    file { "main.cf":
            name => "$etc/postfix/main.cf",
            ensure => present,
            owner => root,
            group => $admingroup,
            mode  => 644,
            content => template("postfix/main.cf.erb"),
    }

As can be seen, use of a few well-chosen OS-specific variables can save us from writings tons of conditional code.

Adding conditionals to puppet classes and templates

Conditionals are suited well for handling occasional OS differences. e.g. for managing package names and configuration file templates, for example:

class postfix {
   package { "postfix":
      ensure => installed,
      name => $operatingsystem ? {
         "FreeBSD" => "mail/postfix25",
         default => "postfix",
      },
   }
}

Defining a separate variable for every package one wishes to install would clutter the node definitions badly, so use of a conditional makes more sense here. Similarly, use of conditionals in templates is occasionally very useful:

# Snippet from modules/postfix/templates/main.cf.erb
<% if operatingsystem == "FreeBSD" %>
alias_maps = hash:/etc/mail/aliases
alias_database = hash:/etc/mail/aliases
daemon_directory = /usr/local/libexec/postfix
command_directory = /usr/local/sbin
<% elsif operatingsystem == "RedHat" %>
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
daemon_directory = /usr/libexec/postfix
command_directory = /usr/sbin
<% else %>
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
daemon_directory = /usr/lib/postfix
command_directory = /usr/sbin
<% end %>

There’s really no clean way to handle the above with variable names. This approach works well with relatively complex configuration files that require quite a lot of OS-specific tailoring.

Using variables in file names

Sometimes the same puppet module needs to install files which have the same purpose on the target OS, but which have very different contents. This can happen if each OS is using a different daemon for the same purpose, say, to provide NTP services:

class ntp {
   file { "ntp.conf":
      ensure => present,
      name  => "$etc/ntp.conf",               
      owner => root,
      group => $admingroup,
      mode  => 644,
      source => "puppet:///ntp/ntp.conf.$operatingsystem",
   }
}

In this case the correct ntp.conf is selected based on the target operating system. Any number of variables can be chained for granularity. This approach works well if use of conditionals in templates would be impractical.

Handling multiple puppetmasters and site-specific content

The real test of portability and reusability comes when trying to use same puppet code for two sites with different puppetmasters: this creates a bunch of issues with non-generalized puppet code:

  • Site-specific data such as passwords or SSL keys leak to wrong servers, potentially causing breach of security
  • Modules breaking due to assumptions about the operating environment (e.g. hardcoded IP addresses)

The best solution is to write very generic puppet code and isolate site-specific code to separate classes. This approach is probably easiest if code is split into two directories as described here:

  • http://wiki.debian.org/MultipleSitePuppetmasterSetup

In this approach the global modules are simply generic resources for the puppetmaster: everything else is stored locally. This way most of the puppet code can be shared easily between puppetmasters. In practice you shouldn’t have more than a handful of local modules. For example, you could have something like this:

  • user_local: contains site-specific user account configuration
  • sshd_local: contains authorized SSH keys (e.g. for automated backups)
  • sslkeys_local: contains site and host-specific SSL certificates and keys (e.g. for authenticating clients)
  • puppetmaster_local: contains iptables rules to block puppet agent access from unauthorized IP addresses

Some modules will contains both site-specific and generic code. In this case the generic code should not depend on the site-specific code. Node definitions should include the local module, which in turn should include it’s corresponding generic part.