History

Using Puppet with Augeas

Augeas is a lovely tool that treats config files (well, anything really, but it’s mostly about config files) as trees of values. You then modify the tree as you like, and write the file back.

For example, the “PermitRootLogin” setting in sshd_config can be referenced like this:

$ augtool print /files/etc/ssh/sshd_config/PermitRootLogin
/files/etc/ssh/sshd_config/PermitRootLogin = "yes"

And it can be modified:

$ augtool
augtool> set /files/etc/ssh/sshd_config/PermitRootLogin no
augtool> save
Saved 1 file(s)
$ grep PermitRootLogin /etc/ssh/sshd_config
PermitRootLogin no

This is basically the solution to the problem of dealing with upstream configuration changes combined with local modifications: you can allow the upstream changes through and then apply changes with Augeas to the new version.

Assuming you want to work with Augeas, this is a description of how to perform Augeas changes using Puppet. You’ll need Puppet >= 0.24.7 for this. The basic usage is in the Type Reference. The command-line utility augtool will be used to demonstrate things here, but it is not required to use Augeas within Puppet. On most Linux distributions, augtool can be found in a separate package.

The somewhat more important, and unfortunately complicated, part is figuring out what the tree for a file looks like so you can manipulate it properly. The definition that Augeas uses to turn a file into a tree is called a lens, and understanding the trees is more difficult than it should be, because many lenses are not documented sufficiently, or at all. The documentation for those that are has its own surprisingly hard to find page on the Augeas site. You can see what lenses are available by looking in /usr/share/augeas/lenses/ (or /usr/local/share/augeas/lenses/, or possibly somewhere else, depending on your setup).

You can see which files Augeas has successfully parsed by running augtool ls /files/ and drilling down from there. If a file hasn’t been properly parsed by Augeas, it simply won’t show up. This could mean that the file has a syntax error, the file doesn’t exist, you don’t have permission to read the file, or it could imply a failure in the lens itself.

The easiest way to understand how Augeas handles a particular file is to examine it using augtool ls and/or augtool print. In fact, you might want to make your changes by hand in a text editor, then use augtool print to give you an idea what the final result should look like.

Here’s an example of how to determine the tree structure of a file, in this case /etc/exports. This is based on examples from from the bottom of man 5 exports:

$ augtool
augtool> ls /files/etc/exports/
comment[1] = /etc/exports: the access control list for filesystems which may be exported
comment[2] = to NFS clients.  See exports(5).
comment[3] = sample /etc/exports file
dir[1]/ = /
dir[2]/ = /projects
dir[3]/ = /usr
dir[4]/ = /home/joe

From here you can investigate the structure, like so:

augtool> ls /files/etc/exports/dir[1]
client[1]/ = master
client[2]/ = trusty

The corresponding line in the file is:

/   master(rw) trusty(rw,no_root_squash)

Digging further:

augtool> ls /files/etc/exports/dir[1]/client[1]
option = rw

So, to add a new entry, you’d do something like this:

augtool> set /files/etc/exports/dir[last()+1] /foo
augtool> set /files/etc/exports/dir[last()]/client weeble
augtool> set /files/etc/exports/dir[last()]/client/option[1] ro
augtool> set /files/etc/exports/dir[last()]/client/option[2] all_squash
augtool> save
Saved 1 file(s)

Which creates the line:

/foo weeble(ro,all_squash)

Now that we’ve seen some examples in augtool, let’s make the same changes using Puppet.

Here’s the sshd_config example:

augeas { "sshd_config":
  changes => [
    "set /files/etc/ssh/sshd_config/PermitRootLogin no",
  ],
}

The Augeas resource in Puppet has a useful attribute called “context” which allows you to specify a root for all of your changes. This is especially nice when making many changes to the same file. The above could also be written like this:

augeas { "sshd_config":
  context => "/files/etc/ssh/sshd_config",
  changes => [
    "set PermitRootLogin no",
  ],
}

Note that values containing whitespace need to be quoted. In this example, we use single-quotes since the set statement itself is already enclosed in double-quotes. You could also do the opposite.

"set kernel.sem '500 512000 64 1024'",

Here’s the /etc/exports example:

augeas{ "export foo" :
    context => "/files/etc/exports",
    changes => [
        "set dir[last()+1] /foo",
        "set dir[last()]/client weeble",
        "set dir[last()]/client/option[1] ro",
        "set dir[last()]/client/option[2] all_squash",
    ],
}

This adds the line as described above. In fact, it works too well. It will add the line every time Puppet runs. This is one of the biggest gotchas to using Augeas in Puppet. It’s very easy to create resources that repeat changes on every run. (Technically, it will stop when you run out of space on that filesystem.) In almost every case, if you use a path containing last(), you will also need to use the Augeas resource’s “onlyif” attribute to keep it under control.

In the above example, you could add something like this:

onlyif => "match dir[. = '/foo'] size == 0",

Which essentially says “only add it if it’s not already there”. The problem with this approach is that it only considers one thing (the name of the export in this case). If you were to change something else like “client” later, it would never get applied because an entry named “/foo” is found. You can combine multiple tests into the “onlyif” attribute to look for changes in all the various components, but then you are practically defining the entire resource twice. There’s a better way.

A Better Way

In many cases, if you set a value for a non-existent path, Augeas will create the path for you. By using a more dynamic path to refer to things, you can make sure changes to any part of the tree will get applied, and you can do it without a messy “onlyif”.

Since the share name is unique in /etc/exports you can use that to refer to a particular entry. In the example above, it would be /files/etc/exports/dir[. = '/foo'].

Here’s an improved version of the example above:

augeas{ "export foo" : 
    context => "/files/etc/exports",
    changes => [ 
        "set dir[. = '/foo'] /foo",
        "set dir[. = '/foo']/client weeble",
        "set dir[. = '/foo']/client/option[1] ro",
        "set dir[. = '/foo']/client/option[2] all_squash",
    ],
}

This has several advantages.

  • If the entry doesn’t exist at all, it will get created.
  • If the entry already exists and you change any value, it will get updated.
  • The line won’t be added to /etc/exports on every single Puppet run.
  • You don’t need to figure out an “onlyif” attribute to control Augeas because there is no “onlyif” attribute.

The next section examines this technique in detail.

Paths for Numbered Items

Note: This is based on experience and trial & error, not some deep understanding of Augeas or studying of documentation, so terminology might not be 100% correct and there could be a better/simpler way, so always experiment on your own. You may want to look this over, then come back here for some practical examples.

Changing things like sshd_config or Postfix’s main.cf (where you have a series of simple key=value pairs) is pretty straightforward. Unfortunately, not every configuration item in a file has a unique “left hand side” to use in the path. In situations like this, Augeas will assign numbers to each item.

There are three ways things might get numbered. We’ll show a working example for each (and some limitations).

  1. The items themselves are numbers

     # augtool ls /files/etc/hosts
 1/ = (none)
 2/ = (none)

  2. A simple array of items with the same name

     # augtool ls /files/etc/sudoers
 spec[1]/ = (none)
 spec[2]/ = (none)

  3. An array of items with a value assigned

     # augtool ls /files/etc/exports
 dir[1]/ = /foo
 dir[2]/ = /bar

Starting with /etc/hosts, say you want to make sure the “localhost” entry also contains the system’s current hostname and FQDN.

augtool> ls /files/etc/hosts
#comment[1] = Do not remove the following line, or various programs
#comment[2] = that require network functionality will fail.
1/ = (none)
2/ = (none)
3/ = (none)
4/ = (none)

Which one of these is the line we want? It’s most likely /files/etc/hosts/1, but you can’t be sure. Fortunately, Augeas lets us identify a unique path using components of the item at that path. Here’s the complete entry we want to change, referenced by number:

augtool> print /files/etc/hosts/1
/files/etc/hosts/1
/files/etc/hosts/1/ipaddr = "127.0.0.1"
/files/etc/hosts/1/canonical = "localhost"
/files/etc/hosts/1/alias[1] = "webserver1"
/files/etc/hosts/1/alias[2] = "webserver1.domain.com"

Here’s the same entry, picked out by matching the “ipaddr” component:

augtool> print /files/etc/hosts/*[ipaddr = '127.0.0.1']
/files/etc/hosts/1
/files/etc/hosts/1/ipaddr = "127.0.0.1"
/files/etc/hosts/1/canonical = "localhost"
/files/etc/hosts/1/alias[1] = "webserver1"
/files/etc/hosts/1/alias[2] = "webserver1.domain.com"

This not only works for listing and printing, but for changing values as well.

augtool> set /files/etc/hosts/*[ipaddr = '127.0.0.1']/alias[1] webserver2

So our Puppet manifest could maintain our desired “localhost” entry without knowing the order of the lines in the file:

augeas { "localhost":
  context => "/files/etc/hosts",
  changes => [
    "set *[ipaddr = '127.0.0.1']/canonical localhost",
    "set *[ipaddr = '127.0.0.1']/alias[1] $hostname",
    "set *[ipaddr = '127.0.0.1']/alias[2] $hostname.domain.com",
  ],
}

Note that the above example assumes a line for “127.0.0.1” is already defined.

With sudoers, you could add a simple entry by matching on the value for “user”.

augeas { "sudojoe":
  context => "/files/etc/sudoers",
  changes => [
    "set spec[user = 'joe']/user joe",
    "set spec[user = 'joe']/host_group/host ALL",
    "set spec[user = 'joe']/host_group/command ALL",
    "set spec[user = 'joe']/host_group/command/runas_user ALL",
  ],
}

It might seem strange to set the value for “user” by matching “user”, but it works. You just have to be sure “user” is defined first in this example so the rest of the values can be assigned. (This is a simplified example. Since the user alone doesn’t necessarily uniquely identify an entry in sudoers, you should be careful matching on that alone.)

We’ve already seen how to handle an array of items with values, like /etc/exports, but just to be clear, when the item itself has a value, you can match it using “.”.

augtool> print /files/etc/exports/dir[. = '/foo']

Wherever possible, try to find a unique path that can be used to define entries and avoid “onlyif”. If you get stuck, refer to the Path Expressions section of the Augeas Wiki.

Limitations

In some cases, (/etc/sudoers, /etc/services) there may not be a single item that makes an entry unique, but you can combine them to ensure you’re targeting the right one. For most things in /etc/services, there are two lines (for TCP and UDP) with the same name and port.

augtool> print /files/etc/services/service-name[. = 'ssh']
/files/etc/services/service-name[23] = "ssh"
/files/etc/services/service-name[23]/port = "22"
/files/etc/services/service-name[23]/protocol = "tcp"
/files/etc/services/service-name[23]/#comment = "SSH Remote Login Protocol"
/files/etc/services/service-name[24] = "ssh"
/files/etc/services/service-name[24]/port = "22"
/files/etc/services/service-name[24]/protocol = "udp"
/files/etc/services/service-name[24]/#comment = "SSH Remote Login Protocol"

augtool> print /files/etc/services/service-name[port = '22']
(same output as above)

But we can target just one line by using multiple components:

augtool> print /files/etc/services/service-name[port = '22'][protocol = 'tcp']
/files/etc/services/service-name[23] = "ssh"
/files/etc/services/service-name[23]/port = "22"
/files/etc/services/service-name[23]/protocol = "tcp"
/files/etc/services/service-name[23]/#comment = "SSH Remote Login Protocol"

You can use complicated paths like this to modify existing values, but unfortunately, they can’t be used to create new entries. Remember, with sudoers we had to set “user” first in order to match on it in subsequent lines. To match on two values they both need to be set, and you can’t set them at the same time, which is why this only works for things that already exist.

In cases like this, you will need to revert back to using a combination of last() and “onlyif”, until http://projects.puppetlabs.com/issues/6494 is released. At that point you can create the whole tree and them move it into place.

Working Examples

You can test a manifest containing and Augeas resource like this:

sudo puppet apply --verbose --debug --trace --summarize test.pp

/etc/sysctl.conf

works on puppet 2.6.1; fedora 12; ubuntu 10.04

class sysctl {

  # nested class/define
  define conf ( $value ) {

    # $name is provided by define invocation

    # guid of this entry
    $key = $name

    $context = "/files/etc/sysctl.conf"

     augeas { "sysctl_conf/$key":
       context => "$context",
       onlyif  => "get $key != '$value'",
       changes => "set $key '$value'",
       notify  => Exec["sysctl"],
     }

  } 

   file { "sysctl_conf":
      name => $operatingsystem ? {
        default => "/etc/sysctl.conf",
      },
   }

   exec { "sysctl -p":
      alias => "sysctl",
      refreshonly => true,
      subscribe => File["sysctl_conf"],
   }

}

use case:

include sysctl

sysctl::conf { 

  # prevent java heap swap
  "vm.swappiness": value =>  0;

  # increase max read/write buffer size that can be applied via setsockopt()
  "net.core.rmem_max": value =>  16777216;
  "net.core.wmem_max": value =>  16777216;

}

/etc/security/limits.conf

works on puppet 2.6.1; fedora 12; ubuntu 10.04

class limits {

  # nested class/define
  define conf (
    $domain = "root",
    $type = "soft",
    $item = "nofile",
    $value = "10000"
    ) {

      # guid of this entry
      $key = "$domain/$type/$item"

      # augtool> match /files/etc/security/limits.conf/domain[.="root"][./type="hard" and ./item="nofile" and ./value="10000"]

      $context = "/files/etc/security/limits.conf"

      $path_list  = "domain[.=\"$domain\"][./type=\"$type\" and ./item=\"$item\"]"
      $path_exact = "domain[.=\"$domain\"][./type=\"$type\" and ./item=\"$item\" and ./value=\"$value\"]"

      # TODO add duplicate entry cleanup

      augeas { "limits_conf/$key":
         context => "$context",
         onlyif  => "match $path_exact size==0",
         changes => [
           # remove all matching to the $domain, $type, $item, for any $value
           "rm $path_list", 
           # insert new node at the end of tree
           "set domain[last()+1] $domain",
           # assign values to the new node
           "set domain[last()]/type $type",
           "set domain[last()]/item $item",
           "set domain[last()]/value $value",
         ],
       }

  } 

}

use case:

include limits

limits::conf { 

  # maximum number of open files/sockets for root
  "root-soft": domain => root, type => soft, item => nofile, value =>  9999;
  "root-hard": domain => root, type => hard, item => nofile, value =>  9999;

}

sshd_config

Configure sshd:

augeas { "sshd_config":
  context => "/files/etc/ssh/sshd_config",
  changes => [
    # track which key was used to logged in
    "set LogLevel VERBOSE",
    # permit root logins only using publickey
    "set PermitRootLogin without-password",
  ],
  notify => Service["sshd"],
}

service { "sshd":
  name => $operatingsystem ? {
    Debian => "ssh",
    default => "sshd",
  },
  require => Augeas["sshd_config"],
  enable => true,
  ensure => running,
}

Default Runlevel

Set the default runlevel to 3:

augeas { "runlevel":
  context => "/files/etc/inittab",
  changes => [
    "set id/runlevels 3",
  ],
}

Kernel Boot Options

Suppose a vendor (like VMWare) has some recommended kernel parameters that you want to apply. Assuming you have a class that only applied to VMWare guests, you could do the following.

# improve time keeping for VMs
case $hardwareisa {
  "i386": {
    # not going to worry about these
  }
  default: {
    augeas { "vmtime":
      context => "/files/etc/grub.conf",
      changes => [
        "set title[1]/kernel/divider 10",
        # clear sets the left hand side but assigns no value
        "clear title[1]/kernel/notsc",
      ],
    }
  }
}

Adding a service that uses both TCP and UDP to /etc/services

Augeas needs a unique identifier when using set. So the trick to adding a service that uses both TCP and UDP is to use child nodes of the service. For example, to add zabbix-agent you can use this (note the [2] for the second zabbix-agent):

augeas { "zabbix-agent":
   context =>  "/files/etc/services",
   changes => [
      "ins service-name after service-name[last()]",
      "set service-name[last()] zabbix-agent",
      "set service-name[. = 'zabbix-agent']/port 10050",
      "set service-name[. = 'zabbix-agent']/protocol tcp",
      "ins service-name after /files/etc/services/service-name[last()]",
      "set service-name[last()] zabbix-agent",
      "set service-name[. = 'zabbix-agent'][2]/port 10050",
      "set service-name[. = 'zabbix-agent'][2]/protocol udp",
   ],  
   onlyif => "match service-name[port = '10050'] size == 0",
}

Works on CentOS 5.5 with Puppet 2.6 and Augeas 7.2-2