The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Keep your Puppet manifests under version control

It’s a good idea to keep your Puppet manifests and other configuration files under version control, for example Subversion or CVS. To do this, just set up the Puppet Master the way you want, and then import the whole of /etc/puppet into Subversion:

$ svn import /etc/puppet https://www.your-svn-server-here.com/svn/puppet/trunk

Once you have this imported, you will need to remove the files from /etc/puppet and check it out from your SVN repo, to make it a working copy, so that svn up /etc/puppet works.

$ svn co https://www.your-svn-server-here.com/svn/puppet/trunk /etc/puppet

In order to edit your puppet config, you can then check out a working copy somewhere else:

$ svn co https://www.your-svn-server-here.com/svn/puppet/trunk puppet

Edit it, then commit your changes, and update the master copy on the server:

$ svn up /etc/puppet

The Puppet master automatically detects that its configuration files have changed.

Remember when you create new certificates, you are modifying the Puppet master’s working copy, so you need to commit these changes every so often. This has the added benefit that if you should lose the Puppet master server, you can easily recreate it by just checking out a copy of the Puppet tree into /etc/puppet.

Configuration Versioning

From version 0.25.0, a new configuration option, config_version, is now available:

config_version = /usr/local/bin/return_version

The option allows you to specify a command that returns a version for the configuration that is being applied to your hosts. The command should return a string, such as a version number or name.

One such example of a command to return a version number would be:

svn info /etc/puppet/ 2>/dev/null | grep Revision: | egrep -o [0-9]+

or better (because it’s not require access to /root/.subversion/ from puppet user)

svnversion -n /etc/puppet/

Puppet then runs this command at compile time. Each resource is marked with the value returned from this command. This value is also added to the log instance, serialised and sent along with any report generated. This allows you to parse your report output and ascertain which configuration version was used to generate the resource.

Using Hooks

Hooks let you extend the value of Subversion (or Git, CVS, etc) to perform error checking, stage files and even produce audit trails.

SVN Hooks

SVN Pre-Commit Hook

To catch syntax errors and other basic problems, you can use a Subversion pre-commit hook like this:

#!/bin/sh
# SVN pre-commit hook to check Puppet syntax for .pp files
# Modified from http://mail.madstop.com/pipermail/puppet-users/2007-March/002034.html
REPOS="$1"
TXN="$2"
tmpfile=`mktemp`
export HOME=/
SVNLOOK=/usr/bin/svnlook
$SVNLOOK changed -t "$TXN" "$REPOS" | awk '/^[^D].*\.pp$/ {print $2}' | while read line
do
    $SVNLOOK cat -t "$TXN" "$REPOS" "$line" > $tmpfile
    if [ $? -ne 0 ]
    then
        echo "Warning: Failed to checkout $line" >&2
    fi
    puppet --color=false --confdir=/tmp --vardir=/tmp --parseonly --ignoreimport $tmpfile >&2
    if [ $? -ne 0 ]
    then
        echo "Puppet syntax error in $line." >&2
        exit 2
    fi
done
res=$?
rm -f $tmpfile
if [ $res -ne 0 ]
then
    exit $res
fi

If you get errors like this when committing:

/usr/lib/ruby/site_ruby/1.8/puppet/defaults.rb:102:in `handle': private method `split' called for nil:NilClass (NoMethodError)

It’s because the PATH environmental variable doesn’t exist. It doesn’t matter what it is, but puppet wants it to be there. Try adding PATH=“” somewhere in the top of the pre-commit script.

SVN Post-Commit Hook

Using a post-commit hook can be handy if you want your commits to automatically be seen by Puppet, e.g. you don’t have to do the last step shown above (svn up). Also, by integrating cvsspam you can provide an audit trail with nicely formatted, coloured diffs.

Here’s a simplified example of a post-commit that simply updates the files (previous checked-out) in /etc/puppet.

#!/bin/sh
REPOS="$1"
REV="$2"
svn up /etc/puppet

Of course much more can be done here, but that is a nice start.

Git Hooks

Git Update Hook

To catch syntax errors and other basic problems, you can use a server-side Git update hook like this:

#!/bin/bash

NOBOLD="\033[0m"
BOLD="\033[1m"
BLACK="\033[30m"
GREY="\033[0m"
RED="\033[31m"
GREEN="\033[32m"
YELLOW="\033[33m"
BLUE="\033[34m"
MAGENTA="\033[35m"
CYAN="\033[36m"
WHITE="\033[37m"

# V +1007

# Peff helped:
# http://thread.gmane.org/gmane.comp.version-control.git/118626

# For Puppet 0.25.x:
# syntax_check="puppet --color=false --confdir=/tmp --vardir=/tmp --parseonly --ignoreimport"
#
# For Puppet 2.7.x:
# syntax_check="puppet parser validate --ignoreimport"
#
# NOTE: There is an outstanding bug against `puppet parser` which causes
#       the --ignoreimport option to turn the syntax check into a no-op. Until
#       the bug is resolved, the syntax check hook should not include the
#       --ignoreimport option and will only work correctly on manifests which
#       do not contain "import" lines.
#       See http://projects.puppetlabs.com/issues/9670
#
syntax_check="puppet parser validate"
tmp=$(mktemp /tmp/git.update.XXXXXX)
log=$(mktemp /tmp/git.update.log.XXXXXX)
tree=$(mktemp /tmp/git.diff-tree.XXXXXX)

git diff-tree -r "$2" "$3" > $tree

echo
echo diff-tree:
cat $tree

exit_status=0

while read old_mode new_mode old_sha1 new_sha1 status name
do
  # skip lines showing parent commit
  test -z "$new_sha1" && continue
  # skip deletions
  [ "$new_sha1" = "0000000000000000000000000000000000000000" ] && continue
  # Only test .pp files
  if [[ $name =~ [.]pp$ ]]
  then
    git cat-file blob $new_sha1 > $tmp
    set -o pipefail
    $syntax_check $tmp 2>&1 | sed "s|$tmp|$name|"> $log
    if [[ $? != 0 ]]
    then
      echo
      echo -e "$(cat $log | sed 's|JOJOMOJO|'\\${RED}${name}\\${NOBOLD}'|')" >&2
      echo -e "For more details run this: ${CYAN} git diff $old_sha1 $new_sha1 ${NOBOLD}" >&2 
      echo
      exit_status=1
    fi
  fi
done < $tree

rm -f $log $tmp $tree
exit $exit_status

Git Pre-Commit Hook

To catch syntax errors and other basic problems, you can use a client-side Git pre-commit hook like this:

#!/bin/sh

syntax_errors=0
error_msg=$(mktemp /tmp/error_msg.XXXXXX)

if git rev-parse --quiet --verify HEAD > /dev/null
then
    against=HEAD
else
    # Initial commit: diff against an empty tree object
    against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

# Get list of new/modified manifest and template files to check (in git index)
for indexfile in `git diff-index --diff-filter=AM --name-only --cached $against | egrep '\.(pp|erb)'`
do
    # Don't check empty files
    if [ `git cat-file -s :0:$indexfile` -gt 0 ]
    then
        case $indexfile in
            *.pp )
                # Check puppet manifest syntax
                #git cat-file blob :0:$indexfile | puppet --color=false --parseonly --ignoreimport > $error_msg ;;
                # Updated for 2.7.x
                puppet parser validate $indexfile > $error_msg ;;
            *.erb )
                # Check ERB template syntax
                # -P : ignore lines which start with "%" 
                git cat-file blob :0:$indexfile | erb -P -x -T - | ruby -c 2> $error_msg > /dev/null ;;
        esac
        if [ "$?" -ne 0 ]
        then
            echo -n "$indexfile: "
            cat $error_msg
            syntax_errors=`expr $syntax_errors + 1`
        fi
    fi
done

rm -f $error_msg

if [ "$syntax_errors" -ne 0 ]
then
    echo "Error: $syntax_errors syntax errors found, aborting commit."
    exit 1
fi

Bazaar (bzr) Hooks

Bazaar Pre-Commit Hook

This is a client side pre-commit hook to catch basic syntax errors.

#!/usr/bin/env python
#
# BZR pre-commit hook, which will run some basic syntax checking on manifests
# in the current branch.
#
# To use this script, place it in your ~/.bazzar/plugins directory (create
# this directory if it doesn't exist).
#

from bzrlib import branch
import os
import sys
import subprocess

def get_branch_root(directory):
    """Find the root directory of the current BZR branch."""
    while os.path.exists(directory):
        if os.path.exists(os.path.join(directory, '.bzr')):
            return directory
        if directory == '/':
            break
        (parent, dir) = os.path.split(directory)
        directory = parent
    print "Commit FAILED:  Can't locate BZR Root."
    sys.exit(1)

def check_puppet_syntax(local, master, old_revno, old_revid, future_revno,
                       future_revid, tree_delta, future_tree):
    """This will run some basic syntax checking on the puppet manifests."""

    # Check syntax on changed files
    errors = []
    os.chdir(get_branch_root(os.getcwd()))
    print "\n" # make some space so we aren't clobbered by bzr's status msgs
    for file in tree_delta.added + tree_delta.renamed + tree_delta.modified:
        file = file[0]
        if file.endswith(".pp"):
            print "Checking syntax in: %s" % (file)
            try:
                process = subprocess.Popen(["puppet", "--parseonly", file],
                    stderr=subprocess.STDOUT, stdout=subprocess.PIPE)
                process.wait()
                if process.returncode != 0:
                    errors.append((file, ''.join(process.stdout.readlines())))
            except OSError, e:
                print "\n\n Error: failed to execute 'puppet': %s" % (e)
                sys.exit(1)
    if errors:
        print "\nSyntax errors were found:\n"
        for error in errors:
            print "%s: %s" % (error[0], error[1]),
        print "\nCommit FAILED"
        sys.exit(1)
    else:
        print "\nAll syntax checks PASSED"


# This is where the magic happens
branch.Branch.hooks.install_named_hook('pre_commit', check_puppet_syntax,
                                       'Check puppet manifests for syntax errors.')