The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Introduction

This page describes the squid recipe I’ve developed as a relatively new Puppet user. I am eager for feedback and suggestions for improvement.

In particular, I can see myself using the check_file function over and over again, and I wonder whether there is a better way to include it in the template (as a library module), and whether there’s a way to avoid setting the squiddir and basedir variables (or at least have them adjust relatively intelligently for other modules).

Thanks

Kudos to the Puppet community for being a great place to ask questions. In particular QMan and joe-mac1 on the #puppet IRC channel have put up with lots of stupid questions. QMan’s suggestion of IO.read() set me on the path of realising I needed to understand and exploit the potential of Ruby in templates.

Background

My requirements were to support a relatively small, reasonably homogeneous network of CentOS 5.2 systems which had been administered separately for quite some time. The two main configuration files i needed to manage were squid.conf and a url_regex ACL file called nopasswordsites.txt (for listing sites which do not require proxy authentication to access). So I gathered the existing configuration files, reduced the differences to the minimum possible, and allowed for arbitrary additions to the configuration.

squid class

Here’s the /etc/puppet/modules/squid/manifests/init.pp for the squid class which manages the two configuration files:

#
# puppet class to manage squid.conf and nopasswordsites.txt
#

class squid {
    # directory on the puppet server where configurations are kept
    $squiddir = "/etc/puppet/modules/squid"

    # ensure package is installed
    package { "squid": ensure => installed }

    # call this with appropriate parameters to define squid.conf
    define squid_conf( $visible_hostname = $fqdn, $emulate_httpd_log = "on",
            $cache_size = "256", $squiddir = $squiddir) {
        file { "/etc/squid/squid.conf":
            ensure  => file,
            owner   => root,
            group   => root,
            mode    => 644,
            content => template("squid/squid.conf/MASTER.erb"),
            require => Package["squid"],
        }
    }

    # squid's no password url regex file
    file { "/etc/squid/nopasswordsites.txt":
        ensure  => file,
        owner   => root,
        group   => squid,
        mode    => 644,
        content => template("squid/nopasswordsites/MASTER.erb"),
    }

    # reload squid when the configuration file changes
    service { "squid":
        require     => Package["squid"],
        subscribe   => File["/etc/squid/squid.conf"],
        restart     => "/usr/sbin/squid -k reconfigure",
    }

}

Node definition

Here’s a snippet of /etc/puppet/manifests/nodes.pp which instantiates the squid configuration:

node 'mynode.example.com' {
        $isp = "iinet"

        include basenode
        include squid

        squid::squid_conf { "DUMMY NAME":
                cache_size => 10000,
        }

}

squid.conf template

Here’s the master squid.conf template (/etc/puppet/modules/squid/templates/squid.conf/MASTER.erb). Most of it is just the distro’s default squid.conf with the comments stripped out.

##############################################################################
# Squid configuration file
##############################################################################
# NOTE: This file is automatically generated by puppet on <%= servername %>.
# Changes to this file will be overwritten periodically by puppet!
##############################################################################

# Setup
<%
        def check_file( name )
                basedir = squiddir + "/templates/squid.conf/"
                includeheader = "#####\n# Included file: \n# " + basedir + name + "\n#####\n"
                includefooter = "#####\n# End of included file: \n# " + basedir + name + "\n#####\n"
                if File.exists?( basedir + name ) then
                        return  includeheader + IO.read( basedir + name ) + includefooter
                end
        end
%>

<%= check_file( hostname + ".header") %>


#################################
# Global configuration section
#################################

http_port 3128
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY

cache_dir ufs /var/spool/squid <%= cache_size %> 16 256

emulate_httpd_log <%= emulate_httpd_log %>
access_log /var/log/squid/access.log squid
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443 563     # https, snews
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

#################################
# nopasswordsite file inclusion
#################################

acl NoPasswordSites url_regex "/etc/squid/nopasswordsites.txt"
http_access allow NoPasswordSites
acl LAN_SRC src 192.168.0.0/255.255.0.0
acl passwd proxy_auth REQUIRED

# ...

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow passwd
http_access allow LAN_SRC
http_access allow localhost
http_access deny all
http_reply_access allow all
icp_access allow all
visible_hostname <%= visible_hostname %>
follow_x_forwarded_for allow localhost
acl_uses_indirect_client on
delay_pool_uses_indirect_client on
log_uses_indirect_client on
coredump_dir /var/spool/squid

<%= check_file( hostname + '.footer') %>

nopasswordsites.txt template

Here’s the template for the nopasswordsite file (/etc/puppet/modules/squid/templates/nopasswordsites/MASTER.erb):

##############################################################################
# squid ACL definition for sites which do not need authentication to browse
##############################################################################
# NOTE: This file is automatically generated by puppet on <%= servername %>.
# Changes to this file will be overwritten periodically by puppet!
##############################################################################

##############################################################################
# NOTE: This file should not be used for a general whitelist.  It is only for
# essential services that require access to web sites via squid WITHOUT 
# AUTHENTICATION (e.g. username/password).
# 
# Really the only things on this list should be:
# - operating system updates and patches
# - anti-malware updates (including anti-virus and anti-spyware)
# - updates for applications which don't support proxy authentication
# - internal sites with their own authentication
#
# For content filtering bypasses, use Dan's Guardian instead.
##############################################################################

<%
        def check_file( name )
                basedir = squiddir + "/templates/nopasswordsites/"
                includeheader = "#####\n# Included file: \n# " + basedir + name + "\n#####\n"
                includefooter = "#####\n# End of included file: \n# " + basedir + name + "\n#####\n"
                if File.exists?( basedir + name ) then
                        return  includeheader + IO.read( basedir + name ) + includefooter
                end
        end
%>

<%= check_file( "COMMON-permanent.txt" ) %>
<%= check_file( "COMMON-temporary.txt" ) %>

<% if defined? isp then %>
        <%= check_file( "ISP-" + isp + ".txt" ) %>
<% end %>

<%= check_file( hostname + ".permanent.txt" ) %>
<%= check_file( hostname + ".temporary.txt" ) %>

Other components

The nopasswordsites.txt files are just lists of domain names. Here’s an example COMMON-permanent.txt:

# operating system updates
centos.org
planetmirror.com
planetmirror.com.au
redhat.com
update.microsoft.com
windowsupdate.microsoft.com

# anti-malware
avg.com
ca.com
clamav.net
download.lavasoft.de
ftpav.ca.com
ftp.ca.com
grisoft.com
grisoft.cz
lavasoft.de
lavasoftsupport.com
liveupdate.symantecliveupdate.com
mcafee.com
safer-networking.org
symantecliveupdate.com

Example ISP-iinet.txt:

iinet.net.au

Example mynode-temporary.txt:

gov.au
edu.au