Puppet

relevant info from pkg_info:

• ruby-1.8.7.248_3,1 (compiled with debug)
• puppet-2.6.1.r2
• facter-1.5.7_1

uname -a
FreeBSD localhost 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Wed Jul 28 11:44:21 UTC 2010     root@localhost:/usr/obj/usr/src/sys/GENERIC  amd64

I did also see this problem in FreeBSD 8.0 and puppet 0.25. Puppet does not hang every time it’s run, only on random Exec statements. When debugging this I simply created an endless loop which ran the puppet configuration until it hang. Sometimes it would only run a few times before it hang and sometimes it run for hours. The manifest is quite big, but contains some sensitive stuff so I can’t upload it here, but this is probably reproducable with any manifest that uses a great deal of Exec statements.

A few examples of output when it hang with command ‘ruby18 —debug /usr/local/bin/puppet agent —test —debug —trace’:

/usr/local/lib/ruby/site_ruby/1.8/puppet/type.rb:1881: warning: instance variable @virtual not initialized
/usr/local/lib/ruby/site_ruby/1.8/puppet/type.rb:1881: warning: instance variable @virtual not initialized
/usr/local/lib/ruby/site_ruby/1.8/puppet/type.rb:1881: warning: instance variable @virtual not initialized
debug: /Stage[main]/Webbase::Backup/Line[sudo_backup]/Exec[echo 'backup  ALL= ( root )   NOPASSWD: /usr/sbin/backup' >> '/usr/local/etc/sudoers']: Executing check '/usr/bin/grep -qFx 'backup     ALL= ( root )   NOPASSWD: /usr/sbin/backup' '/usr/local/etc/sudoers''
debug: Executing '/usr/bin/grep -qFx 'backup     ALL= ( root )   NOPASSWD: /usr/sbin/backup' '/usr/local/etc/sudoers''
removing /tmp/puppet20100901-36732-16mgs1p-0...done
[here puppet hangs]

debug: /Stage[main]/Webbase::Apache/Exec[apache_purge]: Executing check '/bin/test -e /usr/local/etc/apache/ssl.crt'
debug: Executing '/bin/test -e /usr/local/etc/apache/ssl.crt'
removing /tmp/puppet20100901-50535-1p6j13j-0...done
[here puppet hangs]

/usr/local/lib/ruby/site_ruby/1.8/puppet/type.rb:1881: warning: instance variable @virtual not initialized
debug: /Stage[main]/Webbase::Nas/Exec[/sbin/mount /proc]: Executing check '/sbin/mount | /usr/bin/egrep ^procfs'
debug: Executing '/sbin/mount | /usr/bin/egrep ^procfs'
removing /tmp/puppet20100901-95974-1n5jg13-0...done 
[here puppet hangs]

Since I have very little experience debugging stuff, and no previous experience with ruby I just hacked in a few debug statements in type/exec.rb, and the relevant code appears to be this code from the run function:

          withenv environment do
            Timeout::timeout(self[:timeout]) do
              output, status = Puppet::Util::SUIDManager.run_and_capture(
                [command], self[:user], self[:group]
              )
              Puppet.debug "returned from run_and_capture #{self[:timeout]}"
            end
            Puppet.debug "timeout ended"
            # The shell returns 127 if the command is missing.
            if status.exitstatus == 127
              Puppet.debug "argument error"
              raise ArgumentError, output
            end
            Puppet.debug "no argument error"
          end

When puppet hangs it is after

Puppet.debug "returned from run_and_capture #{self[:timeout]}"

but before

Puppet.debug "timeout ended"

Since I don’t really have time to figure out how the timeout –> do –> end thing works I’ll just file this bug here so hopefully someone can say if this is a bug in puppet or an upstream bug in ruby. Let me know if you need any more info.