The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

This issue tracker is now in read-only archive mode and automatic ticket export has been disabled. Redmine users will need to create a new JIRA account to file tickets using https://tickets.puppetlabs.com. See the following page for information on filing tickets with JIRA:

Bug #1254

puppetd client randomly hangs

Added by Redmine Admin almost 8 years ago. Updated over 5 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Nigel Kersten% Done:

0%

Category:Debian
Target version:-
Affected Puppet version:0.24.4 Branch:
Keywords:

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com


Description

This one has been a hard one tracking down but maybe someone else can shed some light on it

puppetd clients occasionally hang. and by hang i mean, no longer do config runs

the process still appears active:

ps aux | grep puppet

root 23419 0.0 0.6 27008 17024 ? Ss May21 0:02 ruby /usr/sbin/puppetd -w 0

if i do an strace on the process, it doesnt appear like anything is borked

strace -p 23419

Process 23419 attached – interrupt to quit select(8, r7, [], [], NULL

by looking at the syslog daemon logfile

i can see the last thing it was doing was a run: puppetdr23419: Starting Puppet client version 0.24.4

Client os: debian sarge and etch puppetd is started in an init script using: puppetd -w 0 with the following puppetd.conf

/etc/puppet/puppet.conf: [main] logdir=/var/log/puppet vardir=/var/lib/puppet ssldir=/var/lib/puppet/ssl

[puppetd] server = mypuppetserver.com report = true factsync = true factsignore = .svn

[puppetmasterd] templatedir=/var/lib/puppet/templates


Related issues

Related to Puppet - Bug #2888: puppetd doesn't always cleanup lockfile properly Needs More Information 12/04/2009
Related to Puppet - Bug #504: Puppetd lock files seem to be staying around Closed
Related to Puppet - Feature #190: Clear lockfile if the associated PID no longer exists Closed
Related to Puppet - Bug #4153: notice: Run of Puppet configuration client already in pro... Rejected 07/07/2010

History

#1 Updated by AJ Christensen almost 8 years ago

Cool!

Haven’t experienced this one myself – anyone else?

#2 Updated by Redmine Admin almost 8 years ago

  • Status changed from 1 to Needs More Information
  • 6 changed from Needs more info to Needs more information

#3 Updated by Andrew Shafer almost 8 years ago

  • Affected Puppet version set to 0.24.4

It’s hard to work on occasionally, can anyone reproduce this consistently?

I’m working on performance and memory problems for 0.25

Would love more info

#4 Updated by Trevor Vaughan about 7 years ago

Do you happen to be managing something to do with user accounts?

I’ve noticed that the 0.24.6 puppet daemon will hang if it can’t find user account information or if the package type hangs.

#5 Updated by Marcin Deranek about 7 years ago

Not sure if this is the same scenario as we see, but here’s why things might break..

We use puppet 0.24.6 on CentOS/RedHat 4/5 machines. When puppet is running (doing stuff) it creates puppetdlock file. If it happens that during puppetd activity you decide to restart it (or stop + start) the script will issue TERM signal to the daemon, it will wait some time (~5 seconds) and issue KILL signal. In such situation puppetd will be killed, but puppetdlock file will stay. After (re)start puppetd won’t work due to existence of the file. How much time does puppetd needs to be stopped by TERM:

bash# kill cat /var/run/puppet/puppetd.pid; while pgrep puppetd > /dev/null; do echo -n ‘x’; sleep 1; done; echo xxxxxxxxxxxxxxxxxxxxx

Why it takes so long ? I don’t know..

#6 Updated by Luke Kanies about 7 years ago

gringo wrote:

Not sure if this is the same scenario as we see, but here’s why things might break..

We use puppet 0.24.6 on CentOS/RedHat 4/5 machines. When puppet is running (doing stuff) it creates puppetdlock file. If it happens that during puppetd activity you decide to restart it (or stop + start) the script will issue TERM signal to the daemon, it will wait some time (~5 seconds) and issue KILL signal. In such situation puppetd will be killed, but puppetdlock file will stay. After (re)start puppetd won’t work due to existence of the file. How much time does puppetd needs to be stopped by TERM:

bash# kill cat /var/run/puppet/puppetd.pid; while pgrep puppetd > /dev/null; do echo -n ‘x’; sleep 1; done; echo xxxxxxxxxxxxxxxxxxxxx

Why it takes so long ? I don’t know..

It should generally not need much time at all to stop after a TERM, but if it is in the middle of a run it will try to finish that run, which can take some time.

#7 Updated by Marcin Deranek about 7 years ago

luke wrote:

It should generally not need much time at all to stop after a TERM, but if it is in the middle of a run it will try to finish that run, which can take some time.

In our case puppet manages itself, so it is very likely that puppet will try to restart itself. Another case is when you use ‘splay=true’ and start the daemon. Till the first run it also takes ~20 sec to stop the daemon with TERM.

#8 Updated by James Turnbull almost 7 years ago

  • Assignee deleted (Puppet Community)

#9 Updated by Nigel Kersten over 5 years ago

  • Status changed from Needs More Information to Closed

If anyone still sees this behavior, feel free to reopen.

As per the below thread, we’re more aggressively closing tickets whose state is unsure, particularly old tickets with little to no inactivity in a long time.

You are free to reopen them.

http://groups.google.com/group/puppet-users/browse_thread/thread/a040cb9bc5c5b647

#10 Updated by Nigel Kersten over 5 years ago

  • Assignee set to Nigel Kersten

Also available in: Atom PDF