The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com
https://tickets.puppetlabs.com. See the following page for information on filing tickets with JIRA:
undefined method `has_openstack_mac?' for Facter::Util::EC2:Module
|Keywords:||Affected Facter version:||1.6.6|
Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com
This is on CentOS 6.2 w/ Puppet 2.6.14-1.el6 both from the Puppet yum repository.
The problem is only occurring in daemon mode of puppet.
This is in the syslog repeatedly:
Feb 29 08:14:55 hostname puppet-agent: Could not run Puppet configuration client: Could not retrieve local facts: undefined method `has_openstack_mac?' for Facter::Util::EC2:Module
I’ve checked on several systems and it appears to be a bug introduced in 1.6.6-1. The last successful run in the logs contains:
Feb 28 14:55:18 hostname puppet-agent: (/Stage[puppet]/Facter/Package[facter]/ensure) ensure changed '1.6.5-1.el6' to '1.6.6-1.el6'
#1 Updated by zach armstrong about 4 years ago
I am seeing this too in RHEL5 with puppet 2.7.11.
It doesn’t seem to occur every time puppet makes a run. I have not been able to figure out how to reliably repro it.
This error seems to be due to facter 1.6.6. Problem started from this commit: https://github.com/puppetlabs/facter/commit/cb598aab3e36767abd6ce5827e47099ecb49ef67#diff-0
If you want to revert these two files, you can grab them from here: https://github.com/puppetlabs/facter/blob/1df5b46e0ec0bd8a42254b571ca6eb1acd814ed6/lib/facter/util/ec2.rb https://github.com/puppetlabs/facter/blob/1df5b46e0ec0bd8a42254b571ca6eb1acd814ed6/lib/facter/ec2.rb
#2 Updated by Ken Barber about 4 years ago
- Category set to library
- Status changed from Unreviewed to Investigating
- Assignee set to Ken Barber
So I’m trying to understand this, as it doesn’t appear that the code is doing anything abnormal here, it feels like either a library loading issue or perhaps resident files (either from an old install or an alternative install). But alas, I need more information.
You say here:
The problem is only occurring in daemon mode of puppet.
Now I found this pastie with a log of the issue:
In that particular case the problem seems obvious, the agent has upgraded facter – the util class & the fact has changed, but because the util class doesn’t automatically reload on its own, we are seeing a missing method. A reload of the puppet agent should solve this.
Really in that kind of scenario – the agent needs to be reloaded. I would expect this to be the normal process for a facter or puppet upgrade – ie. always restart the agent. Otherwise, libraries that have changed (like the EC2 library) will still reflect the old revision in memory.
So my question is – have you tried restarting the agent? And does that alleviate the problem?
#4 Updated by Ken Barber about 4 years ago
So I can’t replicate it yet, I’m putting it on a 30 second run cycle to see if I can replicate it in time.
- If you run just ‘facter’ or ‘facter -p’ on their own, do you have a problem? Does the EC2 fact work as expected? (ie. do you get ec2_* entries in facter). Of course if you are not running this on Amazon, then you probably won’t.
- Can you run ‘puppet agent -t’ and does it work/fail?
- How easy is this to replicate for yourself? Is running puppet in agent mode now completely broken?
Also – can you show me the outcome of the following:
- rpm -qa | grep facter
- grep ‘has_openstack_mac?’ ‘/usr/lib/ruby/site_ruby/1.8/facter/util/ec2.rb’
- find / -name ‘ec2.rb’
(the third item will take quite some time I presume)
Of course my objective right now is to replicate the problem, to help me understand the fault … so anything else you can think of would be great.
#5 Updated by Chris Möser about 4 years ago
I have the same issue running Ubuntu 10.04 LTS server. On top I have a duplicate server running the same version(s)/packages not showing this error. On that one rig the command
puppet agent --test --verbose returns the error. The client puppet.conf looks like this:
[main] logdir=/var/log/puppet vardir=/var/lib/puppet ssldir=/var/lib/puppet/ssl rundir=/var/run/puppet factpath=$vardir/lib/facter templatedir=$confdir/templates [master] # These are needed when the puppetmaster is run by passenger # and can safely be removed if webrick is used. ssl_client_header = SSL_CLIENT_S_DN ssl_client_verify_header = SSL_CLIENT_VERIFY [agent] server=server.domain.tld
which is pretty much unmodified. The same configuration was running pre-update to the current version. If there is any further information you need, just ask. If there is any command to run, just drop a line.
facter -p just drops the same error on this erroneous machine. It doesn’t generate any further output like the other machine when I run that command.
The service on this machine can be established with
service puppet start after which it is present in the process list. The error occurs when I run it manually to verify correct processing of new manifests.
#7 Updated by Steve Traylen about 4 years ago
Also observed this on RHEL6 from EPEL pkgs to puppet lab packages.
- Update facter-1.6.5-1.el6 to facter-1.6.6-1.el6
- Update Puppet 2.6.13-2.el6 to 2.7.11-2.el6
The following happened:
- The puppet deamon was running with 2.6.13-2.el6 and first puppetrun updated puppet (before facter)
- This SIGTERMS puppet ‘Caught TERM; calling stop’ so that puppet run stops.
- 15 minutes later puppet runs again and does the facter update.
- 15 minutes later puppet runs again and spits out the EC2 error.
- 15 later same as last step and repeat.
For me anyway restarting the puppet daemon was enough for everything to settle down, all seems well now… Guessing there is confusion between the facter at puppet startup and facter on disk?
#8 Updated by Orion Poplawski about 4 years ago
For me this is triggered when running puppet in daemon mode and facter was upgraded from 1.6.5 to 1.6.6. I filed https://bugzilla.redhat.com/show_bug.cgi?id=806370 against Fedora for it suggesting that the factor package restart puppet agent daemon on upgrade. That fixes the problem, but we’re wondering if that is the right thing to do in general or if this was an unexpected change introduced in 1.6.6 and unlikely to re-occur.
Perhaps the puppet agent daemon needs to be changed to reload the facter modules when it starts a run rather on startup?
#9 Updated by Todd Zullinger about 4 years ago
In particular, I’m unsure if adding ‘service puppet condrestart’ is going to cause us pain. If we restart puppet and it is mid-run, classes could be left partially updated. I’m unclear how resilient puppet is to stopping the agent while it is in the middle of a run. I know there were some tickets on what it should do in this case, but I’m not sure if those are resolved. It’s a tricky issue.
#13 Updated by Charlie Sharpsteen about 3 years ago
May not be the full story, but this is reproducible when running facter on a machine with multiple versions installed—-similar to what Chris Moser reported. For example, on Ubuntu with Facter installed from the PuppetLabs APT repository alongside a local git clone running under
root@ubuntuagent:~# facter Error: undefined method `kernel_fact_value' for Facter::Util::Processor:Module
Then if I remove the version installed through APT:
root@ubuntuagent:~# apt-get remove facter Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libshadow-ruby1.8 libaugeas0 ruby-json libruby ruby libaugeas-ruby1.8 hiera libjson-ruby augeas-lenses Use 'apt-get autoremove' to remove them. The following packages will be REMOVED: facter puppet puppet-common ...and so on root@ubuntuagent:~# facter No LSB modules are available. architecture => i386 augeasversion => 0.10.0 ...many more facts
And if I re-install Facter, the problem returns:
root@ubuntuagent:~# apt-get install facter Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: facter ...blah blah Setting up facter (1.6.17-1puppetlabs1) ... root@ubuntuagent:~# facter Error: undefined method `kernel_fact_value' for Facter::Util::Processor:Module
Similar behavior occurs on CentOS with yum. So, this definitely looks like some sort of library loading issue.