The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Bug #12140

puppetqd incorrectly updating hosts.ip of agent to IP of system puppetqd is running on

Added by Jacob McCann about 2 years ago. Updated about 2 years ago.

Status:AcceptedStart date:01/25/2012
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:queuing
Target version:-
Affected Puppet version: Branch:
Keywords:

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This ticket may be automatically exported to the PUP project on JIRA using the button below:


Description

I have recently started to use async_storeconfigs. My setup for this is I have puppetmaster, activemq and puppetqd all running on a system.

So what is happening is an agent performs a run. After the run the storeconfigs DB has the hosts.ip for the agent updated to the IP of the system that puppetqd is running on. If I turn async_storeconfigs off then the hosts.ip for the agent is updated to the correct IP of the agent itself.

I figured out that the incorrect IP being updated in hosts.ip from where puppetqd is running from because I took puppetqd off of the puppetmaster and ran it temporarily on another system to consume the queue from the puppetmaster. In this setup the IP changed from the IP of the puppetmaster (where puppetqd was previously running) to the new system running puppetqd.

I am using puppet 2.7.5 on SLES11.

History

#1 Updated by Daniel Pittman about 2 years ago

  • Category set to queuing
  • Status changed from Unreviewed to Needs More Information

Hey. Trying to work out what is going on here, I would appreciate a couple of bits more information from you:

Can you tell me the certname of one of the catalogs that is being improperly updated, and the certname of the machine that runs the Puppet queue daemon?

You can extract that with puppet agent --configprint certname

Can you also show puppet agent --configprint node_terminus on the queue machine only?

It looks like what is happening is that the queue daemon is getting node details that it thinks belong to the target node, but are really local, and is updating the request as a consequence. Those details will help track down the configuration causes of the problem.

#2 Updated by Jacob McCann about 2 years ago

Thanks for taking a look at this.

Here is the certname and node_terminus of one of the puppetqd/activemq/puppetmaster systems:

puppetmaster:~ # puppet agent --configprint certname
puppetmaster.domain.com
puppetmaster:~ # puppet agent --configprint node_terminus
plain

And here is the certname of one of my agents:

agent:~ # puppet agent --configprint certname
agent.domain.com

Let me know if you need anything else!

#3 Updated by Daniel Pittman about 2 years ago

Could you also give me puppet agent --configprint facts_terminus on the queue machine, please?

#4 Updated by Jacob McCann about 2 years ago

puppetmaster:~ # puppet agent --configprint facts_terminus 
facter

#5 Updated by Daniel Pittman about 2 years ago

  • Status changed from Needs More Information to Accepted

Awesome, thanks. That helps form my theory, which is that we are merging the local facts, from Facter, into that request incorrectly, and then using that to extract the IP.

The problem seems to be the plain node terminus invokes node.fact_merge, which grabs facts from the configured fact terminus – in this case Facter. That terminus doesn’t actually check the node name on the request, and will return local facts for any query.

Unfortunately, the details of that node get consumed to set the IP, leading to this problem.

#6 Updated by Jacob McCann about 2 years ago

This seems to be related to Issue #3923 which was rejected from no one reproducing. That was set to High, should this one be set to High as well? I’d really love to use async_storeconfigs but this really kind of stops us from doing so. Thanks!

#7 Updated by Cory Stoker about 2 years ago

I would like to see this fixed soon as well. Our environment has forced us to go to async configuration and having the data in hosts table incorrectly, especially the “environment” column causes us pain.

Also available in: Atom PDF