The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

This issue tracker is now in read-only archive mode and automatic ticket export has been disabled. Redmine users will need to create a new JIRA account to file tickets using https://tickets.puppetlabs.com. See the following page for information on filing tickets with JIRA:

Feature #9546

Plugin sync support for external facts

Added by Ken Barber over 4 years ago. Updated over 2 years ago.

Status:ClosedStart date:05/19/2012
Priority:NormalDue date:05/19/2012
Assignee:-% Done:

0%

Category:-
Target version:3.4.0
Affected Puppet version: Branch:
Keywords:external facts

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com


Description

So now we have external facts (#2157) we would want to be able to synchronise these somehow using pluginsync.


Related issues

Blocked by Puppet - Bug #4135: pluginsync tries to parse readme files Closed 07/02/2010
Follows Facter - Feature #2157: External fact support Closed 05/18/2012 05/18/2012

History

#1 Updated by James Turnbull over 4 years ago

  • Category set to interface
  • Status changed from Unreviewed to Needs Decision
  • Assignee set to Nigel Kersten

#2 Updated by Nigel Kersten over 4 years ago

  • Status changed from Needs Decision to Accepted
  • Assignee deleted (Nigel Kersten)

I totally agree we should be able to sync them somehow… but I note we’ve just committed a change to only pluginsync .rb files…

#3 Updated by Ken Barber over 4 years ago

Nigel Kersten wrote:

I totally agree we should be able to sync them somehow… but I note we’ve just committed a change to only pluginsync .rb files…

Your talking about #4135? It will still sync other files – just won’t load them. This is definitely a precursor to this ticket (hence why its on the dependency list). So its all good :–).

#4 Updated by Nigel Kersten over 4 years ago

brilliant :)

#5 Updated by R.I. Pienaar over 4 years ago

it would be good to find some way to have the ability to copy some files to only some hosts from pluginsync. for facts created with yaml/json that would be needed.

#6 Updated by Nigel Kersten over 4 years ago

Hrm. Now that’s a little more problematic…

RI, do you think a client-side filter is sufficient or does this control need to be mandated centrally?

#7 Updated by R.I. Pienaar over 4 years ago

Nigel Kersten wrote:

Hrm. Now that’s a little more problematic…

I know, I consider it a personal challenge to keep your life interesting :P

RI, do you think a client-side filter is sufficient or does this control need to be mandated centrally?

hmm, i can’t think of a reason client side wouldnt be ok right now, but will give it some more thought – however imagine a big site with lots of facts unique to hsots etc, we could be doing a lot of file copies down to the client – and file copies are slow as heck.

#8 Updated by Nigel Kersten over 4 years ago

gah :)

#9 Updated by Ken Barber over 4 years ago

  • Target version set to 186

#10 Updated by Anonymous about 4 years ago

  • Target version deleted (186)

#11 Updated by Knight Samar over 2 years ago

Can’t we leave the problem of putting the right fact on the right host, to the Puppet user ? Or to be more specific, the problem of the fact doing the right thing on a host

For example, if I am writing a piece of Python code which generates a fact, I should take care that it does the right thing on the right host, just like I would do in a Ruby custom fact.

At least support for .py and other such external executable, can be added.

#12 Updated by John Julien over 2 years ago

Capturing a discussion from the mailing list on this topic. There were 3 questions discussed about the implementation of this feature, see the details below.

Question 1: What is the justification for this feature? Why is it needed? Right now if a person is using an external fact that is fully puppet controlled they are creating a file resource in Puppet to get it to the client. This means the fact won’t actually be available during the catalog compile until the 2nd Puppet run. It usually causes errors in the 1st Puppet run or undesired behavior around the portion of code that requires that fact. This is a clumsy implementation.

Many people using Puppet are sysadmins who all are very familiar with shell, probably some python and perl in there too. Ruby is not as common of a language in the sysadmin tool box. Allowing users to write facts in a language they already know brings Puppet one step closer to a level people already understand which increases the adoption rate.

Question 2: Where will the external facts reside in the module path? The external facts will reside in a facts.d/ directory at the base of the module. So an example tree would be:

mymodule
├── facts.d
│   └── my_external_fact.sh
├── lib
│   └── facter
│       └── custom_ruby_fact.rb
└── manifests
    └── init.pp

Question 3: Will there be a way to confine which facts are synchronized to what hosts for compatibility reasons? Several suggestions were made and discussed at length.

  1. Sync all facts to all clients and fail silently if a fact could not be evaluated
  2. Have a folder structure based on kernel type that would sync facts to hosts with kernels matching the directory name
  3. Have an optional “.confine” file for each fact that uses s-exrpessions to define a list of conditions that would have to be met for the fact to be synchronized. The conditions would be based on other already existing facts
  4. Take a simplified approach to number 3 and have the confine only evaluate equality of values for existing facts. So “osfamily: RedHat” would be allowed, but no regex or && or || operators.
  5. Adjust Facter to not execute files with extensions [exe, com, bat] on unix systems (Currently these are the only extensions executed on Windows). Sync all facts to all systems. Cross platform compatibilty is up to the programmer of the fact to fail silently if need be by returning nil or perform different operations depending on the OS it is being run on.

The discussion concluded that 1 was too passive, 2-4 were too complex, so 5 was the best option.

The full details of the conversation can be found here: https://groups.google.com/forum/#!topic/puppet-dev/jKgqWO0y4H0

#13 Updated by Adrien Thebo over 2 years ago

  • Project changed from Facter to Puppet
  • Category deleted (interface)

Since this is mainly a matter of how Puppet could pluginsync external facts and has less to do with Facter, I’m moving this to the Puppet project.

#14 Updated by Anonymous over 2 years ago

  • Status changed from Accepted to Merged - Pending Release
  • Target version set to 3.4.0

There were several different efforts around getting this to work. The initial functionality was merged into master in 19d28e1. After a few small fixes made once it started running through the entire test suite it got to some more exploratory testing and had another issue uncovered: the PMT stripped the execute permissions making bundling external fact scripts useless. This was fixed in merge e84ddb3.

#15 Updated by Anonymous over 2 years ago

Another fix that corrects problems on solaris and windows. Merged in c5abb6

#16 Updated by Melissa Stone over 2 years ago

  • Status changed from Merged - Pending Release to Closed

Released in Puppet 3.4.0-rc1

#17 Updated by Melissa Stone over 2 years ago

Released in Puppet 3.4.0-rc1

#18 Updated by Melissa Stone over 2 years ago

Released in Puppet 3.4.0-rc1

#19 Updated by Melissa Stone over 2 years ago

Released in Puppet 3.4.0-rc1

#20 Updated by Melissa Stone over 2 years ago

Released in Puppet 3.4.0-rc1

#21 Updated by Melissa Stone over 2 years ago

Released in Puppet 3.4.0-rc1

#22 Updated by Josh Cooper over 2 years ago

A fix for excluding .bat, .cmd, etc files on non-windows platforms was made in facter in commit 20a2de6e

Also available in: Atom PDF