facter custom fact ruby files should be loaded deterministically
|Keywords:||Affected Facter version:|
When defining a set of custom facts via ruby libraries, the libraries are not loaded deterministically on different operating systems and/or filesystems. When a ruby-defined fact definition relies on the definition of other ruby-defined facts, the facts may not be evaluated depending upon the operating system / filesystem in use (e.g., Mac OSX by default appears to sort, while some variants of linux on ext3 appears to use another ordering).
The problem is that the return result from ruby’s Dir.entries varies from platform to platform.
The obvious fix is to sort the return value from Dir.entries before use.
A pull request with specs for the problem and a resolution is forthcoming.
#2 Updated by Daniel Pittman almost 3 years ago
Hey. Can you identify an actual problem caused by this, other than the order of facts changing when they are loaded?
My expectation would be that this should not matter – ordering dependencies between facts should be explicit, if they exist, rather than depending on an implementation detail like load order.
I would also expect that committing to loading facts in a specific order runs the risk that we are tied to providing that API forever, which means that any changes to the implementation later (like lazy evaluation, say) might suddenly run into the additional complexity of having to emulate this commitment to implementations…
Having a real world problem caused by that failure would make it much easier to work out if this is a commitment we should make. :)
#4 Updated by Michael Stahnke almost 3 years ago
Basically, we have N files with facts in them (not just one fact per file). Some facts in file 03-whatever.rb depend on files in 01-whatever.rb. Initially, we thought facter was loading the facts in filename order, after testing on some different systems, it was clear that it was not, and was actually dependent on inode ordering.
As for creating an API that you’re afraid might break something in the future, I think that’s how this needs to be. If I can’t rely on facter to use previous facts when making new ones, I would have to put all facts in one file and order them top down. This is pain when we already have a few dozen facts and certainly will have more by the time we are complete with our implementation.
For example, if I have facts doing some external lookup, I need to know the location of the URI to look something up, before I I try to grab a value form the URI.
The other option would be to have some formal dependency ordering with facts. Filename ordering certainly isn’t totally ideal, but also does not violate the principle of least surprise when working with custom facts.
#5 Updated by Nigel Kersten almost 3 years ago
- Status changed from Unreviewed to Investigating
- Assignee set to Nigel Kersten
First, thanks for the patch.
I’m worried we’re not addressing the actual problem here though.
If I can’t rely on facter to use previous facts when making new ones
Is that the real problem here? How are you referencing the dependent facts? Facter.value ?
#6 Updated by Michael Stahnke almost 3 years ago
Yes, I am using Facter.value(:fact_name) to load previous facts.
It seems like it might be possible to use Facter.load, but this could cause performance hits if we have to rescan facts several times in order to build out to the current dependency required, specifically if facts are doing external lookups. Yes, I could do some form of caching on external lookups, but if they only have to run once, it’s a much simpler problem to solve.
#8 Updated by Andrew Forgue almost 3 years ago
What if the fact depends on facts in different directories? How do you ensure those get loaded in the right order? IMHO, other facter issues aside I don’t think Facts should depend on load order, they should be located in a file with the name of the fact as the filename. Fact.value already searches the search path for factname.rb to load it.
#9 Updated by Paul Nasrat almost 3 years ago
The thinking for 2.0 is to completely rework the load system anyway, as noted by the reporter it is fragile and obtuse to fact developers.
However just sorting here doesn’t make sense – what we want is a well defined set of core facts with coherent meaning (see architecture fact discussion amongst others) and a reliable way of stating dependencies/loading.
However this patch does get us incrementally there – by adding a sort, we could potentially then add a custom sort to order the core first. I’d want to look at this in a bit more depth – Rick can you mail this to list using the instructions below, I find larger design discussions over email easier to deal with than a textbox in a redmine.