The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com
Flexible, deterministic, unguessable application order
|Assignee:||Markus Roberts||% Done:|
|Affected Puppet version:||Branch:||MarkusQ:feature/next/resource_application_order|
Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com
This issue is currently not available for export. If you are experiencing the issue described below, please file a new ticket in JIRA. Once a new ticket has been created, please add a link to it that points back to this Redmine ticket.
We would like to change the order in which resources are applied to meet a very specific set of constraints:
- The order should permit responsive adaptation to externalities (graph frontiers)
- The order in which resources that do not depend on externalities should be consistent (deterministic) from run to run provided that no structural changes occur in the manifest
- Changes in the ordering due to manifest changes should be limited in scope and correspond to the areas that were changed (that is, adding/removing/changing one resource should not affect the relative order of any unrelated pair of resources)
- It should not be possible to guess the ordering a priori — this mechanism is not a substitute for declaring dependencies.
- It should not violate any implicit or explicit dependencies
#1 Updated by R.I. Pienaar over 3 years ago
This is a good summary, I’d like to add to it:
- Remove parser functions like defined()
- Remove access to the list of loaded classes in scope
- Remove access to list of tags in the scope
Really remove all the bits of the language that remains weirdly order broken and that simply cannot work at all till the futures work is completed at which point their design should be re-evaluated.
#3 Updated by Markus Roberts over 3 years ago
- Branch set to MarkusQ:feature/next/resource_application_order
Core change up for review. Note that is not a resolution, just a key way-point on the route to a resolution.
(6911) Core change -- replace topsort with frontier ordered by salted SHA1 This is the core change of the ticket; rather than using a topological sort to statically determine the order in which resources should be applied, we use the graph wrapper introduced in the prior commit to dynamically determine the order in which to apply resources based on 1) the status of the resource (ready, done) 2) the explicit & implied dependencies, 3) the salted SHA1 of the title (for stability). Further work is needed: 1) Resolving the handling of failed resources 2) Tests of the new behavior, to the extent possible 3) Newly-dead-code removal in simple_graph & transaction 4) Fix the name-prefix ordering hack in eval_generate by either: a) Moving the logic into file b) Refactoring Type#eval_generate to return a tree c) ....? 5) Rough performance testing to look for hotspots 6) Investigation of possible interaction with #3788, #5351, #5414, #5876, #6020, #6810, and #6944 which may simplify or complicate their resolution. Paired-with: Jesse Wolfe
#5 Updated by R.I. Pienaar over 3 years ago
Markus Roberts wrote:
R.I. it’s late Friday afternoon, and I’m being a little dense. Could you elaborate on how the points you mentioned are related to this ticket? I’m deep enough in the trees on this that I suspect I’m not always seeing the forest.
As we’re overhauling how ordering works and doing lots of work in this area to avoid certain pitfalls I thought i’d mention the other closely related ones to ordering that we are not addressing – but that doesnt really work at the moment.
defined() is parsing order dependent for example so it just doesn’t do what its advertised to do. I realize this ticket is more about sorting the graph than it is about parsing ordering.
Feel free to take it as an aside or punt it to other tickets etc, I just hate the way functionality that exist now doesnt work as a user might expect and its ordering related and we’re not fixing that functionality. I’d favor removing broken functions then.