Puppet

I’ve been dreaming of functionality like this for a while, but I haven’t been able to envisage how it would work.

From the server, you don’t necessarily know what repositories are visible to the client, and you may indeed be creating these repositories as part of your puppet run, so you can’t do something where the client sends a package dependency graph up to the server at the start of the run.

This would be a killer feature though…

Well, the theory of execution was this:

1) Server passes compiled manifest to client 2) Client loads manifest into memory 3) Client executes all yumrepo/whatever statements 4) Client begins to execute 4a) All repo types are executed (no sense in doing them later) 5) Client searches tree for all ‘package’ objects (perhaps set these aside in compiled manifest as an object tree?) 5a) Client rebuilds execution tree based on known packages, links, and package tree 6) Execution starts from the top of the repo as usual.

Alternatively:

1) Server compiles manifest and selects package types 2) Server sends package request to client 3) Client returns package tree to server 4) Server builds entire tree with in-memory package types and dependencies

In both cases, similar package names are grouped. This means taking architectures, etc… into account.

It should work but, then, I haven’t had time to hack it, so what do I know :-P.

So repos can only be added by a whitelist of types to get this functionality? You’d preclude repos being created by file or exec or defined types that don’t use the whitelisted types?

What if a package is required to be installed before the repositories? (Say a repository key signature)

What if someone is distributing preferences that impact upon the dependency resolution? Or anything else for that matter.

The reason I want this is to solve this kind of problem I’ve run into:

A requires B in the packaging system.

package { A: ensure => installed, }

package { B: ensure => installed, }

file { foo:
  before => Package[B],
}

This doesn’t do what you want, because there’s no relationship between Package[A] and File[foo] as far as Puppet is aware, so you’ll get unpredictable results where sometimes Package[B] (via Package[A]) will be installed before File[foo], but not always. This is really frustrating to debug, and will only become more of an issue as people share modules more.

I do think this would be a useful feature though, and it would make lots of things possible. You could have Puppet fail if you provide it inconsistent manifests, like ensuring two conflicting packages are both installed.

Is there a link to the archives for this proposal somewhere? I’m curious about how the package relationship graphs are communicated from each client to the master.

This ticket is it Nick.

We’ve been rather busy recently, and this hasn’t been a particularly high priority task.

I have similar questions. I’m worried about the relationship graph changing during the Puppet run, making it difficult to be satisfied that the correct graph is being merged into the Puppet-manifest specified graph if we only upload that graph at the start of the run.

If we defer it to the client evaluation stage, and merge client-side, then the server doesn’t have the actual version of the catalog that the client is applying, which also seems less than ideal.

I guess I’ll re-flog the proper subgraph horse.

I was thinking about this and, in theory, stages are sort of subgraphs but don’t quite optimize to the single point graph level.

If you abstract this concept and always have a subgraph/stage called “packages” or the like, then you could ensure that the rest of the compilation is properly formed and you’re authorizing the client to make changes to this one subgraph which can only contain “package” resources and upon which all internally generated package resources will automatically belong.

So, you end up moving all packages to their own stage and preserving the linking of the internal components appropriately. Then, the client can add internal dependencies to meet the requirements of the package manager.

Alternatively, I suppose that you could add a client-only type called ‘autopackage’ or something that cannot be declared in a manifest but which can be programatically added by the client at runtime. It would be a proper subclass of ‘package’ and the client would only accept modifications to the runtime catalog of that type.

I honestly don’t see the issue with the master not knowing what precise manifest the client is applying. To me it’s akin to using ‘exec’ as a black hole, the master has absolutely no idea what happened in an exec, it’s up to the admin. Likewise, if you’re using YUM and it pukes, Puppet has no idea what happened except that it failed. At least, with the ‘autopackage’ resource, you would know that it was a problem with an automatically added resource instead of a normal package.

The actual problem applies to a broader field than just packages, though they are fairly commonly the cause of this, in that the content of recursive file operations, tidy operations, and some other types and providers that have internal dependency information do the same thing. Finally, this is already done for arbitrary types using the autorequire mechanism, on the client side.

In view of that, just integrating the data is pretty fine, although it SHOULD be done together with getting the final graph back up to the master in the report, so that users have some visibility of the whole process.

All in all, still valuable, and absolutely to be done, but not presently scheduled.