Puppet

This has one drawback though: You’re not able to do a noop run (or at least you dont see what will really happen) because the prerequisites are never met.

One alternative I want to throw in here is to just let resources (and dependencies of course) fail when puppet cannot find a default provider.

That said you still have to run puppet twice but you’re at least able to run your manifest in the first place (instead of throwing and error “no default provider” and do nothing at all)

Stefan Schulte wrote:

This has one drawback though: You’re not able to do a noop run (or at least you dont see what will really happen) because the prerequisites are never met.

How come? You still pluginsync on noop runs (one reason why I really think we need to rename —noop to —simulate as it’s not true that there are “no operations”)

One alternative I want to throw in here is to just let resources (and dependencies of course) fail when puppet cannot find a default provider.

Isn’t that what we have now? And essentially a bunch of our problems are around a provider being synced to the client, but Puppet failing to recognize this and instead saying “no default provider” ?

That said you still have to run puppet twice but you’re at least able to run your manifest in the first place (instead of throwing and error “no default provider” and do nothing at all)

I’m pretty sure this is what we have now? I think I’m missing something Stefan, can you elaborate a bit more?

I wasn’t talking about pluginsync. I encountered a problem where a custom provider of mine uses the executables of an application I want to install with puppet at the same catalog run.

Unfortunately it is not only impossible to do in one run, I am just not able to apply the catalog at all.

Here’s a simple version of what I mean with a dummy type and provider and manifest:

/tmp/test/modules/lib/puppet/provider/foo/bar.rb
/tmp/test/modules/lib/puppet/type/foo.rb
/tmp/test/test.pp

Now I have the following type:

Puppet::Type.newtype(:foo) do
  ensurable

  newparam(:name) do
  end

end

And the following provider

Puppet::Type.type(:foo).provide(:bar) do
  commands :some_binary => '/tmp/foobinary'

  def exists?
    true
  end
  def create
  end
  def destroy
  end

end

Notice that the provider needs a special command. I want to install that command with puppet. Simple manifest:

foo { 'my_foo_resource':
  ensure => present,
  require => File['/tmp/foobinary']
}
file { '/tmp/foobinary':
  ensure => present,
}

What I want to do here is install the binary and use the a foo resource which should pickup the bar provider. Unfortunately I’m not able to apply the catalog

# puppet apply --modulepath=/tmp/test -v -d test.pp
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Puppet::Type::Foo::ProviderBar: file /tmp/foobinary does not exist
Could not find a default provider for foo
# ls -l /tmp/foobinary
ls: cannot access /tmp/foobinary: No such file or directory

Because I cannot apply the catalog I will never get the binary so all subsequent puppet runs will also fail.

The only way to get to my desired state is to first remove all foo types and apply the catalog. Then I have the binary on my node and I can put the foo type back.

In my opinion there are two solutions:

• install the binary during the first catalog run but let all foo resources fail. The foo resources are checked at a second run
• do both in the first run (some kind of lazy provider binding) but then I am not able to do a noop run anymore because puppet naturally cant show me what it would do with the foo resources if the binary is not yet present.

There is another manifestation of the same problem (at least I think): You might actually have a default provider for a given type on the target system, but the provider given in the manifest might not yet be functional. This will lead to

Could not run Puppet configuration client: Provider ... is not functional on this host at ...

with the same consequences as described above.

(Background: I am using vcsrepo and have f. e. Subversion installed, so a default provider is available. But installing git through Puppet and checking out a git repo in the same manifest is impossible, as “git is not functional” and the agent run fails before installing it.)

If I am not mistaken, the pip provider tries to work around this particular situation by not declaring “pip” as a command, but has a lazy_pip method that tries to check for the command as late as possible.

I wonder if that is a smart API compliant approach as I haven’t seen it in the few other providers I have looked at. If it is, shouldn’t it be implemented in a more fundamental place (provider.rb?) to keep provider implementations as straightforward as possible?

you should be able to declare your command with the optional_commands method instead of the commands method. optional_commands does not an confine :exists => your_command so your provider should not fail and work in the first puppet run. While I haven’t tried it myself I expect two problems

• your provider may be picked even if the package that is supposed to provide the needed binaries (in your case git) is not installed (no package resource in the manifest, failed installation or missing dependencies). This may be a bad idea if you have more than one provider for your type and another provider would have been functional.
• you may get a lot of failures if you do a noop run because now the binaries are definitely absent.

Besides the two possibe solutions I mentioned earlier: What about just skipping resources that do not have a suitable provider (instead of failure)?

Thanks Stefan – I will try this. I just wanted to point out that the two issues seem to be related regarding the cause and consequences.

I agree that skipping the resource is probably the best solution, albeit Puppet purists might disagree because it takes several runs to sort things out.

I have some problems with this patch. If you run in noop mode this raises the following error:

err: /Stage[main]//Foo[my_foo_resource]: Could not evaluate: Provider bar is not functional on this host at /tmp/test/test.pp:4

While I can live with that (I would more like a warning like »skip resource: no suitable provider found«) I think using a provider in a puppetrun eventhough prefetching did not work (because the provider was not yet present during prefetching phase) can cause serious errors because the provider will most likely assume that there are currently no instances present.

debug: Prefetching bar resources for foo
err: Could not prefetch foo provider 'bar': Command some_binary is missing
info: Applying configuration version '1320775712'
notice: /Stage[main]//File[/tmp/foobinary]/ensure: created
debug: /Stage[main]//File[/tmp/foobinary]: The container /tmp/foobinary will propagate my refresh event
debug: /tmp/foobinary: The container Class[Main] will propagate my refresh event
debug: Puppet::Type::Foo::ProviderBar: Executing '/tmp/foobinary '
notice: /Stage[main]//Foo[my_foo_resource]/ensure: created

So I still prefer the “skip all resources with no suitable provider” approach.

Stefan Schulte wrote:

I have some problems with this patch. If you run in noop mode this raises the following error: […]

While I can live with that (I would more like a warning like »skip resource: no suitable provider found«) I think using a provider in a puppetrun eventhough prefetching did not work (because the provider was not yet present during prefetching phase) can cause serious errors because the provider will most likely assume that there are currently no instances present.

[…]

So I still prefer the “skip all resources with no suitable provider” approach.

I’m a touch confused. We are skipping resources with no suitable provider, we’re just doing it with an error (which causes dependencies to also be skipped) rather than with a warning (which would not skip dependencies).

I’m less concerned about unsuitable providers having instances. Suitable providers downloaded in the same run are actually prefetched; it’s only unsuitable providers that aren’t.

The major point of this bug is that you need to be able to download the plugins and use them in the same run; if you download, say, a gem provider and gems are already present, then the provider will be suitable and there’s no problem. If you download and it’s not suitable until Puppet installs rubygems, then you somewhat axiomatically don’t have any ruby gems already on the system. Yes, there’s a chance that installing the package system also installs packages and that you want to manage those at the same time, but that seems quite unlikely, and a significant amount of extra work for a very rare edge case.

I don’t think that the situation where an unsuitable provider has instances is that unlikely.

Take the gentoo portage provider for example. It depends on eix to install packages. eix is an optional tool, that is recommended (because it allows easier queries and is faster) but not necessarily present. But you cannot expect that you do not have any packages installed if you do not find eix on a system. I guess that is true for a lot of “helper programs”

Another example: I want puppet to

1. install an application
2. set configuration variables for that application with a custom type/provider. The application provides an executeable to do that

If my provider uses prefetching but the application is not yet installed, the provider will assume an empty configuration. But that is not true because the installation of the application already sets certain options.

This approach concerns me as well. While it looks like it will prevent Puppet from blowing up because of missing commands, I also share Stefan’s concerns.

It’s more complex change, but it seems like a better over-all solution would be to move checking for provider suitability and pre-fetching to as late as possible (at the time we attempt to apply the first resource of that type/provider combination). If the (or a) provider is not available at this time then move the resource off into a “retry these later” list, repopulating the ready list from this “do it later” list once the ready list has been exhausted (re-checking for suitability as we try again to re-evaluate the resources). On every pass through the ready list we would still move things off into the “do it later” list, re-filling the ready list once it’s exhausted until we have evaluated all resources (Yay!), or detect that we are no longer removing resources from the ready list by marking them as done. If we’ve detected that we’re no longer removing resources form the ready list by marking them as done then the resources should fail as they would have before with a message about no suitable providers being available.

If we lazily evaluate suitability and move prefetching to after we’ve lazily determined that a particular provider is suitable, then it should take care of Stefan’s concerns, allow people to have puppet install non-standard packages needed by providers, and take care of the case where things are provided by pluginsync.

I’m happy to prefer your solution over mine if you can get it ready in time for the next bugfix release.

Otherwise, I prefer fixing it now and doing it the right way when it’s worth the investment.

Along similar lines to Luke’s comment, the more complex solution does look like The Right Way™, but I’d much rather we made incremental improvement than waiting for the more complex solution to arrive.

I’m naturally not against incremental improvement but the current fix contains a fundamental change:

Without the fix you can define confines to say »don’t pick me (the provider) if condition A and B are not true« and in the provider code I can take my conditions for granted. With the fix applied (correct me if I am wrong) puppet may pick you (the provider) even if not all confines are met in case puppet does not find any suitable one.

And what bothers me is that the provider may now have to handle situations where the inital confines are not true. One example is prefetching but there are other possibilties (e.g. a confine :exists => '/my/file' which suddenly may not be true).

Just wanted to make that clear.

That’s a change, but it’s in line with user expectations, so I think it makes sense. Users can already select an unsuitable provider (to support exactly this kind of lazy suitability evaluation), and it makes more sense to lazily fail if the provider isn’t suitable later in the transaction than at the start.

Unsuitable providers will still never be used, this just allows them to be selected (in the off chance they become suitable within the transaction).

As to true conditions becoming false, that’s already a problem, isn’t it?

At the risk of adding noise, I’d like to ask if Luke’s patch aims at the problem outlined in comment 4 to comment 6.

Probably I am missing the big picture, but AFAIK that has nothing to do with downloading providers/using pluginsync. Instead Stefan and I would like to be able to use a provider (that might already have been downloaded in the past) in the same run as installing its commands.

There are two other issues #10882 and #10440 where we worked around the issue by making the commands optional. This cannot be the right solution however – instead of porting this fixing pattern everywhere there must be a more centralized approach which at least I was expecting to come up in this case.

Can someone please clarify on this? If I am in the wrong case here, I’d be glad to open another one for it.

Matthias Pigulla wrote:

At the risk of adding noise, I’d like to ask if Luke’s patch aims at the problem outlined in comment 4 to comment 6.

Probably I am missing the big picture, but AFAIK that has nothing to do with downloading providers/using pluginsync. Instead Stefan and I would like to be able to use a provider (that might already have been downloaded in the past) in the same run as installing its commands.

The short answer is yes, my patch fixes those issues. It’s being summarized as a pluginsync problem, because we’re expecting you to pluginsync a provider down, install the commands, and use it, all within the same run. My patch (and the better fix Jacob is working on) fix that case, which means they also fix the simpler case of installing the commands and using the provider at the same time.

There are two other issues #10882 and #10440 where we worked around the issue by making the commands optional. This cannot be the right solution however – instead of porting this fixing pattern everywhere there must be a more centralized approach which at least I was expecting to come up in this case.

Can someone please clarify on this? If I am in the wrong case here, I’d be glad to open another one for it.

This is the right case. It might be worth testing that my fix actually works for you, but you could also wait until Jacob’s fix is ready.

Luke — thanks for clarification. I’ll stand by and see if I can help :–).

Although I double-checked Stefan’s simple example doesn’t work with your feature branch. Making the command optional solves it.

Matthias Pigulla wrote:

Luke — thanks for clarification. I’ll stand by and see if I can help :–).

Although I double-checked Stefan’s simple example doesn’t work with your feature branch. Making the command optional solves it.

Can you paste the failure with —trace —debug?

Luke Kanies wrote:

Matthias Pigulla wrote:

Luke — thanks for clarification. I’ll stand by and see if I can help :–).

Although I double-checked Stefan’s simple example doesn’t work with your feature branch. Making the command optional solves it.

Can you paste the failure with —trace —debug?

The problem looks like Stefan’s manifest doesn’t specify that the file /tmp/foobinary ought to be executable. So even though the file exists, it doesn’t satisfy the requirements for the command being present, and the provider never becomes suitable. It works if the file resource specifies mode appropriately.

Nick Lewis wrote:

Luke Kanies wrote:

Matthias Pigulla wrote:

Luke — thanks for clarification. I’ll stand by and see if I can help :–).

Although I double-checked Stefan’s simple example doesn’t work with your feature branch. Making the command optional solves it.

Can you paste the failure with —trace —debug?

The problem looks like Stefan’s manifest doesn’t specify that the file /tmp/foobinary ought to be executable. So even though the file exists, it doesn’t satisfy the requirements for the command being present, and the provider never becomes suitable. It works if the file resource specifies mode appropriately.

whew

I was aware of the chmod +x thing. The problem was that Ruby included code from site-ruby, not the test checkout. Not being a Ruby developer, what is the best way to make sure only code from a particular dir is included?

So sorry – Luke’s patch seems to solve it. Also seems to work for the case in comment 5.

Oh, before I forget – the two cases I mentioned above where there have been workarounds in external modules should probably have the fixes reverted when this is released. How to make sure that happens…? ;–)

Hooray!

+1 … awesome dudes.