Puppet

Problem here seems to be (in my humble opinion), that the package type and provider don’t support specifying an architecture for the package, so unless you want to ensure a specific version, everything works fine. But if you try to specify both architecture (in the package name) and version (via the ensure parameter), the package provider at least for yum works out a wrong package name+arch+version (instead of name+version+arch) for the installation (note: If you install the package manually, puppet detects the installed version fine and thus doesn’t try to reinstall the package).

I would propose adding an architecture parameter to the package type.

Here is an anonymized log for this:

debug: Puppet::Type::Package::ProviderYum: Executing '/bin/rpm -q boost-thread.x86_64 --nosignature --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}
'
debug: //Node[some-node.example.com]/virtual_packages::some-class::build/Package[boost-thread.x86_64]: Changing ensure
debug: //Node[some-node.example.com]/virtual_packages::some-class::build/Package[boost-thread.x86_64]: 1 change(s)
debug: Package[boost-thread.x86_64](provider=yum): Ensuring => 1.39.0-6
debug: Puppet::Type::Package::ProviderYum: Executing '/usr/bin/yum -d 0 -e 0 -y install boost-thread.x86_64-1.39.0-6'
debug: Puppet::Type::Package::ProviderYum: Executing '/bin/rpm -q boost-thread.x86_64 --nosignature --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}
'
err: //Node[some-node.example.com]/virtual_packages::some-class::build/Package[boost-thread.x86_64]/ensure: change from absent to 1.39.0-6 failed: Could not update: Could not find package boost-thread.x86_64 at /etc/puppet/manifests/classes/virtual_packages.pp:1323

Some lazy fix that works at least for me. I guess it will break on some Red Hat packages that do include a dot in their normal package name … (patch is against yum.rb, btw)

64,65c64,73
<             # Add the package version
<             wanted += "-%s" % should
---
>             # Check if there's an architecture in the pkg name
>             # FIXME: Do a real check for known architectures instead of lazy matching
>             if wanted =~ /(.*)\.(.*)/
>                 pkg = $1
>                 arch = $2
>                 wanted = "%s-%s.%s" % pkg, should, arch
>             else
>                 # Add the package version
>                 wanted += "-%s" % should
>             end

Thanks Patrick.

That’s logical.

Is this logic in 0.25.0?

David Ward wrote:

Is this logic in 0.25.0?

That’s puppet 0.24.8 in RHEL 5.3 (puppet-0.24.8-1.el5.1).

Patrick Otto wrote:

Some lazy fix that works at least for me. I guess it will break on some Red Hat packages that do include a dot in their normal package name … (patch is against yum.rb, btw)

[…]

Does this mean no-one will be able to install any of the java or openjdk packages with puppet?

R.I. Pienaar aka Volcane wrote:

Does this mean no-one will be able to install any of the java or openjdk packages with puppet?

Yes, my patch is a quick&dirty workaround for the setup over here. I do not recommend using this patch at all. It’s just a pointer.

I think the suggested fix is spot on; clearly, we need to put the version in between the package name and the arch. We could intrdoduce a new arch parameter for that, but that seems more trouble than it’s worth.

The one thing I would change about the patch is to restrict what suffixes we’d consider to be an arch. The canonical list is in arch.py in the yum source, but something like the following should work pretty well:

# Check if there's an architecture in the pkg name
if wanted =~ /(.*)\.(noarch|i[3-6]86|x86_64|ppc|ppc64|sparc.*|s390.*)/
  pkg = $1
  arch = $2
  wanted = "%s-%s.%s" % pkg, should, arch
else
  # Add the package version
  wanted += "-%s" % should
end

I honestly think that simply listing the architectures in the RegExp is not the right way to go. Do you really want to keep updating that list of architectures whenever some yum-using Distribution decides to support some new architecture? Imagine someone porting RedHat-like stuff to the amount of architectures currently supported by Debian. I am quite convinced that providing an architecture parameter would be the right way. Especially when considering that at least Debian is also working on going multiarch (soon), so also the apt/dpkg providers (and rpm in any case, I would think) will need to know which architecture they are supposed to work on.

I agree, arch => would be best.

If we say for yum we need name.arch and for debian probably we’d need some other kludgy naming convention on the package name to achieve the same and ditto for other operating systems.

Adding arch => solves it elegantly and the providers can do the right thing with the information for their OS, no need to try and come up with Regexes etc.

There’s one problem with moving the architecture from the package name to a new arch parameter: as Puppet is today, it makes it difficult to install multiple architectures of a package.

One might allow an array as arch parameter, but that still makes it difficult to manage the different packages from different parts of the manifests.

Thomas Bellman wrote:

There’s one problem with moving the architecture from the package name to a new arch parameter: as Puppet is today, it makes it difficult to install multiple architectures of a package.

One might allow an array as arch parameter, but that still makes it difficult to manage the different packages from different parts of the manifests.

Thanks Thomas for reminding me of why I didn’t do that right off the bat – I couldn’t remember why I initially dismissed an arch parameter, but I believe that was it.

David Lutterkort wrote:

Thomas Bellman wrote:

There’s one problem with moving the architecture from the package name to a new arch parameter: as Puppet is today, it makes it difficult to install multiple architectures of a package.

One might allow an array as arch parameter, but that still makes it difficult to manage the different packages from different parts of the manifests.

Thanks Thomas for reminding me of why I didn’t do that right off the bat – I couldn’t remember why I initially dismissed an arch parameter, but I believe that was it.

Problem here is: As puppet is today, it doesn’t make it hard to install multiple architectures of the same package. To the contrary: It makes it hard to install only a single architecture. Because with RedHat, if you don’t specify an architecture, it will install all available architectures of the package. And this behaviour along with the wish to control the version of the package that is being installed is what caused us stumbling over this bug.

And to be honest: If I want to install two different architectures of the same package (I don’t know any CPU architecture that would be able to natively support more than two package architectures), I still want them to be of the same version (or handled the same at that).

Though I understand the difficulty (having two package definitions that would have the same title if you minimize the amount of text in the manifest), I would still favour having an arch parameter. You could still do

package { “yum.i386”:

      arch => "i386",
      name => "yum",
      ensure => latest,

}

to have different titles for different architectures if need be. But I would really prefer an arch parameter (which I could set a default for to $architecture, so that only the main system architecture is installed by default), which would accept an array to support multiple architectures.

Regards, Sven

Problem here is: As puppet is today, it doesn’t make it hard to install multiple architectures of the same package. To the contrary: It makes it hard to install only a single architecture. Because with RedHat, if you don’t specify an architecture, it will install all available architectures of the package. And this behaviour along with the wish to control the version of the package that is being installed is what caused us stumbling over this bug.

Since RHEL 5.3 this is configurable in yum.conf, I set my machines to install just the best match package, so better multi arch support would be good.

David Lutterkort wrote:

I think the suggested fix is spot on; clearly, we need to put the version in between the package name and the arch. We could intrdoduce a new arch parameter for that, but that seems more trouble than it’s worth.

The one thing I would change about the patch is to restrict what suffixes we’d consider to be an arch. The canonical list is in arch.py in the yum source, but something like the following should work pretty well:

[…]

Just for the record, to match all arches from arch.py, the list should probably be: /(.).(noarch|i[3-6]86|x86_64|ppc|ppc64|sparc.|s390.|alpha.|arm.|sh.|ia64)/ Apparently, there is also “athlon”, “geode”, “amd64”, “ia32e”, but I don’t think any of that was ever used as a package architecture.

If an arch parameter is added, won’t that make a backwards-incompatible change? Currently, for folks that don’t need a specific version, package { ‘foo.i386’: } works. Would an arch parameter change that behavior? If not, then it may be needlessly confusing to require an arch parameter only when a version is used with the ensure parameter.

Todd Zullinger wrote:

If an arch parameter is added, won’t that make a backwards-incompatible change? Currently, for folks that don’t need a specific version, package { ‘foo.i386’: } works. Would an arch parameter change that behavior? If not, then it may be needlessly confusing to require an arch parameter only when a version is used with the ensure parameter.

Well package.arch only works for the yum provider currently, if I’m not mistaken. And that is mostly accidental as far as I can tell. It should not be too much of a problem to extract the architecture from the title of the package resource if none is specified with an arch parameter (and I think a deprecation note of this notation would be right). Also, no arch parameter (and no .arch appendix to the title) could work just like today: Install the default architecture of the package manager. Also, an array of architectures could be specified if a package should be installed in more than one architecture.

Not that I have the ruby skills or time needed to implement this.

regards, Sven

I do think we should be heading towards an arch parameter, as well as a dedicated version parameter, and then we should stop overloading “ensure” with version information.

Does https://projects.puppetlabs.com/issues/7463 help anyone else in the short term? I believe that’s a sane middle ground to head for next.

Nigel Kersten wrote:

Does https://projects.puppetlabs.com/issues/7463 help anyone else in the short term? I believe that’s a sane middle ground to head for next.

I am not sure how it is any different in practice than the solution I offered in my initial bug report.

David Ward wrote:

Nigel Kersten wrote:

Does https://projects.puppetlabs.com/issues/7463 help anyone else in the short term? I believe that’s a sane middle ground to head for next.

I am not sure how it is any different in practice than the solution I offered in my initial bug report.

It doesn’t, but the rest of the log of this ticket contains a bunch of other suggestions, some of which we’ll implement in the longer term.