Bug #1175

Custom function doesn't work when using multiple environments

Added by almost 4 years ago. Updated over 1 year ago.

Status:Closed Start date:
Priority:High Due date:
Assignee:Markus Roberts % Done:

0%

Category:language
Target version:2.6.0
Affected Puppet version:0.22.1 Branch:luke/tickets/master/1175-functions_in_environments; MarkusQ:ticket/master/1175-part2
Keywords:
Votes: 7

Description

Environments settings in @puppet.conf@:

client:

[main]
    environment = production

server:

[main]
    environments = production,development

[production]
    manifest = /usr/local/etc/puppet/manifests/production/site.pp
    modulepath = /usr/local/etc/puppet/modules/production

I have module for postfix configuration with custom function

My postfix module located in @/usr/local/etc/puppet/modules/production/postfix@ and my custom function in @/usr/local/etc/puppet/modules/production/postfix/plugins/puppet/parser/functions@.

I get the following error running puppet client:

err: Could not retrieve catalog: Unknown function .... at /usr/local/etc/puppet/modules/production/postfix/manifests/init.pp:60 on node .....

If I disable using this function in @postfix/manifests/init.pp@ module works.

If I use setup without environments with configuration:

[main]
    manifest = /usr/local/etc/puppet/manifests/site.pp
    modulepath = /usr/local/etc/puppet/modules

and place module into @/usr/local/etc/puppet/modules/postfix@ and function into @/usr/local/etc/puppet/modules/postfix/plugins/puppet/parser/functions@

everything works as expected.

os version: FreeBSD-6.2 puppet version: 0.24.4

sample.tgz (479 Bytes) , 04/08/2008 09:58 am


Related issues

related to Puppet - Refactor #3015: Known resource types should be managed by Environments, n... Closed 01/08/2010
related to Puppet - Bug #4409: puppetmasterd does not find custom types for environment Closed 07/30/2010
related to Puppet - Bug #4461: Module path should be cached Closed 08/03/2010

History

Updated by AJ Christensen almost 4 years ago

  • Status changed from 1 to 2

Unable to replicate.

puppet.conf:

[main]
environments=production,testing

[production]
manifest=/etc/puppet/env/production/site.pp
modulepath=/etc/puppet/env/production/modules

[puppetd]
environment=production

tests (gsub is a parser function provided by DavidS' common module):

root@junglist (branch: master) /etc/puppet/env/production# cat site.pp
node junglist {
   $ifs = gsub(split($interfaces, " "), "(.+)", "if_\\1")
   notify {
      $ifs:
   }
}
root@junglist (branch: master) /etc/puppet/env/production# puppetd --test --environment production
info: Loading fact configured_ntp_servers
info: Loading fact netmask
info: Loading fact acpi_available
info: Loading fact interfaces
info: Retrieving plugins
notice: Ignoring cache
info: Caching catalog at /var/lib/puppet/state/localconfig.yaml
notice: Starting catalog run
notice: if_eth0,eth0:0,eth0:1,eth0:2,eth0:3,eth1
notice: Finished catalog run in 1.08 seconds

Please provide more details, with specific replication steps if possible.

Updated by AJ Christensen almost 4 years ago

  • Status changed from 2 to 1

Updated by AJ Christensen almost 4 years ago

  • Status changed from 1 to 2

Updated by Luke Kanies almost 4 years ago

I can basically guarantee that this bug is valid, because all functions are dynamically defined on the Scope class, which means that they’re global to environments.

The only real fix is to move the functions to a stand-alone class or module and make that module be per-environment, or to keep the functions as just blocks, looking up and evaluating them as necessary.

Updated by almost 4 years ago

Please provide more details, with specific replication steps if possible.

See sample.tgz

This sample module works in setup without environments and doesn’t work when using environments

Updated by Peter Meier almost 4 years ago

I can confirme this behaviour. I even tried to manually sync the functions, however they always got overwritten/emptied before compiling and then this failed.

Disabling environments just work fine.

This bug is kind of critical and a bit a locker of proceeding people to newer versions, if they want to use environments.

As Luke mentioned in a mail this fix won’t be quite easy to fix. So where would be the best place to start to investigate? How can people help?

Updated by David Schmitt almost 4 years ago

After a bit debugging with immerda, we found out that setting an additional global modulepath at least lets the puppetmaster work with those functions, even when using environments.

I’m running with this configuration already for quite a while and ahd no unexpected problems with it:

[main]
environment=production
# Default modulepath without environments
modulepath=/srv/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules

[production]
modulepath=/srv/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules

[development]
modulepath=/srv/puppet/modules-development:/etc/puppet/modules:/usr/share/puppet/modules

Updated by AJ Christensen over 3 years ago

  • Status changed from 2 to 1

I’d go so far as to say that the workaround below is sufficient, and I’m not familiar enough with the underlying substructures to actually implement the work suggested by Luke, as such, here you go community!

Replying to [comment:7 DavidS]:

After a bit debugging with immerda, we found out that setting an additional global modulepath at least lets the puppetmaster work with those functions, even when using environments.

I’m running with this configuration already for quite a while and ahd no unexpected problems with it:

[main]
environment=production

Default modulepath without environments

modulepath=/srv/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules

[production] modulepath=/srv/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules

[development] modulepath=/srv/puppet/modules-development:/etc/puppet/modules:/usr/share/puppet/modules

Updated by Luke Kanies over 3 years ago

  • Status changed from 1 to 2

Updated by Redmine Admin over 3 years ago

  • Status changed from 2 to Accepted

Updated by Luke Kanies over 3 years ago

  • Priority changed from Normal to High
  • Target version changed from 0.24.5 to 4

I’m bumping this to the next release, because it’s too much work for not enough reward, at this point. I’ve increased the priority, since it is important.

Updated by Nigel Kersten about 3 years ago

  • Affected Puppet version set to 0.22.1

so it wasn’t clear from reading this ticket, but I just did some tests on 0.24.6, and plugins in modules don’t work with environments.

The default modulepath is used for the plugin mount source, not the environment one.

This kind of defeats the point of plugins in modules if you’re using environments.

Updated by Luke Kanies almost 3 years ago

I’ve figured out how to solve half of this problem: The functions should get added to a module, and then that module should get included into each Scope. This provides the environment-specific collection.

There’s more to it than that, though — the loader needs to work in multiple environments, and we need some way to tell the function which environment it belongs in (it obviously can’t be declared as part of the function declaration).

Updated by Luke Kanies almost 3 years ago

  • Target version changed from 4 to 2.6.0

Updated by Luke Kanies almost 2 years ago

  • Assignee changed from Luke Kanies to Rein Henrichs

I took a crack at this and found that it’s totally dependent on #3015 – once that’s merged into master, I’ll try again.

Updated by Luke Kanies almost 2 years ago

  • Assignee changed from Rein Henrichs to Markus Roberts
  • Branch set to luke/tickets/master/1175-functions_in_environments

I’ve pushed an incomplete version of this.

My code moves all of the scope functions into modules, but doesn’t successfully support environments – Markus will fix that.

The problem is that we have no real way of knowing which environment a given function is for; this problem also needs to be solved for the internal dsl work.

Updated by Markus Roberts over 1 year ago

  • Status changed from Accepted to In Topic Branch Pending Review
  • Branch changed from luke/tickets/master/1175-functions_in_environments to luke/tickets/master/1175-functions_in_environments; MarkusQ:ticket/master/1175-part2

Updated by Nick Lewis over 1 year ago

  • Status changed from In Topic Branch Pending Review to Closed

Pushed in ccc869ea48397235d7ba2a5695424eee4923cb9d to master.

Also available in: Atom PDF