The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Bug #8174

External Node Classifer makes puppet think Facter variables are dynamic lookup issues

Added by Chuck Schweizer about 3 years ago. Updated over 1 year ago.

Status:ClosedStart date:06/30/2011
Priority:NormalDue date:
Assignee:Andrew Parker% Done:

0%

Category:-
Target version:2.7.15
Affected Puppet version:2.7.1 Branch:https://github.com/puppetlabs/puppet/pull/657
Keywords: customer

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This issue is currently not available for export. If you are experiencing the issue described below, please file a new ticket in JIRA. Once a new ticket has been created, please add a link to it that points back to this Redmine ticket.


Description

I have found that if your class is part of the External Node Classifier definition puppet 2.7.1 thinks variables from Facter are dynamic lookups.

EG.

External config

---
classes:
- hadoop
environment: production
parameters:
  puppetmaster: puppet

Dynamic lookup of $operatingsystem at /var/opt/puppet/environments/production/modules/hadoop/manifests/base.pp:53 is deprecated. Support will be removed in Puppet 2.8. Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.

$ facter operatingsystem RedHat

puppet-8174.tar.gz - Tarball of my minimum-reproducing manifest for this issue under Puppet 2.7.14 (967 Bytes) Ed Brannin, 05/15/2012 01:41 pm

puppet-8174.log - Script file showing me apply it (contains CRLF and ANSI color codes) (854 Bytes) Ed Brannin, 05/15/2012 01:41 pm


Related issues

Related to Puppet - Bug #13312: Dynamic scoping deprecation warnings should be given at t... Closed 03/20/2012
Duplicated by Puppet - Bug #14388: define warns incorrectly about deprecated dynamic lookup Duplicate 05/09/2012

History

#1 Updated by Nick Fagerlund almost 3 years ago

  • Status changed from Unreviewed to Needs More Information

I can’t make this work the way you describe. Here’s what I’ve got:

My ENC:

#!/usr/bin/env perl
use strict;
use warnings;

my $yaml = <<EOT;
---
classes:
    templator:
parameters:
    thevar: "Set from ENC."
EOT

print $yaml;

modules/templator/manifests/init.pp:

class templator {
  file {'/tmp/templator.txt':
    content => template("templator/text.erb"),
    ensure => file,
  }
  notify {'nemplator':
    message => "Testing issue 8174. FQDN is $fqdn and OS is $operatingsystem",
  }
}

modules/templator/templates/text.erb:

Text file for temp, coming right up! Here's the var:

<%= scope.lookupvar("::thevar") %>
Also, here's a fact:
<%= fqdn %>
And here's a top scope var:
<%= randomtopscopevar %>

From what you said, I should be getting scope lookup warnings from the notify and probably the template, but it’s not throwing any warnings at all. Am I missing something? What’re the differences between your case and my test case?

#2 Updated by Chuck Schweizer almost 3 years ago

I was able to replicate my issue with the following setup. This setup is just using the built in webrick running puppetmasterd as a service on RHEL 5.

In this example init.pp is part of site.pp and test.pp is part of ENC.
I tested also verified if I moved init.pp to the ENC I would get the Dynamic lookup error for it also.

I also restart the puppet master process after every puppet client run. The warning from the puppet master only occurs once, and a restart will force it to display it again.

Puppet/Ruby version

# puppet --version
2.7.1

# ruby --version
ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux]

My ENC

#!/usr/bin/env perl
use strict;
use warnings;
my $yaml = <<EOT;
---
classes:
  linux_base::test
parameters:
  thevar: "Set from ENC."
EOT
print $yaml;

manifests/site.pp

# Setup Global defaults

# Setup pre and post stage for Puppet
stage { "pre": before => Stage[main] }
stage { "post": require => Stage[main] }

# Setup default modules for all servers
node basenode {

    if ( $kernel == 'Linux' ){
      class {"linux_base": }
    }
}

node default inherits basenode {}

modules/linux_base/manifests/init.pp

class linux_base {
  notify {'linux_base':
    message => "Testing issue 8174. linux_base: FQDN is $fqdn and OS is $operatingsystem",
  }
}

modules/linux_base/manifests/test.pp

class linux_base::test {
  notify {'linux_base_test':
    message => "Testing issue 8174. linux_base::test: FQDN is $fqdn and OS is $operatingsystem",
  }
}

Puppet client run

# puppet agent -vt
info: Caching catalog for generic_server.domain.org
info: Applying configuration version '1313410581'
notice: Testing issue 8174. linux_base::test: FQDN is generic_server.domain.org and OS is RedHat
notice: /Stage[main]/Linux_base::Test/Notify[linux_base_test]/message: defined 'message' as 'Testing issue 8174. linux_base::test: FQDN is generic_server.domain.org and OS is RedHat'
notice: Testing issue 8174. linux_base: FQDN is generic_server.domain.org and OS is RedHat
notice: /Stage[main]/Linux_base/Notify[linux_base]/message: defined 'message' as 'Testing issue 8174. linux_base: FQDN is generic_server.domain.org and OS is RedHat'
notice: Finished catalog run in 0.60 seconds

/var/log/messages

Aug 15 07:16:08 generic_server puppet-master[10354]: Caught TERM; calling stop
Aug 15 07:16:11 generic_server puppet-master[11626]: Reopening log files
Aug 15 07:16:11 generic_server puppet-master[11626]: Starting Puppet master version 2.7.1
Aug 15 07:16:21 generic_server puppet-master[11626]: Dynamic lookup of $fqdn at /var/opt/puppet/environments/production/modules/linux_base/manifests/test.pp:3 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.
Aug 15 07:16:21 generic_server puppet-master[11626]: Dynamic lookup of $operatingsystem at /var/opt/puppet/environments/production/modules/linux_base/manifests/test.pp:3 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.
Aug 15 07:16:21 generic_server puppet-master[11626]: Compiled catalog for generic_server.domain.org in environment production in 0.18 seconds
Aug 15 07:16:21 generic_server puppet-agent[11631]: Caching catalog for generic_server.domain.org
Aug 15 07:16:22 generic_server puppet-agent[11631]: Applying configuration version '1313410581'
Aug 15 07:16:22 generic_server puppet-agent[11631]: Testing issue 8174. linux_base::test: FQDN is generic_server.domain.org and OS is RedHat
Aug 15 07:16:22 v puppet-agent[11631]: (/Stage[main]/Linux_base::Test/Notify[linux_base_test]/message) defined 'message' as 'Testing issue 8174. linux_base::test: FQDN is generic_server.domain.org and OS is RedHat'
Aug 15 07:16:22 generic_server puppet-agent[11631]: Testing issue 8174. linux_base: FQDN is generic_server.domain.org and OS is RedHat
Aug 15 07:16:22 generic_server puppet-agent[11631]: (/Stage[main]/Linux_base/Notify[linux_base]/message) defined 'message' as 'Testing issue 8174. linux_base: FQDN is generic_server.domain.org and OS is RedHat'
Aug 15 07:16:22 generic_server puppet-agent[11631]: Finished catalog run in 0.60 seconds

/etc/puppet/puppet.conf

[main]
# Where Puppet stores dynamic and growing data.
# The default value is '/var/lib/puppet'.
vardir = /var/lib/puppet
# The Puppet log directory.
# The default value is '$vardir/log'.
logdir = /var/opt/puppet/log
# Where Puppet PID files are kept.
# The default value is '$vardir/run'.
rundir = /var/run/puppet
# Setup plugins
#pluginsync = true
factpath = $vardir/lib/facter
[agent]
# The file in which puppetd stores a list of the classes
# associated with the retrieved configuratiion.  Can be loaded in
# the separate ``puppet`` executable using the ``--loadclasses``
# option.
# The default value is '$confdir/classes.txt'.
classfile = $vardir/classes.txt
# Where puppetd caches the local configuration.  An
# extension indicating the cache format is added automatically.
# The default value is '$confdir/localconfig'.
localconfig = $vardir/localconfig
# Where SSL certificates are kept.
# The default value is '$confdir/ssl'.
ssldir = $vardir/ssl-client
# setup the server
server = generic_server.domain.org
# Enable reporting
report = false
ca_server = generic_server.domain.org
environment = production
splay = true
noop = false
[master]
# Where SSL certificates are kept.
# The default value is '$confdir/ssl'.
ssldir = $vardir/ssl
certname=generic_server.domain.org
ca=true
storeconfigs = false
autosign = true
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
# External node script
external_nodes = /var/opt/puppet/scripts/puppet_node_classifier
node_terminus = exec
####################################################
#
#  Setup Pupppet environments
#
####################################################
#Production this is the default puppet enviornment
manifestdir = /var/opt/puppet/environments/production/manifests
manifest = /var/opt/puppet/environments/production/manifests/site.pp
modulepath = /var/opt/puppet/environments/production/modules

#3 Updated by Chuck Schweizer almost 3 years ago

Verified I still see this issue on puppet 2.7.3 and RHEL 6.

#4 Updated by Jeff Blaine almost 3 years ago

  • Status changed from Needs More Information to Needs Decision

I can confirm this happens with 2.7.3 only when using an ENC.

I got the same deprecation warning when my ENC is turned on.

#5 Updated by John Arundel almost 3 years ago

Happening here with 2.7.3 as well when using ‘puppet apply’ (no ENC). No warning when running Puppet via server.

      ...
      file    => "/root/iptables/hosts/${hostname}",
      …

warning: Dynamic lookup of $hostname at /root/puppet/modules/iptables/manifests/iptables.pp:39 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.

#6 Updated by James Turnbull almost 3 years ago

  • Status changed from Needs Decision to Investigating
  • Assignee set to Nick Fagerlund

Nick – can you take a further look? Thanks.

#7 Updated by Nick Fagerlund almost 3 years ago

  • Status changed from Investigating to Accepted

HUP, my mistake: this is easily reproducible as described (warning only appears when using an ENC) on 2.7.1 and 2.7.3. (For some reason, I was expecting the warning to propagate down to the agent. Didn’t it used to do that?)

This is definitely a bug.

#8 Updated by Paul Seymour over 2 years ago

This is currently happening on my 2.7.9 RHEL based setup. With ENC based systems and facter variables (inbuilt and custom) show this deprecation warning:–

puppet-master[3713]: Dynamic lookup of $lsbmajdistrelease at /etc/puppet/envs/legacy/modules/autofs/manifests/init.pp:68 is deprecated. Support will be removed in Puppet 2.8. Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.

It can produce a lot of noise.

#9 Updated by Russ Allbery over 2 years ago

Are these warnings safe to ignore, or is Puppet really going to break all use of facts without scoping in Puppet 2.8 as the warning seems to imply? Do we have to go through all of our manifests and change every use of, e.g., $operatingsystem to $::operatingsystem?

The implications of that for templates look, so far as I can tell from Google, absolutely awful. I’m hoping that, since it was indicated above that this is a bug, facts will just stay global and the warning can just be ignored….

#10 Updated by Nick Fagerlund over 2 years ago

  • Assignee changed from Nick Fagerlund to Nigel Kersten

HMM, this shouldn’t be assigned to me anymore. Nigel, where should this end up? Platform team?

Russ, we definitely consider this a bug, not a feature. Facts ARE meant to be global, and we don’t intend to require $:: for facts in Telly.

#11 Updated by Russ Allbery over 2 years ago

Thanks! That’s good to know; I’ll back out of the places we added :: to our manifests.

#12 Updated by Paul Hinze over 2 years ago

Hi folks. Are there plans to fix this warning pre-Telly? I ended up backing out of an upgrade to 2.7.x because we get tons of warnings stemming from this bug. I’d love to get us on the latest and greatest without ballooning our puppet output.

#13 Updated by Nigel Kersten over 2 years ago

  • Assignee changed from Nigel Kersten to Anonymous

Paul, yes, we are planning to fix this pre-Telly.

It’s important we’re correctly producing deprecation warnings in 2.7.x so that users get expected behavior in Telly itself.

Daniel, assigned to you, we’ll chat about getting this in the priority list.

Nick, added you as a watcher as I feel like we’ve got a duplicate bug or two around this…

#14 Updated by Michael Arnold over 2 years ago

Nick Fagerlund wrote:

Facts ARE meant to be global, and we don’t intend to require $:: for facts in Telly.

How will Puppet be able to tell if $foo is a fact or a local variable without $:: or some other notation? I can imagine writting “class myclass { $foo = ‘bar’ }” and then a year later $foo shows up as a fact, causing havok. (Or am I misunderstanding?)

#15 Updated by Nigel Kersten over 2 years ago

Local will always be checked first. A subsequent fact wouldn’t break an existing manifest.

What would break would be you wanting to use $foo the variable and $foo the fact in the same scope with the same name, but that doesn’t seem very likely…

#16 Updated by Michael Arnold over 2 years ago

Understood. Thanks Nigel.

Please don’t forget to update the Style Guide.

#17 Updated by Andrew Parker over 2 years ago

  • Assignee changed from Anonymous to Andrew Parker

#18 Updated by Andrew Parker over 2 years ago

Turns out that this only happens if there is a node definition in addition to the enc.

An enc:

#!/usr/bin/env sh

cat <<END
---
classes:
  a
parameters:
  enc_var: "Set from ENC."
END
exit 0

manifests/site.pp

$top_scope = "set from site.pp"
node default {}

modules/a/manifests/init.pp

class a {
  $locally = "locally declared"
  notify { "fqdn from facts": message => $fqdn }
  notify { "locally declared var": message => $locally }
  notify { "var via enc": message => $enc_var }
  notify { "declared top scope": message => $top_scope }
}

This causes the following warnings to appear on the master:

warning: Dynamic lookup of $fqdn at /Users/andy/work/test/conf/master/modules/a/manifests/init.pp:3 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.
warning: Dynamic lookup of $enc_var at /Users/andy/work/test/conf/master/modules/a/manifests/init.pp:5 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.
warning: Dynamic lookup of $top_scope at /Users/andy/work/test/conf/master/modules/a/manifests/init.pp:6 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.

If the node default {} line is removed from site.pp then none of these warnings are produced.

#19 Updated by Jeff Weiss over 2 years ago

  • Status changed from Accepted to Merged - Pending Release
  • Target version set to 2.7.14

#20 Updated by Andrew Parker over 2 years ago

  • Branch set to https://github.com/puppetlabs/puppet/pull/657

#21 Updated by Ed Brannin about 2 years ago

I’m still seeing this when I run “puppet apply” with Puppet 2.7.14 running on Ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux].

I have a few module-based defines that reference $hostname from facter.

#22 Updated by Andrew Parker about 2 years ago

Could you provide an example? I just tried it out and it seems to be working correctly.

I’m using the same enc as I had above with a scoping.pp file:

define b() {
  notify { $hostname: }
  notify { $enc_var: }
  notify { $dynamic: }
}

class a {
  $dynamic = "dynamic"
  b { "testing": }
}

Results in output from puppet apply scoping.pp:

warning: Dynamic lookup of $dynamic at /Users/andy/tmp/scoping.pp:4 is deprecated. For more information, see http://docs.puppetlabs.com/guides/scope_and_puppet.html. To see the change in behavior, use the --debug flag.
notice: dynamic
notice: /Stage[main]/A/B[testing]/Notify[dynamic]/message: defined 'message' as 'dynamic'
notice: Set from ENC.
notice: /Stage[main]/A/B[testing]/Notify[Set from ENC.]/message: defined 'message' as 'Set from ENC.'
notice: aparker
notice: /Stage[main]/A/B[testing]/Notify[aparker]/message: defined 'message' as 'aparker'
notice: Finished catalog run in 0.03 seconds

I also tried this with b being in a module on the modulepath and got the same result.

#23 Updated by Ed Brannin about 2 years ago

I took some time today to reduce the manifest to the smallest thing I can get to show the issue.

  • I’m using “puppet apply” with —modulepath specified on the command-line, per apply.sh
  • There is a default node (an empty one is enough), but it behaves properly if there is no “node default”.
  • There is a module-based define that invokes $hostname.
  • Invoking $hostname from site.pp doesn’t count.

Output:

(From a throwaway VM; I don’t just traipse around as root)

Script started on Tue 15 May 2012 04:12:07 PM EDT
[root@localhost bug-8174]# cd manifests/
[root@localhost manifests]# puppet --version
ruby --2.7.14
[root@localhost manifests]# ruby --version
ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux]
[root@localhost manifests]# ./apply.sh site.pp 
warning: Dynamic lookup of $hostname at /root/dev/bug-8174/manifests/modules/bug/manifests/hostname_warning.pp:2 is deprecated. For more information, see http://docs.puppetlabs.com/guides/scope_and_puppet.html. To see the change in behavior, use the --debug flag.
notice: Scope(Bug::Hostname_warning[Name]): localhost
info: Applying configuration version '1337112746'
notice: Finished catalog run in 0.21 seconds
[root@localhost manifests]# exit
Script done on Tue 15 May 2012 04:12:27 PM EDT

Files:

Listing

manifests/
manifests/apply.sh
manifests/site.pp
manifests/modules
manifests/modules/bug
manifests/modules/bug/manifests
manifests/modules/bug/manifests/hostname_warning.pp

manifests/apply.sh (used to invoke Puppet; give “site.pp” as its argument)

#!/bin/bash

DIR="$( cd "$( dirname "$0" )" && pwd )"
sudo -E puppet apply -v --modulepath=$DIR/modules $@

manifests/site.pp

node default {
}

bug::hostname_warning { "Name": }

manifests/modules/bug/manifests/hostname_warning.pp

define bug::hostname_warning() {
    notice $hostname
}

#24 Updated by Andrew Parker about 2 years ago

Ed, thanks for those steps to reproduce. They are excellent!

I’ve got it reproducing now and it looks like what I was missing before was having a node statement. The following small snippet shows it happening as well (no ENC required, as you showed):


node default {}
problem { "hmm": }

define problem() {
  notice $hostname
}

If the node default {} line is removed then the incorrect warning isn’t produced.

#25 Updated by Andrew Parker about 2 years ago

  • Status changed from Merged - Pending Release to Code Insufficient
  • Target version changed from 2.7.14 to 2.7.x

Reopening since the code didn’t solve the case of defines.

#26 Updated by Andrew Parker about 2 years ago

  • Status changed from Code Insufficient to In Topic Branch Pending Review

https://github.com/puppetlabs/puppet/pull/794

This should probably try to get into 2.7.15 to minimize the time that we are incorrectly warning on these things.

#27 Updated by Andrew Parker about 2 years ago

  • Target version changed from 2.7.x to 2.7.15

#28 Updated by Andrew Parker about 2 years ago

Closed the previous pull request to target the 2.7rc branch instead of 2.7.x.

https://github.com/puppetlabs/puppet/pull/795

#29 Updated by Anonymous about 2 years ago

  • Status changed from In Topic Branch Pending Review to Merged - Pending Release

#30 Updated by Moses Mendoza almost 2 years ago

  • Status changed from Merged - Pending Release to Closed

Released in 2.7.15rc3.

#31 Updated by Charlie Sharpsteen over 1 year ago

  • Keywords set to customer

Also available in: Atom PDF