Bug #5268

hyphen in class name messes with qualified variables

Added by Eric Snow over 2 years ago. Updated 5 months ago.

Status:Re-openedStart date:11/11/2010
Priority:NormalDue date:
Assignee:eric sorenson% Done:

0%

Category:-
Target version:3.x
Affected Puppet version: Branch:
Keywords:language, class, name, naming

Description

$module::class-name::variable

This tricks puppet. I suppose it thinks the “–” is a minus…


Related issues

Related to Puppet - Bug #7523: Refactor the grammar to reduce duplication Closed 05/13/2011
Related to Puppet - Bug #10506: Naming rules are inconsistent Accepted 11/03/2011
Related to Puppet - Bug #10146: Puppet interpolates variables differently in 2.7.x Closed 10/18/2011
Related to Puppet - Feature #17260: Optional deprecation warning for hyphens in names Closed
Duplicated by Puppet - Bug #902: Qualified-variable names cannot contain hyphens (but clas... Duplicate
Duplicated by Puppet - Bug #1686: Class names containing '-' cannot be used to scope variables Duplicate 10/24/2008
Duplicated by Puppet - Bug #16250: Problems with pluginsync Duplicate 09/05/2012
Duplicated by Puppet - Feature #4350: Should be able to interpolate variables in classes contai... Duplicate 07/23/2010

History

#1 Updated by Nigel Kersten over 2 years ago

  • Status changed from Unreviewed to Rejected

Hyphens aren’t allowed in class names, as in Ruby.

We should be documenting this better, and we do have an open ticket about documenting all the reserved words and allowed characters in Puppet.

#2 Updated by R.I. Pienaar over 2 years ago

it’s more complex than that.

The documentation used to state that –’s didnt work, but in reality they kind of worked like they do today. I asked for the ability to be removed since the behavior is just not consistent across the language instead the docs got updated to reflect the fact that they are supported.

So at this point i think we should fix the code so it works, since everyone has been explicitly told it would work.

The docs says:

You can use the characters A-Z, a-z, 0-9, dashes (‘-‘), and underscores in variables, resources and class names. In Puppet releases prior to 0.24.6, you cannot start a class name with a number.

#3 Updated by Peter Meier over 2 years ago

I use that a lot for site specific stuff: site-apache etc.

For sure it would be possible to change things and actually I even agree on the argument that in most languages this is also not possible.

However, as R.I. Pienaar said, up to now it is possible to use dashes in class names and a lot of people are using them!

#4 Updated by Nigel Kersten over 2 years ago

  • Status changed from Rejected to Re-opened

RI, I see what you mean, but the reality is that they never worked completely. There have always been edge cases like this one.

I’m still leaning towards simply not allowing them in the future to remove all these edge cases rather than doing the development work to fix all the edge cases, but I’m willing to listen to more arguments against that approach.

#5 Updated by Markus Roberts over 2 years ago

Two thoughts: first, historically many languages (lisp, cobol, forth, etc.) have supported them, and second the main area where there’s a challenge, ambiguity in the parser w.r.t subtraction, has already been crossed. This remaining problems are mostly just stray regular expressions where someone (probably me in this case) was lazy or made a bad assumption.

Third, bonus thought:

diff --git a/lib/puppet/parser/lexer.rb b/lib/puppet/parser/lexer.rb
index 31d39ae..abb088d 100644
--- a/lib/puppet/parser/lexer.rb
+++ b/lib/puppet/parser/lexer.rb
@@ -240,11 +240,11 @@ class Puppet::Parser::Lexer
   end
   #:startdoc:
 
-  TOKENS.add_token :DOLLAR_VAR, %r{\$(\w*::)*\w+} do |lexer, value|
+  TOKENS.add_token :DOLLAR_VAR, %r{\$([-\w]*::)*[-\w]+} do |lexer, value|
     [TOKENS[:VARIABLE],value[1..-1]]
   end
 
-  TOKENS.add_token :VARIABLE, %r{(\w*::)*\w+}
+  TOKENS.add_token :VARIABLE, %r{([-\w]*::)*[-\w]+}
   #:stopdoc: # Issue #4161
   def (TOKENS[:VARIABLE]).acceptable?(context={})
     [:DQPRE,:DQMID].include? context[:after]
@@ -545,7 +545,7 @@ class Puppet::Parser::Lexer
     token_queue << [TOKENS[token_type[terminator]],preamble+value]
     if terminator != '$' or @scanner.scan(/\{/)
       token_queue.shift 
-    elsif var_name = @scanner.scan(%r{(\w*::)*\w+|[0-9]})
+    elsif var_name = @scanner.scan(%r{([-\w]*::)*[-\w]+|[0-9]})
       token_queue << [TOKENS[:VARIABLE],var_name]
       tokenize_interpolated_string(DQ_continuation_token_types)
     else

#6 Updated by Nigel Kersten over 2 years ago

Markus, if you believe that we can solve the edge cases reasonably easily, I’m in favor of allowing hyphens.

#7 Updated by Eric Snow over 2 years ago

Wanted to point out that I have been doing the following to work around this:

inline_template("<%= scope.lookupvar('syslog-ng::server::version') %>")

Thanks for looking into this

#8 Updated by Nigel Kersten over 2 years ago

  • Status changed from Re-opened to Needs More Information
  • Assignee set to Markus Roberts

Assigning to Markus for info on whether we can indeed accomplish this easily. Pass back to me after that please Markus.

#9 Updated by Matt Dainty over 1 year ago

Just got bitten by this, also noticed it’s in the examples here:

http://docs.puppetlabs.com/guides/language_guide.html#classes

apache-ssl, etc.

#10 Updated by Jason McKerr over 1 year ago

  • Assignee changed from Markus Roberts to Jacob Helwig

This was still assigned to Markus.

#11 Updated by Nigel Kersten over 1 year ago

Didn’t we actually commit this fix and that’s why people have been having to interpolate variables in strings with hyphens?

e.g.

foo-bar-$baz-spa used to work but now you have to do foo-bar-${baz}-spa ?

#12 Updated by eric sorenson over 1 year ago

Nigel Kersten wrote:

Didn’t we actually commit this fix and that’s why people have been having to interpolate variables in strings with hyphens?

e.g.

foo-bar-$baz-spa used to work but now you have to do foo-bar-${baz}-spa ?

That’s not exactly the use case in question (though it might be related under the hood). The thing we’re getting bit by (PE1.2/2.6.9) is when the class' name is “foo-bar” and you want to use variable baz elsewhere:

$localvar = "${foo-bar::baz}" "syntax error at 'bar::baz', expected '}'"

Unquoted without curly braces, same result.

Unquoted with curly braces.

$localvar = ${foo-bar::baz} ...Could not match ${foo-bar::baz}

#13 Updated by Jacob Helwig over 1 year ago

  • Assignee changed from Jacob Helwig to Nigel Kersten

Nigel, this is definitely related to, and my end up a duplicate of the other ‘–’ in $entity names ticket (#10146).

#14 Updated by Daniel Pittman about 1 year ago

  • Status changed from Needs More Information to Accepted
  • Assignee deleted (Nigel Kersten)
  • Target version set to 3.x

We need to, and intend, to enforce the stated rule that - is not acceptable in class names. That is going to end up in a Telly release, but isn’t going to make the .0 target, unlike enforcement on variable names.

#15 Updated by Brian Gupta 12 months ago

If you make this change it will break a lot of puppet code in the field. Enough code that refactoring could be pretty much a non-starter for certain sites, and preventing upgrading. The reality is class-name is visually a useful way to name classes, and as stated earlier.. this was “supported” and published in official examples.

#16 Updated by Erik Dalén 12 months ago

I’m fine with doing the change, but I think it should issue deprecation warnings first before becoming entirely unsupported.

#17 Updated by Brian Gupta 12 months ago

Best as I can tell dashes are still allowed in module names, which pretty much means that dashes in class names should be supported. I have some disagreements about some of the other changes in 3.0, but none so far that I feel strongly enough about to raise as an issue.

#18 Updated by Nigel Kersten 12 months ago

Brian Gupta wrote:

Best as I can tell dashes are still allowed in module names, which pretty much means that dashes in class names should be supported. I have some disagreements about some of the other changes in 3.0, but none so far that I feel strongly enough about to raise as an issue.

Ugh. We can’t be enforcing this if we’re still allowing hyphens in class names in Telly.

#19 Updated by Paul Armstrong 11 months ago

We need to, and intend, to enforce the stated rule that - is not acceptable in class names. That is going to end up in a Telly release, but isn’t going to make the .0 target, unlike enforcement on variable names.

This is going to mean re-writing most of my classes which is extremely frustrating (and from the copious comments in various tickets, I’m not the only one). I see a discussion in one of the linked tickets about a decision being made to kill off hyphens, but no reasoning for it. Is there a discussion around that explains why this is being made (as discussed here, solving for subtraction has been done)?

#20 Updated by Dennis Benzinger | hybris 11 months ago

Thank you for your message.

I am currently out of office with no access to emails from July, 13th 2012 till July, 30th 2012. Your email will not be forwarded.

Kind regards, Dennis Benzinger

#21 Updated by Brian Gupta 11 months ago

Paul\,

Welcome-to-the-club/.Apparently-parser-writing-is-a-bitch.(Which-in-all-fairness\,-it-probably-is,-but-it-really-shocks-me-that-the-really-smart-programmers-at-PuppetLabs-can’t-solve-this-problem.)That said,-I-leave-it-at-${Fail}– and-disappointment.(The-decision-has-been-made-by-upper-management\,-don\’t-bother-to-complain.)Do-understand,-this-probably-impacts-less-than-15\%-of-their-userbase\,-so-it\’s-\“fine\”.(I-was-told-in-IRC-to-fork-if-I-had-an-issue.)

Brian

#22 Updated by James Turnbull 11 months ago

Brian – I am sorry you’re frustrated by this. This is unfortunately a non-trivial problem to fix both technically and from the hole we dug. We messed up and allowed an inconsistent model – mea culpa – we’re annoyed and embarrassed it happened like this (I take some of the blame here – I am partially responsible for thinking this was a good idea long ago).

So now we’re trying to fix it and make it consistent. That resulting fix is probably not going to make everyone happy. We understand that and we’ll do our best to help people with it by using deprecation, publishing clear rules on what works and what doesn’t work, updating documentation and it may be feasible to provide some find-n-replace regexes to fix some of this when we have a path of action.

I will have a conversation internally with Eric, Nigel and the relevant Engineering folks and update the ticket with a clear rationale, approach and how we plan to proceed.

#23 Updated by Brian Gupta 11 months ago

Thanks James. I’m looking forward to the rationale, as I do believe I am not alone in wanting this, and am still not 100% clear if there are reasons beyond it being hard to code the parser, as to why this is being disabled. I feel it should be able to be addressed by documenting the usage, and fixing any broken edge cases.

#24 Updated by Brian Gupta 11 months ago

So it’s clear that the core puppet team has been struggling with this for a while, and understand all the edge cases, which make this challenging to support. I think a clear explanation to the community would be helpful. (With room for feedback, and an opportunity to see if there are alternatives to dropping support.)

I’m also like to apologize if anyone took my note personally, reading back it was more snarky than warranted, and probably expressed more frustration than was helpful.

#25 Updated by Dustin Mitchell 10 months ago

So, #16213 was just duped here. This bug is almost 2 years old. At the moment, dashes in classes sort of work, and cause surprising failures on upgrades, where support waxes and wanes without warning.

On the theory that a bird in hand (that is, consistent behavior) is worth two in the bush (an open-ended bug), can we commit something to make classes with dashes in their names illegal, or at least issue a warning?

#26 Updated by Nigel Kersten 10 months ago

  • Assignee set to eric sorenson

#27 Updated by Nick Fagerlund 8 months ago

Relevant docs links:

http://docs.puppetlabs.com/puppet/2.7/reference/lang_reserved.html#classes-and-types http://docs.puppetlabs.com/puppet/3/reference/lang_reserved.html#classes-and-types

The documentation matches our current beliefs, but the code doesn’t enforce those beliefs. This is the case in a few other places, as well.

#28 Updated by Andrew Parker 8 months ago

  • Status changed from Accepted to Closed

I’m closing this ticket with the following resolution:

There have been several attempts to come to a solution on this issue, but none have lead to a better situation. So here is where it is going to have to live: hyphens are allowed in class names by the parser, but they are highly discouraged and not a documented feature. If you use hyphens in your class names, then you won’t be able to reference variables inside the class. There is also the possibility that pluginsync will fail because of the hyphens (when they appear in the filename). Messing with such a fundamental aspect of the syntax of the language is prone to breaking far too much existing manifest code.

From #17260, there is a flag that allows hyphens to be used in variable names, but that is a deprecated feature of the language and will be removed later.

#29 Updated by Justin Honold 8 months ago

  • Status changed from Closed to Re-opened

If you use hyphens in your class names, then you won’t be able to reference variables inside the class. There is also the possibility that pluginsync will fail because of the hyphens (when they appear in the filename). Messing with such a fundamental aspect of the syntax of the language is prone to breaking far too much existing manifest code.

All of this, without even a warning?

#30 Updated by Dustin Mitchell 6 months ago

  • Status changed from Re-opened to Closed

The patch for #17260 gives a warning in 2.7.20 and higher. AIUI, dashes are illegal in 3.0.0 and higher. So this one is definitely done.

#31 Updated by Erik Dalén 6 months ago

  • Status changed from Closed to Re-opened

Unfortunately dashes still work in class names in 3.0.x:

test.pp:

class test-case {
  notice('test')
}
include test-case

console:

$ puppet apply test.pp
Notice: Scope(Class[Test-case]): test
Notice: Finished catalog run in 0.05 seconds
$ puppet --version
3.0.2

#32 Updated by Andrew Parker 5 months ago

  • Keywords set to language, class, name, naming

Erik is right, the issue of dashes in class names is still open.

I think that the correct fix here is to deprecate dashes in class names as well.

Also available in: Atom PDF