Bug #2884

Test isolation: spec/unit/property/keyvalue.rb generates 6 spurious errors

Added by Markus Roberts over 2 years ago. Updated 4 months ago.

Status:Closed Start date:12/03/2009
Priority:Low Due date:
Assignee:- % Done:

0%
Category:testing
Target version:-
Affected Puppet version:0.25.1 Branch:
Keywords:
Votes: 0

Description

I was optimistic that catching/cataloging the pair-wise interactions would solve most of our test isolation problems, but that hope is seeming unwarranted. Yesterday I wound up tracing two sets of errors that are apparently three-way interactions. This is the first:

spec/unit/property/keyvalue.rb generates six spurious errors:

1) 
Puppet::DevError in 'Puppet::Property::KeyValue as an instance when calling should should call process_current_hash'
Validate method failed for class : undefined method `validate' for nil:NilClass
/Users/markus/projects/puppet/lib/puppet/parameter.rb:483:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/property.rb:381:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/parameter.rb:489:in `validate'
/Library/Ruby/Site/1.8/puppet/type/property.rb:370:in `should='
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `each'
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `should='
./spec/unit/property/keyvalue.rb:85:
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:22:in `run'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

2)
Puppet::DevError in 'Puppet::Property::KeyValue as an instance when calling should should return the hashed values of @should and the nilled values of retrieve if inclusive'
Validate method failed for class : undefined method `validate' for nil:NilClass
/Users/markus/projects/puppet/lib/puppet/parameter.rb:483:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/property.rb:381:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/parameter.rb:489:in `validate'
/Library/Ruby/Site/1.8/puppet/type/property.rb:370:in `should='
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `each'
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `should='
./spec/unit/property/keyvalue.rb:92:
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:22:in `run'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

3)
Puppet::DevError in 'Puppet::Property::KeyValue as an instance when calling should should return the hashed @should + the unique values of retrieve if !inclusive'
Validate method failed for class : undefined method `validate' for nil:NilClass
/Users/markus/projects/puppet/lib/puppet/parameter.rb:483:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/property.rb:381:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/parameter.rb:489:in `validate'
/Library/Ruby/Site/1.8/puppet/type/property.rb:370:in `should='
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `each'
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `should='
./spec/unit/property/keyvalue.rb:99:
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:22:in `run'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

4)
Puppet::DevError in 'Puppet::Property::KeyValue as an instance when calling insync? should return true if the passed in values is nil'
Validate method failed for class prop_name: undefined method `validate' for nil:NilClass
/Users/markus/projects/puppet/lib/puppet/parameter.rb:483:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/property.rb:381:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/parameter.rb:489:in `validate'
/Library/Ruby/Site/1.8/puppet/type/property.rb:370:in `should='
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `each'
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `should='
./spec/unit/property/keyvalue.rb:149:
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:22:in `run'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

5)
Puppet::DevError in 'Puppet::Property::KeyValue as an instance when calling insync? should return true if hashified should value == (retrieved) value passed in'
Validate method failed for class prop_name: undefined method `validate' for nil:NilClass
/Users/markus/projects/puppet/lib/puppet/parameter.rb:483:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/property.rb:381:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/parameter.rb:489:in `validate'
/Library/Ruby/Site/1.8/puppet/type/property.rb:370:in `should='
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `each'
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `should='
./spec/unit/property/keyvalue.rb:155:
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:22:in `run'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

6)
Puppet::DevError in 'Puppet::Property::KeyValue as an instance when calling insync? should return false if prepared value != should value'
Validate method failed for class prop_name: undefined method `validate' for nil:NilClass
/Users/markus/projects/puppet/lib/puppet/parameter.rb:483:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/property.rb:381:in `unsafe_validate'
/Users/markus/projects/puppet/lib/puppet/parameter.rb:489:in `validate'
/Library/Ruby/Site/1.8/puppet/type/property.rb:370:in `should='
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `each'
/Library/Ruby/Site/1.8/puppet/type/property.rb:369:in `should='
./spec/unit/property/keyvalue.rb:162:
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:22:in `run'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/markus/projects/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

when run with spec/integration/reference/providers.rb and spec/integration/indirector/catalog/queue.rb

touch spec/integration/indirector/catalog/queue.rb
sleep 2
touch spec/integration/reference/providers.rb 
sleep 2
touch spec/unit/property/keyvalue.rb 
spec --format s --loadby mtime spec/unit/property/keyvalue.rb spec/integration/reference/providers.rb spec/integration/indirector/catalog/queue.rb

History

#1
Updated by Jesse Wolfe over 2 years ago

  • Status changed from Accepted to Needs More Information

I haven’t been able to reproduce this failure; I tried several orderings.

#2
Updated by Markus Roberts over 2 years ago

  • Status changed from Needs More Information to Accepted

I’ll see if there’s another factor I missed.

#3
Updated by Markus Roberts over 2 years ago

The more I figure out about this, the less I understand. Four observations:

  • It only seems to happen on older (<1.2.9) versions of rspec
  • The errors are all when a test is trying to assign the “should” value of a property
  • Notice from the stack trace that it’s using the installed version of puppet/type/property.rb but everything else seems to be coming from the test directory
  • It doesn’t always happen, but when it does it objects to every “x.should = blah” in the file.

If I can’t nail it down more than that I’m going to bump the ticket.

#4
Updated by Markus Roberts over 2 years ago

  • Target version changed from 0.25.2 to 0.25.3

#5
Updated by Markus Roberts over 2 years ago

  • Target version changed from 0.25.3 to 0.25.4

#6
Updated by Jesse Wolfe over 2 years ago

This is basically the failure that happens if the call to Puppet::Property::KeyValue.initvalues in the spec isn’t called.

Some testing later I find: puppet/type/property.rb is really the same file as puppet/property.rb , but circa puppet 0.22.4 The hybrid definition that happens when you load the old file followed by the new file overrides Property.initvalues so that it doesn’t bubble up to its parent class (Parameter)

Which is to say, this is really an autoload bug exacerbated by renamed legacy files and klunky object initializations, but I have not yet discovered how this file got loaded.

#7
Updated by James Turnbull over 2 years ago

  • Target version changed from 0.25.4 to 0.25.5

#8
Updated by James Turnbull about 2 years ago

  • Target version changed from 0.25.5 to 49

#9
Updated by James Turnbull over 1 year ago

  • Target version deleted (49)

#10
Updated by Joshua Lifton 5 months ago

  • Assignee deleted (Jesse Wolfe)

This issue was assigned to a former Puppet Labs employee. Adding back to the pool of unreviewed issues.

#11
Updated by Ben Hughes 4 months ago

  • Description updated (diff)
  • Status changed from Accepted to Unreviewed

#12
Updated by Luke Kanies 4 months ago

  • Description updated (diff)
  • Status changed from Unreviewed to Closed

Assuming this is fixed now.

Also available in: Atom PDF