Feature #1519

Speed up checksumming

Added by Steve McIntosh over 3 years ago. Updated 10 months ago.

Status:Accepted Start date:08/18/2008
Priority:Normal Due date:
Assignee:- % Done:

0%
Category:fileserving
Target version:Telly
Affected Puppet version:0.24.4 Branch:
Keywords:
Votes: 1

Description

According to dtrace, checksuming (MD5) takes a lot of time for each catalog run. Anything that can be done to speed this up would be valuable.

3371-If-lite-is-not-specified-we-hand-the-reading-of-the.patch (1.5 kB) Steve McIntosh, 09/02/2008 09:49 pm

History

#1
Updated by James Turnbull over 3 years ago

  • Category set to fileserving
  • Status changed from Unreviewed to Needs Decision
  • Assignee set to Luke Kanies
  • Target version set to 4
  • 3 changed from Unknown to Hard

Might be solved by some work Luke is undertaking now.

#2
Updated by Luke Kanies over 3 years ago

  • Status changed from Needs Decision to Accepted
  • Assignee changed from Luke Kanies to Steve McIntosh

This is orthogonal to the work I’m doing.

Steve, don’t you have code to attach to the ticket?

#4
Updated by James Turnbull over 3 years ago

  • Status changed from Accepted to Needs More Information

#5
Updated by James Turnbull over 3 years ago

  • Status changed from Needs More Information to Needs Decision

#6
Updated by James Turnbull over 3 years ago

  • Assignee changed from Steve McIntosh to Luke Kanies

#7
Updated by Luke Kanies over 3 years ago

  • Status changed from Needs Decision to Ready For Checkin

#8
Updated by Luke Kanies over 3 years ago

  • Status changed from Ready For Checkin to Tests Insufficient

The code is good, but I’m pretty sure this will break the existing tests for the Checksums module.

#9
Updated by Luke Kanies over 3 years ago

I just ran the tests and the mocks have now changed:

1)
Mocha::ExpectationError in 'Puppet::Util::Checksums when using md5 should use incremental Digest::MD5 sums to calculate file checksums'
#.file('/path/to/my/file') - expected calls: 0, actual calls: 1
/Users/luke/git/puppet/lib/puppet/util/checksums.rb:76:in `checksum_file'
/Users/luke/git/puppet/lib/puppet/util/checksums.rb:20:in `md5_file'
./spec/unit/util/checksums.rb:55:in `send'
./spec/unit/util/checksums.rb:55:
/Users/luke/git/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:19:in `run'
/Users/luke/git/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/luke/git/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

2)
Mocha::ExpectationError in 'Puppet::Util::Checksums when using sha1 should use incremental Digest::SHA1 sums to calculate file checksums'
#.file('/path/to/my/file') - expected calls: 0, actual calls: 1
/Users/luke/git/puppet/lib/puppet/util/checksums.rb:76:in `checksum_file'
/Users/luke/git/puppet/lib/puppet/util/checksums.rb:49:in `sha1_file'
./spec/unit/util/checksums.rb:55:in `send'
./spec/unit/util/checksums.rb:55:
/Users/luke/git/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:19:in `run'
/Users/luke/git/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `each'
/Users/luke/git/puppet/spec/monkey_patches/add_confine_and_runnable_to_rspec_dsl.rb:17:in `run'

Finished in 0.082169 seconds

20 examples, 2 failures

#10
Updated by Luke Kanies over 1 year ago

  • Status changed from Tests Insufficient to Accepted
  • Assignee deleted (Luke Kanies)
  • Target version changed from 4 to 2.7.x

This is at least somewhat better now because we’re using 4096 blocks.

#11
Updated by Nigel Kersten 10 months ago

  • Target version changed from 2.7.x to Telly

Aren’t the tests still insufficient?

Also available in: Atom PDF