The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Bug #9084

Mixing and matching ruby versions for puppetmasterd and puppetd causes a "certificate verify failed" error

Added by Omar Qureshi almost 3 years ago. Updated over 1 year ago.

Status:AcceptedStart date:08/17/2011
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:ruby19
Target version:3.x
Affected Puppet version:2.7.3 Branch:
Keywords:ssl ruby19 ree centos certificate verify failed

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This ticket may be automatically exported to the PUP project on JIRA using the button below:


Description

Having realised that puppet now works on Ruby 1.9, I decided to forgo the installation of REE and just use Ruby 1.9.2 (installed via RVM) instead for installing puppet. However, doing so and trying to connect for the initial sign to the puppetmaster results in the following error on the client:

err: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed. This is often because the time is out of sync on the server or client

First thing I did was made sure ntpd was running on both machines and made sure they were synced to the same server, after that the date difference between the two servers was negligible (couple of ms).

Looking on the puppetmaster in the masterhttp.log file I get:

[2011-08-17 23:42:30] ERROR OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1 alert unknown ca

/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:44:in `accept'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:44:in `listen'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:173:in `call'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/local/rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/webrick/server.rb:82:in `start'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:42:in `listen'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:41:in `initialize'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:41:in `new'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:41:in `listen'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:38:in `synchronize'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/http/webrick.rb:38:in `listen'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/server.rb:127:in `listen'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/network/server.rb:142:in `start'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/daemon.rb:124:in `start'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/application/master.rb:202:in `main'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/application/master.rb:144:in `run_command'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/application.rb:307:in `run'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/application.rb:411:in `hook'
/usr/local/rvm/gems/ree-1.8.7-2011.03@puppet/gems/puppet-2.7.3/lib/puppet/application.rb:307:in `run

Ruby version on puppetmasterd is REE 2011.03. Ruby version on puppetd node is 1.9.2p290. Both running on CentOS 5.6 64-bit.

Can anyone else replicate this and clarify this?

Many thanks in advance!


Related issues

Related to Puppet - Bug #8858: Ruby 1.9 defaults HTTPS connections to "peer verify" rath... Closed 06/03/2012

History

#1 Updated by Brian Racer almost 3 years ago

I’m also running into this issue. My environment:

master: clean install of ubuntu 10.04, rvm ruby-1.9.2-p290, puppet 2.7.3
client: clean install of ubuntu 10.04, rvm ruby-1.9.2-p290, puppet 2.7.3

My setup script: https://gist.github.com/1161161

Downgrading to ruby-1.8.7-p352 under rvm seems to solve the issue. I have yet to try other versions of 1.9.2.

#2 Updated by James Turnbull almost 3 years ago

  • Status changed from Unreviewed to Accepted
  • Target version set to 2.7.x

#3 Updated by Nilesh L almost 3 years ago

I am running into this issue while running on agent trying to connect to puppet master on puppet

root@agent:~# puppet agent --server puppet --waitforcert 60 --test
err: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client

My environment on both agent and puppet master are identical.

RubyGems Environment:
- RUBYGEMS VERSION: 1.8.10
- RUBY VERSION: 1.9.2 (2011-07-09 patchlevel 290) [x86_64-linux]
- INSTALLATION DIRECTORY: /usr/lib/ruby/gems/1.9.2
- RUBY EXECUTABLE: /usr/bin/ruby1.9.2
- EXECUTABLE DIRECTORY: /usr/bin
- RUBYGEMS PLATFORMS:
- ruby
- x86_64-linux
- GEM PATHS:
- /usr/lib/ruby/gems/1.9.2
- /root/.gem/ruby/1.9.2
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :benchmark => false
- :backtrace => false
- :bulk_threshold => 1000
- REMOTE SOURCES:
- http://rubygems.org/

#4 Updated by Nilesh L almost 3 years ago

folks seem to have found a workaround

http://groups.google.com/group/puppet-users/msg/72bf694d4e2f3012

http://urgetopunt.com/puppet/2011/09/14/puppet-ruby19.html

#5 Updated by Nilesh L almost 3 years ago

Based on http://urgetopunt.com/puppet/2011/09/14/puppet-ruby19.html

scp user@puppetmaster://etc/puppet/ssl/certs/ca.pem /etc/puppet/ssl/certs/

ln -s /etc/puppet/ssl/certs/ca.pem /usr/lib/ssl/certs/$(openssl x509 -hash -noout -in /etc/puppet/ssl/certs/ca.pem).0

#6 Updated by Sergey Alembekov almost 3 years ago

I got same problems with puppet 2.7.5 and ruby 1.9.2p188

#7 Updated by sumit vij almost 3 years ago

I am facing the same problem. Workaround didn’t worked for me. I am using ruby 1.9.3.

#8 Updated by Marcin Deranek over 2 years ago

Try:

scp user@puppetmaster:/etc/puppet/ssl/certs/ca.pem /etc/puppet/ssl/certs/
ln -s /etc/puppet/ssl/certs/ca.pem $(openssl version -d|cut -d\" -f2)/certs/$(openssl x509 -hash -noout -in /etc/puppet/ssl/certs/ca.pem).0

#9 Updated by Bryan Bishop over 2 years ago

I ran into the same issue using RVM and ruby 1.9.2-p290. This works for me using 1.9.2-p180

#10 Updated by Bryan Bishop over 2 years ago

Sorry spoke too soon, I got further with p180 but it did NOT work.

#11 Updated by Bryan Bishop over 2 years ago

So yes, I CAN get this to work on 1.9.2-p180 using puppet -v 2.7.6. I’m going to try newer versions of 1.9.2 and puppet and see when this breaks.

#12 Updated by Bryan Bishop over 2 years ago

I actually was able to make this work with 1.9.3-p0 and every version of puppet except for 2.7.10. So I’ll be sticking with 2.7.9 and 1.9.3-p0

#13 Updated by Scott Wang over 2 years ago

Marcin Deranek wrote:

Try: […]

We are running CentOS 6.2 with openssl-1.0.0-20

The default part of the certificate to be hashed is changed so running

ln -s /etc/puppet/ssl/certs/ca.pem $(openssl version -d|cut -d\" -f2)/certs/$(openssl x509 -hash -noout -in /etc/puppet/ssl/certs/ca.pem).0    

will produce a different name of the link to the certificate in CentOS 6.

It should be changed to this instead if you are running on the new version:

ln -s /etc/puppet/ssl/certs/ca.pem $(openssl version -d|cut -d\" -f2)/certs/$(openssl x509 -subject_hash_old -noout -in /etc/puppet/ssl/certs/ca.pem).0    

#14 Updated by Anonymous over 2 years ago

  • Target version changed from 2.7.x to 3.x

#15 Updated by Andrew Jones about 2 years ago

Scott Wang wrote:

Marcin Deranek wrote:

Try: […]

We are running CentOS 6.2 with openssl-1.0.0-20

The default part of the certificate to be hashed is changed so running […] will produce a different name of the link to the certificate in CentOS 6.

It should be changed to this instead if you are running on the new version: […]

I found the opposite to be true. I’m using CentOS 6.2 with OpenSSL 1.0.0-20, but with Ruby 1.9.3.

On 5.7 and 5.8 w/ Ruby 1.9.3, the old hash is required. On 6.2 w/ Ruby 1.9.3, the new hash is required.

As a result: with 1.9.3 you always want -hash, not -subject_hash_old.

#16 Updated by David Schoen over 1 year ago

I’ve tried the work around for this both with -hash and -subject_hash_old without success. The times on both master and client are within a few seconds and the ca.pem is way inside the valid range.

I’ve got freshish installs of ruby/puppet straight out of Ubuntu 12.04 repos.

Has anyone got any other tricks to try for working around this issue?

Also available in: Atom PDF