The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

This issue tracker is now in read-only archive mode and automatic ticket export has been disabled. Redmine users will need to create a new JIRA account to file tickets using https://tickets.puppetlabs.com. See the following page for information on filing tickets with JIRA:

Bug #6117

Catalog GETs should shift to POST

Added by eric sorenson about 5 years ago. Updated over 3 years ago.

Status:ClosedStart date:02/02/2011
Priority:HighDue date:
Assignee:-% Done:

0%

Category:API
Target version:2.7.0
Affected Puppet version: Branch:
Keywords:catalog REST

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com


Description

Hi, looking at Bug #2668 after some discussion on IRC today, I thought this would be worth re-opening.

Brice gzip+b64 encoded the URLs which definitely helps, but IMO it just pushes the problem out a little bit. Fact lists are always getting bigger and it’d be great not to have to worry about running into URL length limits and clean up the requests at the same time.

Brice Figureau wrote:

I don’t think so, because we’re GETting a catalog. To be RESTful, we should keep using GET.

IMO this semantic correctness is way, way, way less important than doing something sane. Shifting catalog GETs over to POSTs, with the body as facts in x-www-form-urlencoded , allows arbitrary length of facts without making insane URLs.


Related issues

Related to Puppet - Bug #2668: Too many facts: request-URI Too Large Duplicate 09/22/2009
Related to Puppet - Bug #18781: Be more tolerant of old clients in WEBrick server Closed
Related to Puppet - Bug #20901: puppet --version is unnecessarily slow Closed

History

#1 Updated by eric sorenson about 5 years ago

Whups, deleting my recursive update.

#2 Updated by James Turnbull about 5 years ago

  • Category set to API
  • Status changed from Unreviewed to Needs Decision
  • Assignee set to Nigel Kersten
  • Target version set to 2.7.x

#3 Updated by Nigel Kersten about 5 years ago

  • Status changed from Needs Decision to Accepted
  • Assignee deleted (Nigel Kersten)

#4 Updated by Nigel Kersten about 5 years ago

  • Priority changed from Normal to High

#5 Updated by Anonymous about 5 years ago

From an architecture point of view, I very strongly prefer the “everything has one, unique, URI” part of REST to the “let us create a terrible semantic mapping of user verbs to HTTP verbs, even though they don’t really match, and so make things awfully ugly”.

Especially when the resource is not actually, y'know, idempotent, so you can’t cache it in between, which is what the GET part of REST is supposed to enable here.

#6 Updated by Matt Robinson about 5 years ago

It looks like something to fix this issue will be necessary before we can use puppet under Ruby 1.9, at least with Webrick. Webrick in Ruby 1.9 seems to have a much lower tolarance for large GET requests. A very simple set of facts that works on master running on Ruby 1.8.7 fails under Ruby 1.9.2 like so:

err: Could not retrieve catalog from remote server: Error 414 on SERVER: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
  <HEAD><TITLE>Request-URI Too Large</TITLE></HEAD>    
  <BODY>    
    <H1>Request-URI Too Large</H1>    
     WEBrick::HTTPStatus::RequestURITooLarge    
    <HR>    
    <ADDRESS>WEBrick/1.3.1 (Ruby/1.9.2/2010-12-25) OpenSSL/0.9.8l at      mattmac.puppetlabs.lan:8140    </ADDRESS>    
  </BODY>
 </HTML>

#7 Updated by Cody Robertson about 5 years ago

Matt Robinson wrote:

It looks like something to fix this issue will be necessary before we can use puppet under Ruby 1.9, at least with Webrick. Webrick in Ruby 1.9 seems to have a much lower tolarance for large GET requests. A very simple set of facts that works on master running on Ruby 1.8.7 fails under Ruby 1.9.2 like so:

err: Could not retrieve catalog from remote server: Error 414 on SERVER: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
  <HEAD><TITLE>Request-URI Too Large</TITLE></HEAD>    
  <BODY>    
    <H1>Request-URI Too Large</H1>    
     WEBrick::HTTPStatus::RequestURITooLarge    
    <HR>    
    <ADDRESS>WEBrick/1.3.1 (Ruby/1.9.2/2010-12-25) OpenSSL/0.9.8l at      mattmac.puppetlabs.lan:8140    </ADDRESS>    
  </BODY>
 </HTML>

You can use something like NGINX + Mongrel and raise large_client_header_buffers option. Other web servers (or reverse proxies) have similar options to allow for larger requests.

#8 Updated by Nick Lewis about 5 years ago

  • Status changed from Accepted to Merged - Pending Release

Merged to next in commit:d748338e69b705585f9ac6bf2fd8da1e9163839b.

Find requests in the REST terminus are now turned into POST requests if they are large, and GET requests if they are small for backward compatibility. POST requests coming into the HTTP handler are turned into find requests if they are singular. This has the effect of using POST for large catalog requests.

#9 Updated by James Turnbull almost 5 years ago

  • Status changed from Merged - Pending Release to Closed
  • Target version changed from 2.7.x to 2.7.0

#10 Updated by John Florian over 3 years ago

Am I to understand correctly that this should be fixed now in the 2.7.x series? If so, why did I start experiencing this only after upgrading to 2.7? With a Fedora 17 puppet master (running puppet-2.7.18-1.fc17.src.rpm), I get thousands of worthless tagmail reports saying …

2012-08-27 08:14:22 -0400 Puppet (err): Could not retrieve catalog from remote server: Error 414 on SERVER: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
<HEAD><TITLE>Request-URI Too Large</TITLE></HEAD>
<BODY>
<H1>Request-URI Too Large</H1>
WEBrick::HTTPStatus::RequestURITooLarge
<HR>
<ADDRESS>
WEBrick/1.3.1 (Ruby/1.9.3/2012-04-20) OpenSSL/1.0.0j at
puppet.mycompany.com:8140
</ADDRESS>
</BODY>
</HTML>
2012-08-27 08:14:22 -0400 Puppet (notice): Using cached catalog
2012-08-27 08:14:22 -0400 Puppet (err): Could not retrieve catalog; skipping run

… unless I have the following patch in place:

diff -up /usr/share/ruby/webrick/httprequest.rb.orig /usr/share/ruby/webrick/httprequest.rb
--- /usr/share/ruby/webrick/httprequest.rb.orig 2011-07-22 08:49:34.000000000 -0400
+++ /usr/share/ruby/webrick/httprequest.rb      2012-06-13 09:56:47.944953176 -0400
@@ -290,7 +290,7 @@ module WEBrick
private
-    MAX_URI_LENGTH = 2083 # :nodoc:
+    MAX_URI_LENGTH = 4096 # :nodoc:
def read_request_line(socket)
@request_line = read_line(socket, MAX_URI_LENGTH) if socket

#11 Updated by James Turnbull over 3 years ago

John – your problem is related – the fix is in place with 2.7.x and POST replaced GET. The problem you have is in Webrick not Puppet. I would recommend NOT using the default Webrick server in favour of something like Apache and Passenger.

#12 Updated by John Florian over 3 years ago

It’s a shame the default web server is not better supported than this. Fedora is getting closer to having Passenger, but as yet it’s still not there and working outside the default repositories is considered a bad practice at my company.

Due to this and other problems I keep facing with the default puppet master, I’m a forced to consider abandoning it altogether and rely purely on another solution we’ve worked out where offline clients operate more reliably. This involves rsync'ing the resources to the client and running each client in the “one shot” mode via a cron job/DHCP hooks, etc. I’d rather not go this route as I believe the official puppet master should have some advantages, but as yet that is not the case for us.

#13 Updated by Peter Meier over 3 years ago

John Florian wrote:

It’s a shame the default web server is not better supported than this. Fedora is getting closer to having Passenger, but as yet it’s still not there and working outside the default repositories is considered a bad practice at my company.

you can still run it with Mongrel and an Apache or Nginx fronted for the load balancing amongst the instances, as well as the ssl part. This is supported out of the box by the init.d script on Fedora-based systems (at least I contributed that part to the official repo and it is in EPEL’s rpms). Have a look at /etc/sysconfig/puppetmaster to run the puppetmaster as (multiple) mongrel instances and the documentation to set up the webserver related part. There is no need to install any non-supported packages.

Also, while passenger is “easier” to setup and some people claim that it has better performance than a mongrel/apache setup, there are also reports that tell that the latter has at least a similar performance. And it’s definitely always faster than a pure webrick based installation.

While webrick is easy to setup, there is not much that puppet can do to support it better, because it’s more or less seen as a development server or something easy and quick to go. But if you want to have a proper setup you should go with something else.

#14 Updated by John Florian over 3 years ago

Peter Meier wrote:

John Florian wrote:

It’s a shame the default web server is not better supported than this. Fedora is getting closer to having Passenger, but as yet it’s still not there and working outside the default repositories is considered a bad practice at my company.

you can still run it with Mongrel and an Apache or Nginx fronted for the load balancing amongst the instances, as well as the ssl part. This is supported out of the box by the init.d script on Fedora-based systems (at least I contributed that part to the official repo and it is in EPEL’s rpms).

Peter, thanks for the info. However, I don’t see Mongrel in Fedora 17. It was present in F16, but I was forced to update our master to 17 in order to support clients that are now on 17. To wit:

On F16:

$ yum search mongrel
[...]
rubygem-mongrel_cluster.noarch : GemPlugin wrapper for the mongrel HTTP server
rubygem-mongrel.x86_64 : A small fast HTTP library and server for Ruby apps

On F17:

$ yum search mongrel
[...]
rubygem-shotgun.noarch : Automatic reloading version of the rackup command
rubygem-thin.x86_64 : A thin and fast web server

Or did it just get renamed to “thin”? I know nothing of Mongrel, but I do see this suggesting my hunch might be right:

$ yum info rubygem-thin
Loaded plugins: downloadonly
Available Packages
Name        : rubygem-thin
Arch        : x86_64
Version     : 1.3.1
Release     : 3.fc17
Size        : 180 k
Repo        : fedora
Summary     : A thin and fast web server
URL         : http://code.macournoyer.com/thin/
License     : (GPLv2 or Ruby) and BSD and MIT
Description : Thin is a Ruby web server that glues together three of the best Ruby
: libraries in web history.
: The Mongrel parser, the root of Mongrel speed and security,
: Event Machine, a network I/O library with extremely high scalability and
: Rack, a minimal interface between webservers and Ruby frameworks.

Given my lack of Mongrel knowledge, I’ve no way of knowing for sure and have scarce time to experiment unless I can be confident the effort should be worth it.

#15 Updated by Daniel Drake over 3 years ago

Thanks a lot for improving the client here to avoid the large GET requests.

I’m wondering if we can do slightly better on the backwards compatibility aspect though – as noted above, simple setups that worked under previous versions of ruby are now rejected with URI length errors when old clients are involved. I have posted a patch in new ticket #18781 for consideration.

Also available in: Atom PDF