The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Feature #1154

Allow signed manifests to eliminate single point of compromise

Added by Jeff Goldschrafe about 6 years ago. Updated 12 months ago.

Status:AcceptedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:newfeature
Target version:-
Affected Puppet version:0.24.4 Branch:
Keywords:

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This ticket may be automatically exported to the PUP project on JIRA using the button below:


Description

Puppet, like all configuration management systems, suffers from the possibility of being a single point of compromise, allowing arbitrary instructions to be run on all hosts accessing the Puppetmaster if a malicious manifest is crafted. Since the goal of Puppet more or less necessitates Puppet running as root on client systems, the amount of damage capable of being inflicted on client nodes is virtually limitless, and some optional extra precautions should be provided in order to limit the damage capable of being caused by a single rooted Puppetmaster.

Signed manifests appear to be the easiest and most intuitive way to accomplish this. Like GPG-signed packages, they ensure that Puppet manifests have come from an authenticated source. By verifying the signature on manifests coming from the server, clients may verify that packages have been approved by the organization owning the Puppet server.

History

#1 Updated by Jeff Goldschrafe about 6 years ago

This is definitely a much more difficult problem to solve than static packages, because all of the responses from the Puppet server are obviously not going to be the same and can’t just be signed with the key on a different server. But something like this is what I’m thinking:

  1. A public/private key pair is generated. The public keys are distributed to the Puppet clients. The private key is encrypted with a passphrase.
  2. A secure mode is turned on for the Puppetmaster. In this mode, Puppetmaster does not automatically reload manifests. It must be restarted manually. When restarted, the Puppetmaster’s startup procedure prompts the user for a passphrase, like Apache when using encrypted certificates. This key is used to sign all communications between the Puppetmaster and clients.
  3. The clients verify each message against the public key. If the signature doesn’t match the signature the client is expecting, the manifest is discarded and a warning is raised.

It seems like this kind of model wouldn’t be difficult to shoehorn into Puppet’s existing SSL model, though I don’t know anything about how it ties into WEBrick/Mongrel internally.

This still has the following weaknesses:

  1. The server can be silently compromised, and before attempting to replace any manifests, the Puppetmaster binary can be replaced with one that fishes the passphrase or stores the unencrypted key for the attacker to use later.
  2. Once rooted, the key can be fished out of memory, whether obfuscated or not.

However, assuming a workable security infrastructure is already in place, and the administrator is capable of automatically verifying system binaries before reloading the manifests, this allows an administrator to detect the intrusion and simply correct the problem and reinstall the server before malicious manifests can be deployed across the network.

Regardless of approach, Puppet could really benefit from some mechanism to mitigate cross-network compromises.

#2 Updated by James Turnbull about 6 years ago

I don’t see how this helps prevent a compromised Puppet master from being used for malicious purposes?

#3 Updated by Russell Adams about 6 years ago

Given my efforts to use puppet across customer networks (ie: managed remotely over the net) I’ve been working on using encrypted and signed manifests from tarball distributed outside puppet. Unfortunately since the responses from the puppetmaster are dynamically generated it makes signing impractical. I sidestepped the issue by not using puppetmaster and just running puppet locally from authenticated manifests.

I am very interested in seeing where this discussion leads.

#4 Updated by Jeff Goldschrafe about 6 years ago

I somehow neglected to mention in that whole spiel that the manifests for the production environment could be encrypted using the same key, which would allow the Puppetmaster to verify their authenticity.

#5 Updated by Redmine Admin almost 6 years ago

  • Status changed from 1 to Needs Decision

#6 Updated by Luke Kanies over 5 years ago

  • Status changed from Needs Decision to Accepted
  • Affected Puppet version set to 0.24.4

I’ll accept this as a general goal, but there’s obviously a ton of design work necessary before anything like this is actually feasible. At the least, you need a clean differentiation between the manifests that the server uses to compile the catalogs, and the compiled catalogs themselves.

#7 Updated by James Turnbull almost 5 years ago

  • Assignee deleted (Puppet Community)

#8 Updated by Jason Ling 12 months ago

Perhaps instead of signing the entire manifest, this could be simplified and the admin would sign a timestamp instead – clients will then pull manifests only if master provides a signed timestamp within the last x hours?

Also available in: Atom PDF