Puppet

G'day.

I am curious how you would define the required security properties of the token: is this an arbitrary blob passed beside the CSR, or part of the CSR, or…?

How are you planning on generating and validating the token? How are you defending against forgery, or is that entirely a non-issue in your threat model?

Once we understand more about how you expect to use this, and expect others to use this, we will be in a much better position to understand the implications of the request.

I’m thinking in this case that it’s any blob of text, determined by the user — that’s even optional. The user might write a server-side script that just checks the ip against the hostname and returns 0… or in our case it reads the $ARG and checks it against an internal token system we built to authenticate new systems on bootup. It’s up to the implementer how to use it..

Passing the entire CSR in to the autosign code would be interesting, certainly… but in this particular case, I’m thinking about some “external token” that can be passed on the command line.

ie:

puppet agent --server <puppetserver> --token <some random text here>

… and on the server side i might have a shell script that says

#! /bin/sh
if [ $1 -eq "mytoken" ]; then
  exit 0;
else
  exit 1;
fi

… but obviously thats a very simple example

What does this token give us that the certname doesn’t?

If you use ‘certname’ as the authenticator, then either people have to intentionally override certname with their custom data (semi-OK), or people have to know the hostname in-advance of the host coming up … in that case, the normal autosign code would work just fine.

By allowing a separate token, the ‘certname’ can stay a valid hostname, but this separate token can be used to authenticate the host. In some environments it might be as simple as a password. In others (ours), it would actually check in to a database and see if that token is valid. If its valid, it can authenticate and get a cert. If its not, the host can never get its cert signed and it remains an untrusted host. We generate a ‘token’ in our case right before we create a new instance and we pass that token to the instance on bootup, giving us a small window where that host can properly authenticate into our system.

I’m proposing that this be done in a pretty generic way though,. so that people can adapt it to their own environments.

There’s no necessary relationship between certname and hostname at all.

Is there any reason you can’t just use the certname as your “token”?

Are your tokens always entered by a human operator?

Certname defaults to the hostname, so I say there is a relationship — even if its arbitrary. There is nothing that prevents us from using the token-ID as the certname — its just that in our case its a little strange. The token ID is a temporary thing that is destroyed once its used (or expires), so having it permanently embedded in the SSL certname just feels weird, thats all. In our case, the tokens are pushed automatically through our provisioning system. Its a bit hard to describe that in detail here, but its fair to say that the host boots up and knows that it has “xyz” as a token, and that it can use that token for some period of time to get its puppet cert signed.

Given the API access around all the SSL infrastructure we’ve added, is there any way you can use that instead?

It looks to me that what you really want is to be able to get your certs signed and deployed as part of the provisioning process via API calls rather than this intermediate token.

Sorry, there’s a big desire to use Puppet with some level of Autosigning in cloud-style environments. These are environments where there are likely to be “orchestration components” that handle the instantiation and termination of cloud instances. I think this is a feature that really allows people to be more flexible iwth their puppet environments, and doesnt seem too hard to implement. Is it more difficult than I’m imagining?

(ps, what API? :) )

Matt Wise wrote:

Sorry, there’s a big desire to use Puppet with some level of Autosigning in cloud-style environments. These are environments where there are likely to be “orchestration components” that handle the instantiation and termination of cloud instances. I think this is a feature that really allows people to be more flexible iwth their puppet environments, and doesnt seem too hard to implement. Is it more difficult than I’m imagining?

It does require a reasonable set of changes, and my intuition is that people are going to want access to the CSR itself as well as this token idea, so I’m trying to work out what the minimal set of changes are that can solve the actual problem.

(ps, what API? :) )

It’s all new for 2.7.x, and I’m working on getting those docs up on the site before 2.7.0 release. I’ll post back here as soon as they’re ready.

Nigel Kersten wrote:

Given the API access around all the SSL infrastructure we’ve added, is there any way you can use that instead?

It looks to me that what you really want is to be able to get your certs signed and deployed as part of the provisioning process via API calls rather than this intermediate token.

Yes, I think there’s a bit of confusion here though. Talking with lots of different people about this, I’d like to propose framing the problem in the following way:

A certificate signing request itself should be authenticated. I think we’re all on the same page here. Where a lot of contention comes in is on the question of the certname itself being sufficient authentication or not.

I propose the answer is no. The certname itself is not sufficient authentication because it requires some action to be carried out either on the Puppet Master or on the Puppet Agent to massage the certname into an “authenticated” pattern.

The goal we should be striving for is that new systems can come online without any human interaction with the system at large in an authenticated manner. For the certname to be the authenticator, complex machinery would need to be added to replace the human interaction.

I think a simple string passed along with (but not inside) the CSR in the API call combined with a server side system call passing the string in the argument vector is an easy and robust solution to this problem.

-Jeff

Issue #7244 has been updated by Jeff McCune.

Nigel Kersten wrote:

Given the API access around all the SSL infrastructure we’ve added, is there any way you can use that instead?

It looks to me that what you really want is to be able to get your certs signed and deployed as part of the provisioning process via API calls rather than this intermediate token.

Yes, I think there’s a bit of confusion here though. Talking with lots of different people about this, I’d like to propose framing the problem in the following way:

A certificate signing request itself should be authenticated. I think we’re all on the same page here. Where a lot of contention comes in is on the question of the certname itself being sufficient authentication or not.

I propose the answer is no. The certname itself is not sufficient authentication because it requires some action to be carried out either on the Puppet Master or on the Puppet Agent to massage the certname into an “authenticated” pattern.

Correct me if I am wrong, but wouldn’t setting the additional “token” also require some action to be taken on the Agent and/or the Master to “massage the token into an ‘authenticated’ pattern”?

The goal we should be striving for is that new systems can come online without any human interaction with the system at large in an authenticated manner. For the certname to be the authenticator, complex machinery would need to be added to replace the human interaction.

I think a simple string passed along with (but not inside) the CSR in the API call combined with a server side system call passing the string in the argument vector is an easy and robust solution to this problem.

I think we would be much better, as a strategy for this, drawing up the proposed security model and giving it a thorough review. I still can’t see how this extra token is either less work, or more secure, than using the certname would be. Both are just as verifiable, since they are literally the same data, passed in different places. The only benefit I can see is that the pair of values might, in theory, be more secure, if you have some out of band key used to secure the token against the certname.

Daniel Pittman wrote:

Issue #7244 has been updated by Jeff McCune.

Nigel Kersten wrote:

Given the API access around all the SSL infrastructure we’ve added, is there any way you can use that instead?

It looks to me that what you really want is to be able to get your certs signed and deployed as part of the provisioning process via API calls rather than this intermediate token.

Yes, I think there’s a bit of confusion here though. Talking with lots of different people about this, I’d like to propose framing the problem in the following way:

A certificate signing request itself should be authenticated. I think we’re all on the same page here. Where a lot of contention comes in is on the question of the certname itself being sufficient authentication or not.

I propose the answer is no. The certname itself is not sufficient authentication because it requires some action to be carried out either on the Puppet Master or on the Puppet Agent to massage the certname into an “authenticated” pattern.

Correct me if I am wrong, but wouldn’t setting the additional “token” also require some action to be taken on the Agent and/or the Master to “massage the token into an ‘authenticated’ pattern”?

The goal we should be striving for is that new systems can come online without any human interaction with the system at large in an authenticated manner. For the certname to be the authenticator, complex machinery would need to be added to replace the human interaction.

I think a simple string passed along with (but not inside) the CSR in the API call combined with a server side system call passing the string in the argument vector is an easy and robust solution to this problem.

I think we would be much better, as a strategy for this, drawing up the proposed security model and giving it a thorough review. I still can’t see how this extra token is either less work, or more secure, than using the certname would be. Both are just as verifiable, since they are literally the same data, passed in different places. The only benefit I can see is that the pair of values might, in theory, be more secure, if you have some out of band key used to secure the token against the certname.

The certname itself is not and should not b authenticated. We can’t assume it’s known ahead of time. We (Puppet) use the cert name as the unique identifier of the host thorugh the indirector, reporting, dashboard, etc… We should absolutely not overload this any more by also requiring it to be the sole authenticating factor for CSR’s. We’ll quickly get into competing goal situations. (I want 1,000,000 nodes to all have a unique ID but share a single authentication token, as one example)

I got confused with all the talk of “certname” I think the outstanding question is whether or not to include the authentication data inside the CSR or along side the CSR.

We definitely need something more than certname. Simply by virtue of the fact that we often don’t actually know or care about the certname. The question for me is whether or not the authentication token should be inside the CSR or not.

-Jeff

Although in my case specifically, having the token in the CSR would be nice — I actually think that nobody else in their right mind would use it. :) I suggest that its not part of the CSR, but rather just something that gets brought along in the initail auto sign request.

(on a side note, I have a separate ticket open asking for the ability to throw some unique, unchangeable data into the certificate. this is for a different purpose, and is still extremely important to me down the road. but this has nothing to do with the initial autosign issue)

Matt Wise wrote:

Although in my case specifically, having the token in the CSR would be nice — I actually think that nobody else in their right mind would use it. :) I suggest that its not part of the CSR, but rather just something that gets brought along in the initail auto sign request.

What I’m envisioning regardless of if it’s in CSR or not is the use case of:

puppet agent --test --certname=$(uuidgen) --csrtoken=12345
# and
puppet cert --generate --certname=$(uuidgen) --csrtoken=12345

Matt, do you still think nobody would use this if “behind the scenes” the CSR token were embedded in the CSR itself rather than along side the CSR in the API call?

I think we can largely hide the implementation from the end user.

(on a side note, I have a separate ticket open asking for the ability to throw some unique, unchangeable data into the certificate. this is for a different purpose, and is still extremely important to me down the road. but this has nothing to do with the initial autosign issue)

Thanks for noting this, I’ve marked the two tickets as “related” to each other.

-Jeff

Matt Wise wrote:

(ps, what API? :) )

http://docs.puppetlabs.com/guides/rest_api.html

particularly the stuff around CSRs.

I see these as two very different requests… Bug #7243 will let me implement security around what classes a host can request once it has a cert. This request allows me to create my own ‘authorization’ system for autosigning. They’re both very important, but this one is actually more important for me right now.

Any update on this feature request? Its been a long time now …

Matt – as indicated in the ticket it’s scheduled at the Telly release which is our next major feature release. I can get Nigel to discuss with you the schedule for that release.

+1 for this, it’d be wonderful to have a hook (external script or a class to extend) to link autosigning into business logic. The workflow I’m faced with is:

• new clients should not have to do anything special to get set up with a functional ssl cert
• re-imaged clients should not have to worry about downloading their previous incarnation’s cert (“certificate does not match private key” error)

The original plan was to use a shared client cert but there are substantial problems with that. if we go with per-client certificates, the options are to pre-generate or otherwise escrow the client’s private key centrally, deliver it at bootstrap time and not regenerate it upon re-kick; or permit autosigning but only to authorized clients, and hook into the autosigning to scrub pre-existing certs.

Wow, got into a crazy redmine loop when trying to edit my previous comment — I meant to add:

The hook would check for a “provisioning mode” flag in the CMDB and if it’s set, permit a clean+reissue; if not, signing would be denied.

eric sorenson wrote:

Escrow the client’s private key centrally, deliver it at bootstrap time and not regenerate it upon re-kick

This is exactly what we do now. We have a sinatra application running on our puppet masters that handles the certificate signing, and we have a manual cURL step for clients to download their certificates out-of-band, put them in the right place, and then run puppet. The reason being, is that we check that a host is supposed to be running puppet through various LDAP attributes and validate that the host requesting the Private Key is the actual host (through various methods, since we have a load balancer in front of the puppet masters).

Having the ability to control the autosign process would allow us, I think, to get rid of this extra service and handle this all in band-with puppet so we can use the puppet cert commands, etc.

So… +1.

Of interest to this thread: http://puppetlabs.com/blog/the-design-behind-puppet-sites/

Gah, I could swear I saw another ticket with a github branch that implemented an external policy hook for signing, but I’ve just spent 20 minutes searching in vain. Anyone else on the ticket remember seeing this?

Daniel Pittman wrote:

However, I still object to the idea that the external approval script gets any input other than the CSR and, perhaps, the “live” attributes of the request such as the IP address from which it came. (I see them as risky, however, since a move to, eg, a message based request for the certificate would make them impossible to determine, or NAT may mask the actual origin, or a bounce attack could allow someone to reach from the “trusted” IP to the server. :)

I think providing the “live” attributes as you call them is critical information. This information doesn’t have to be used, but it should be made available as it’s one of the few bits of information that can’t be forged. Really the only bit of information I can think of would be the IP, and while NAT or some other proxy might mask the origin IP, that’s the architect’s problem, not puppet’s (we allow autosigning of certs based on matching of the cert name, and that’s certainly far more insecure than matching on an IP).

The goal I wish to achieve by using this is to autosign certs from EC2 hosts. The idea I have in mind is that cert signing requests will come in containing the EC2 machine’s instance ID. A script would then look up that instance ID and verify that the IP the cert sign request came from matches the IP for that instance. If it does match, the cert is signed.

Providing additional info inside the CSR or API call would not be possible if using something like the Amazon ELB auto scaler, as all nodes will contain the same data. There wouldn’t be any way to give the node a one-time-use passphrase or something to put in the request.

Patrick Hemmer wrote:

Attached is a patch (against 3.1.0) which adds an autosign_command parameter to the config. When this parameter is set, the specified external command will be called when a certificate request comes in. The external command receives the cert name as a command argument. The command must then return 0 if the cert should be signed, or non-zero if it should not be signed.

I have also added a patch to bug #7243 which allows adding extra parameters to the CSR.

This code is functional and in use in my own environment.

Note: An example of what is possible with this. I use it with autoscaled ec2 instances. The instance uses the patch on bug #7243 to add parameters to the CSR. It adds the autosigning secret key to verify that the box is legitimately mine, and the name of the puppet dashboard group the box should be added to. My autosigning command then adds the server to the puppet dashboard group. I can provide this script if anyone is interested.

Thanks for the patch, could you please submit this as a pull request following our contributing guide?

-Jeff

I’ve been meaning to take the time to write up the tests for this, just haven’t had time yet. I did however run into one issue the other day with the patch on #7243 that required a small fix. Updated the patch on that bug.

I’m qubit on irc or patrick.hemmer@gmail.com.