Codename: Lowrider¶

Problem¶

In order for modules to be truly reusuable, you need to be able to supply site-specific data outside the module itself.

Authors however need to be able to supply data themselves with modules, but well separated outside the manifests so that users can override data.

Overview¶

We’re going to build Hiera-like functionality into Puppet itself as a first class citizen, using the Hiera code base as a starting point, and freezing the existing Hiera project with minimal maintenance by Puppet Labs.

“hiera” will generally be retired as a user facing element with the noun “data” and the verb “lookup”.

Backwards compatibility will be provided by existing Hiera installs, so it’s critical that users can continue to run Hiera alongside Territory while they do any necessary migration. Existing Hiera users are important to us.

At this stage we are looking to support a single on-disk data format, JSON.

Goals¶

1. Automatically look up unspecified class parameters in the data store.
2. Allow manual look up of data in the data store by explicit keys.
3. Provide a Face to interact with the data store.

Out of scope, to implement later¶

1. A remote read/write API/service wrapped around Hiera.

Out of scope, no plan to implement later.¶

1. Allow module authors to ship data in their modules.

We have a general principle we’re looking to promote where class parameters are considered your interface for the user of your module. Just like you wouldn’t make GUI elements that control aspects of a program your users shouldn’t need to modify, you shouldn’t make class parameters for everything. This should be kept in mind with the design.

We are not interacting with an external Hiera install here, we are creating Hiera-like functionality in Puppet core, using the Hiera code base.

Automatically look up unspecified class parameters in the data store¶

class ntp($servers=['time.puppetlabs.com']) {
  ...
}

Lookup Order¶

1. Directly specified value
   • class { ntp: servers => ["ntp.puppetlabs.lan", "ntp2.puppetlabs.lan"] }
2. Value via Hiera when not directly specified
   • class { ntp: } will lookup the key ntp::servers
3. Default value in class definition when not directly specified or found in Hiera
   • class { ntp:} and class ntp ($servers=["default.server"]) { ... }

Note the change from existing Hiera where we are now going to be looking up the fully qualified variable name as the key, avoiding the existing problem of a global namespace being polluted with ‘ntp_ntpservers’.

Default Hierarchy¶

Proposed:

:hierarchy:
    - %{certname}
    - %{environment}
    - global

Note the rename of ‘common’ in Hiera terminology to ‘global’, and associated design question.

Allow manual look up of data in the data store by explicit keys¶

Puppet manifests should have access to the data store using a data lookup function like this:

class site {

  class { 'ntp':
    ntpservers => lookup('dmz_ntpservers'),
  }

  $maintenance_message = lookup('dmz_maintenance_state_text')

  file { '/etc/motd':
    content => $maintenance_message,
  }

}

This is where users can lookup arbitrary keys and use the returned data in class or resource declarations, or really for any DSL purpose.

Puppet Data Face:¶

The current Hiera CLI functionality will be ported to a Face with the following feature set:

• Ability to read data from the data store
• Ability to query values based on a user supplied scope

Example:

# Query the data backend for var_name:
$ puppet data search var_name

# Query the data backend for var_name with a specific scope:
$ puppet data search var_name --scope "key=value"

Puppet Deprecations¶

• extlookup function

Hiera functionality that will not survive the transition¶

• “puppet” backend for Hiera
• YAML support
• We won’t port hiera_include() as we’re going to fix #7801

Rough User Stories/Desires¶

• As a user, I want to be able to migrate a node from prod –> test –> prod without needing to copy all the data back and forth.
  • This is the driver behind having the environment in the default hierarchy, rather than environment-specific data stores.

Outstanding Questions¶

General¶

• ???
• ???

Design¶

• Is our default hierarchy right?
• Is ‘global’ the right name for the previously-known-as ‘common’ hierarchical layer?
• What do we name the function(s) for data lookup? Is lookup() right?
• With regard to the replacement of hiera_include(), do we need to enforce a convention here around the key lookup?
• What do we do about the existing hiera config file? That can’t be expressed in puppet.conf can it?