Overview
What?¶
A simple pluggable Hierarchical Database.
Why?¶
Hierarchical data is a good fit for the representation of infrastructure information. Consider the example of a typical company with 2 datacenters and on-site development, staging etc.
All machines need:
- ntp servers
- sysadmin contacts
By thinking about the data in a hierarchical manner you can resolve these to the most correct answer easily:
/------------- DC1 -------------\ /------------- DC2 -------------\
| ntpserver: ntp1.dc1.example.com | | ntpserver: ntp1.dc2.example.com |
| sysadmin: dc1noc@example.com | | |
| classes: users::dc1 | | classes: users::dc2 |
\-------------------------------/ \-------------------------------/
\ /
\ /
/------------- COMMON -------------\
| ntpserver: 1.pool.ntp.org |
| sysadmin: sysadmin@%{domain} |
| classes: users::common |
\----------------------------------/
In this simple example machines in DC1 and DC2 have their own NTP servers, additionaly DC1 has its own sysadmin contact – perhaps because its a remote DR site – while DC2 and all the other environments would revert to the common contact that would have the machines domain fact expanded into the result.
The classes variable can be searched using the array method which would build up a list of classes to include on a node based on the hierarchy. Machines in DC1 would have the classes users::common and users::dc1.
The other environment like development and staging would all use the public NTP infrastructure.
This is the data model that extlookup() have promoted in Puppet, Hiera has taken this data model and extracted it into a standalone project that is pluggable and have a few refinements over extlookup.
Enhancements over Extlookup?¶
Extlookup had just one backend, Hiera can be extended with your own backends and represent a few enhancements over the base Extlookup approach thanks to this.
Multiple backends are queried¶
If you have a YAML and Puppet backend loaded and your users provide module defaults in the Puppet backend you can use your YAML data to override the Puppet data. If the YAML doesnt provide an answer the Puppet backend will get an oppertunity to provide an answer.
More scope based variable expansion¶
Extlookup could parse data like %{foo} into a scope lookup for the variable foo. Hiera retains this ability and any Arrays or Hashes will be recursively searched for all strings that will then be parsed.
The datadir and defaults are now also subject to variable parsing based on scope.
No CSV support by default¶
We have not at present provided a backward compatible CSV backend. A converter to YAML or JSON should be written. When the CSV backend was first chosen for Puppet the Puppet language only supports strings and arrays of strings which mapped well to CSV. Puppet has become (a bit) better wrt data and can now handle hashes and arrays of hashes so it’s a good time to retire the old data format.
Array Searches¶
Hiera can search through all the tiers in a hierarchy and merge the result into a single array. This is used in the hiera-puppet project to replace External Node Classifiers by creating a Hiera compatible include function.
- Homepage: https://github.com/puppetlabs/hiera
- Subprojects: Hiera-Puppet
Issue tracking
- Bug: 17 open / 67
- Feature: 17 open / 35
- Support: 1 open / 2
- Refactor: 0 open / 2
- High Level Feature: 0 open / 0
Members
Manager: Daniel Pittman, Kelsey Hightower, Ken Barber, Max Martin, Moses Mendoza
Developer: Adam Crews, Adrien Thebo, Alanna Brown, Alex Dreyer, alice nodelman, Aliza Earnshaw, Amy Hoang, Ananda Osterhaus, Andrew Parker, Andrew Roetker, Ann Franck, Arnan Outhaythip, Ben Ford, Ben Reich, Brad Hamilton, Branan Purvine-Riley, Brent Caldwell, Brent Nikolaus, Brian Cain, Brian Lake, Brian Stein, Briana de Jong, Candy Parker, Carl Caum, Carrie Odell, Carthik Sharma, Celia Cottle, Chad Jawors, Char Moe, Charles Dunbar, Charlie Sharpsteen, Charlotte Caswell, Chris Barker, Chris Grim, Chris Hoge, Chris Price, Chris Roddy, Chris Spence, Christy McCreath, Cody Herriges, Craig Dunn, Dan Bode, Dan Ryan, Daniel Pittman, Daniel Sauble, David Lutterkort, David Gwilliam, Dawn Foster, Deepak Giridharagopal, Derrick Dymock, Dominic Maraglia, Dustin Thomas, Emma Furth, Eric Shamow, eric sorenson, Evan Pierce, Fei Guo, Fred Lifton, Fritz Siebert, Garrett Honeycutt, Gary Larizza, Hailee Kenney, Harry Pan, Heidi Page, Henrik Lindberg, Hunter Haugen, J.D. Welch, Jack Norman, James Sweeny, James Turnbull, Jamie Rosa, Jason Mero, Jay Wallace, Jeff Weiss, Jenny Mahmoudi, Jeremy Adams, Jill Burrows, Joe Beede, Joe Henderson, Joe Lokis, Joe Rodriguez, Joe Wagner, John Duarte, John Grim, John Hong, john leon, John Morgan, Jose Palafox, Josh Britton, Josh Cooper, Josh Partlow, Julie Crawford, Justin Stoller, Kara Sowles, Kate Kockler, Kelsey Hightower, Ken Barber, Ken Johnson, Kent Bye, Kyle Smith, Kylo Ginsberg, Kyoko Zuch, Laura Stepp, Lauren Rother, Lee Lowder, Lindsey Smith, Luke Dutton, Luke Kanies, Marni Cohen, Matt Damkroger, Matthaus Owens, Matthew Kirby, Max Martin, Melinda Campbell, Melissa Stone, Michael Stahnke, Michal Ruzicka, Michelle Carroll, Michelle Lopez, Mike Hall, Monica Culver, Moses Mendoza, Nan Liu, Nate Wolfe, Nathan Valentine, Nathaniel Smith, Nicholas Cunningham, Nick Fagerlund, Nick Lewis, Nick Walker, Nigel Kersten, Owen Rodabaugh, Paige Huggins, Patrick Carlisle, Peter Irby, Phillip Pham, Pieter Loubser, Pieter van de Bruggen, Puppet Labs Security, R.I. Pienaar, Ralph Luchs, Randall Hansen, Reid Vandewiele, Richard Clamp, Rick Bradley, Rob Reynolds, Ruth Linehan, Ryan Coleman, Scott Campbell, Scott Johnston, Scott Schneider, Scott Vandehey, Shawon Crosen, Shreekar Bhattarai, Stephen Johnson, Steve Axthelm, Susannah Schnabel, Susanne Dickey, Tanya Webb, Thomas Linkin, Tiffany Longworth, Tim Thorpe, Walter Zabaglio, Zach Leslie, Zachary Stern, Zack Smith
Reporter: Adam Crews, Adrien Thebo, Alanna Brown, Alex Dreyer, Alex Harvey, alice nodelman, Aliza Earnshaw, Amy Hoang, Ananda Osterhaus, Andrew Parker, Andrew Roetker, Ann Franck, Arnan Outhaythip, Ben Ford, Ben Reich, Brad Hamilton, Branan Purvine-Riley, Brent Caldwell, Brent Nikolaus, Brian Cain, Brian Lake, Brian Stein, Briana de Jong, Candy Parker, Carl Caum, Carrie Odell, Carthik Sharma, Celia Cottle, Chad Jawors, Char Moe, Charles Dunbar, Charlie Sharpsteen, Charlotte Caswell, Chris Barker, Chris Grim, Chris Hoge, Chris Price, Chris Roddy, Chris Spence, Christy McCreath, Cody Herriges, Craig Dunn, Dan Bode, Dan Ryan, Daniel Pittman, Daniel Sauble, David Lutterkort, David Gwilliam, Dawn Foster, Deepak Giridharagopal, Derrick Dymock, Dominic Maraglia, Dustin Thomas, Emma Furth, Eric Shamow, eric sorenson, Evan Pierce, Fei Guo, Fred Lifton, Fritz Siebert, Garrett Honeycutt, Gary Larizza, Hailee Kenney, Harry Pan, Heidi Page, Henrik Lindberg, Hunter Haugen, J.D. Welch, Jack Norman, James Sweeny, James Turnbull, Jamie Rosa, Jason Mero, Jay Wallace, Jeff Weiss, Jenny Mahmoudi, Jeremy Adams, Jill Burrows, Joe Beede, Joe Henderson, Joe Lokis, Joe Rodriguez, Joe Wagner, John Duarte, John Grim, John Hong, john leon, John Morgan, Jose Palafox, Josh Britton, Josh Cooper, Josh Partlow, Julie Crawford, Justen Walker, Justin Stoller, Kara Sowles, Kate Kockler, Kelsey Hightower, Ken Barber, Ken Johnson, Kent Bye, konrad rzentarzewski, Kyle Smith, Kylo Ginsberg, Kyoko Zuch, Laura Stepp, Lauren Rother, Lee Lowder, Lindsey Smith, Luke Dutton, Luke Kanies, Marni Cohen, Matt Damkroger, Matthaus Owens, Matthew Kirby, Max Martin, Melinda Campbell, Melissa Stone, Michael Stahnke, Michal Ruzicka, Michelle Carroll, Michelle Lopez, Mike Hall, Monica Culver, Moses Mendoza, Nan Liu, Nate Wolfe, Nathan Valentine, Nathaniel Smith, Nicholas Cunningham, Nick Fagerlund, Nick Lewis, Nick Walker, Nigel Kersten, Owen Rodabaugh, Paige Huggins, Patrick Carlisle, Peter Irby, Peter Meier, Phillip Pham, Pieter Loubser, Pieter van de Bruggen, Puppet Labs Security, R.I. Pienaar, Ralph Luchs, Randall Hansen, Reid Vandewiele, Richard Clamp, Rick Bradley, Rob Reynolds, Ruth Linehan, Ryan Coleman, Scott Campbell, Scott Johnston, Scott Schneider, Scott Vandehey, Sean Millichamp, Shawon Crosen, Shreekar Bhattarai, Stephen Johnson, Steve Axthelm, Susannah Schnabel, Susanne Dickey, Tanya Webb, Thomas Linkin, Tiffany Longworth, Tim Thorpe, Walter Zabaglio, Zach Leslie, Zachary Stern, Zack Smith
Support Staff: Adam Crews, Adrien Thebo, Alanna Brown, Alex Dreyer, alice nodelman, Aliza Earnshaw, Amy Hoang, Ananda Osterhaus, Andrew Parker, Andrew Roetker, Ann Franck, Arnan Outhaythip, Ben Ford, Ben Reich, Brad Hamilton, Branan Purvine-Riley, Brent Caldwell, Brent Nikolaus, Brian Cain, Brian Lake, Brian Stein, Briana de Jong, Candy Parker, Carl Caum, Carrie Odell, Carthik Sharma, Celia Cottle, Chad Jawors, Char Moe, Charles Dunbar, Charlie Sharpsteen, Charlotte Caswell, Chris Barker, Chris Grim, Chris Hoge, Chris Price, Chris Roddy, Chris Spence, Christy McCreath, Cody Herriges, Craig Dunn, Dan Bode, Dan Ryan, Daniel Pittman, Daniel Sauble, David Lutterkort, David Gwilliam, Dawn Foster, Deepak Giridharagopal, Derrick Dymock, Dominic Maraglia, Dustin Thomas, Emma Furth, Eric Shamow, eric sorenson, Evan Pierce, Fei Guo, Fred Lifton, Fritz Siebert, Garrett Honeycutt, Gary Larizza, Hailee Kenney, Harry Pan, Heidi Page, Henrik Lindberg, Hunter Haugen, J.D. Welch, Jack Norman, James Sweeny, James Turnbull, Jamie Rosa, Jason Mero, Jay Wallace, Jeff Weiss, Jenny Mahmoudi, Jeremy Adams, Jill Burrows, Joe Beede, Joe Henderson, Joe Lokis, Joe Rodriguez, Joe Wagner, John Duarte, John Grim, John Hong, john leon, John Morgan, Jose Palafox, Josh Britton, Josh Cooper, Josh Partlow, Julie Crawford, Justin Stoller, Kara Sowles, Kate Kockler, Kelsey Hightower, Ken Barber, Ken Johnson, Kent Bye, Kyle Smith, Kylo Ginsberg, Kyoko Zuch, Laura Stepp, Lauren Rother, Lee Lowder, Lindsey Smith, Luke Dutton, Luke Kanies, Marni Cohen, Matt Damkroger, Matthaus Owens, Matthew Kirby, Max Martin, Melinda Campbell, Melissa Stone, Michael Stahnke, Michal Ruzicka, Michelle Carroll, Michelle Lopez, Mike Hall, Monica Culver, Moses Mendoza, Nan Liu, Nate Wolfe, Nathan Valentine, Nathaniel Smith, Nicholas Cunningham, Nick Fagerlund, Nick Lewis, Nick Walker, Nigel Kersten, Owen Rodabaugh, Paige Huggins, Patrick Carlisle, Peter Irby, Phillip Pham, Pieter Loubser, Pieter van de Bruggen, Puppet Labs Security, R.I. Pienaar, Ralph Luchs, Randall Hansen, Reid Vandewiele, Richard Clamp, Rick Bradley, Rob Reynolds, Ruth Linehan, Ryan Coleman, Scott Campbell, Scott Johnston, Scott Schneider, Scott Vandehey, Shawon Crosen, Shreekar Bhattarai, Stephen Johnson, Steve Axthelm, Susannah Schnabel, Susanne Dickey, Tanya Webb, Thomas Linkin, Tiffany Longworth, Tim Thorpe, Walter Zabaglio, Zach Leslie, Zachary Stern, Zack Smith
Viewer: Alex Harvey, Justen Walker, konrad rzentarzewski, Peter Meier, Sean Millichamp