The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com
Add "slow but thorough" mode for CI / Automated testing to the Puppet test suite.
|Affected Puppet version:||Branch:|
Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com
This ticket may be automatically exported to the PUP project on JIRA using the button below:
There are a handful of costly things we could, but don’t, do in our test suite because they slow the whole thing down too much to be justified.
These include, but are probably not limited to,
- scan logs after every test for errors, or whatever, and do something sane
- reset all terminus classes globally
- create and use a real, new CA rather than hacking a fake one
Essentially, when the test suite detects it is running under CI (eg: ENV[‘AUTOMATIC_TESTING’] or something) it should turn on slow things.
Then we can have CI run slowly but thoroughly, and have less thorough but still reasonable tests for the user.
(This would also allow us to defer things like expensive performance and combinatorial testing to CI only, which makes them easier to include in the rest of the system.)