The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

This issue tracker is now in read-only archive mode and automatic ticket export has been disabled. Redmine users will need to create a new JIRA account to file tickets using https://tickets.puppetlabs.com. See the following page for information on filing tickets with JIRA:

Bug #20522

Improve Puppet's handling of non-ASCII character encodings

Added by Charlie Sharpsteen almost 3 years ago. Updated over 2 years ago.

Status:AcceptedStart date:
Priority:NormalDue date:
Assignee:Charlie Sharpsteen% Done:

0%

Category:utf8
Target version:-
Affected Puppet version: Branch:
Keywords:customer

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This ticket is now tracked at: https://tickets.puppetlabs.com/browse/PUP-1031


Description

This is an umbrella bug for issues related to Puppet’s handling of non-ASCII character encodings. Many of these issues crop up when running Puppet on Ruby 1.9.x and newer.

See the ‘related bugs’ field for a full list of issues.


Related issues

Related to Puppet - Bug #19731: Cron name with extended characters could lead to state ya... Closed
Related to Puppet - Bug #10291: Puppet manifests fail when UTF-8 non-breaking space (0xc2... Accepted 10/25/2011
Related to Puppet - Bug #19655: "invalid byte sequence" with service provider upstart Needs Decision
Related to Puppet - Bug #20897: puppet agents complain about master not-being UTF-8 Needs More Information
Related to Puppet - Bug #15062: puppet fails if template contains invalid utf-8 Needs Decision 06/15/2012
Related to Puppet - Bug #16061: Deprecate non-UTF-8 encodings for textual content Accepted 08/21/2012
Related to Puppet - Bug #4695: Puppet should not reset locale Investigating 09/02/2010
Related to Puppet - Bug #11303: Puppet should support UTF-8 comments in manifests Closed 12/08/2011
Related to Puppet - Bug #17010: win32-dir gem returns a string whose encoding is not `asc... Closed
Related to Puppet - Bug #21512: Non-PE OS X Node on PE3 Master throws unknown method error Closed
Related to Puppet - Bug #21655: on Windows: Puppet agent can't start: error "could not fi... Duplicate
Related to Puppet - Bug #22333: puppet corrupts file if non-ascii character found Needs More Information
Related to Puppet - Bug #22703: Comment with non-ASCII characters fails with Ruby 1.9 Closed
Related to Puppet - Bug #22129: file checksumming assumes UTF-8 on ruby-2.0 Closed
Related to Puppet - Bug #22840: Could not intern from pson: “\xE4” on US-ASCII Needs More Information
Related to Puppet - Bug #22848: Encoding mis-matches cause package prefetching to fail Investigating

History

#1 Updated by Anonymous almost 3 years ago

As a rough first pass at what it will take to provide proper data encoding support in puppet:

  • Every manifest can declare its encoding
  • Every command on the master that reads from the external world can declare and encode
  • No encoding in manifests defaults to UTF-8
  • No encoding in external reads defaults to binary
  • Tagging of data in wire formats so we know how to interpret it on the other side
  • Proper conversion to system locale of data when writing and source material is encoded (not binary)
  • String manipulation in puppet has very strict no-conversion semantics
  • Resource parameters are treated as binary unless the type declares otherwise
  • If a binary parameter value is used for a non-binary parameter, an error is raised

Some of these are about basic support, others are about making a safe, usable system (strictness for instance).

In Facter:

  • resolutions must encode their values in UTF-8 (no binary allowed?)

#2 Updated by Celia Cottle over 2 years ago

  • Keywords set to customer

#6 Updated by Charlie Sharpsteen over 2 years ago

Redmine Issue #20522 has been migrated to JIRA:

https://tickets.puppetlabs.com/browse/PUP-1031

Also available in: Atom PDF