The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Bug #9983

Feature #8268: Basic Puppet agent support on Windows

Files should be opened in binary mode

Added by Josh Cooper about 3 years ago. Updated almost 3 years ago.

Status:ClosedStart date:10/07/2011
Priority:NormalDue date:
Assignee:Josh Cooper% Done:

0%

Category:windows
Target version:2.7.8
Affected Puppet version:2.7.6 Branch:https://github.com/puppetlabs/puppet/pull/192/commits
Keywords:

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This issue is currently not available for export. If you are experiencing the issue described below, please file a new ticket in JIRA. Once a new ticket has been created, please add a link to it that points back to this Redmine ticket.


Description

Windows has a concept of text and binary file modes, where text mode automatically converts \r\n. This is not an issue when files are opened, read, etc within the context of a single Windows machine. But it is an issue if say a binary file is restored from a Unix file bucket, as the Windows agent will substitute each occurrence of ‘\n’ to ‘\r\n’, and in doing so corrupt the file.

We should always open files in binary mode, which is a no-op on Unix.


Related issues

Related to Puppet - Bug #11929: Cannot serve local binary files on Windows Closed 01/12/2012
Duplicated by Puppet - Bug #11195: puppet windows is failed to get file from master Duplicate 12/06/2011

History

#1 Updated by Josh Cooper about 3 years ago

  • Assignee set to Josh Cooper

Did some research, and as expected this is a problem. Using a windows agent, Mac puppetmaster, I can backup a binary file from the agent:

Z:\work\puppet>puppet filebucket backup --server puppetmaster mynotepad.exe
mynotepad.exe: 3835c0642b59431254a456c4c6204d5e

The checksum above is actually incorrect, because puppet uses File.open(path, ‘r’) when reading the file. The checksum should be 7200b516a1a5e86ddafa49206abc8715. On the server I see:

info: Could not find file_bucket_file for 'md5/3835c0642b59431254a456c4c6204d5e/Z:\work/puppet/mynotepad.exe'
info: FileBucket adding {md5}3835c0642b59431254a456c4c6204d5e

When I restore the file on the agent, I get a corrupted binary:

Z:\work\puppet>puppet filebucket restore --server puppetmaster mynotepad2.exe 3835c0642b59431254a456c4c6204d5e
Z:\work\puppet>dir *.exe
 Volume in drive Z is Shared Folders
 Volume Serial Number is 0000-0064

 Directory of Z:\work\puppet

03/25/2005  05:00 AM            88,064 mynotepad.exe
10/12/2011  04:57 PM            43,062 mynotepad2.exe

#2 Updated by Josh Cooper about 3 years ago

  • Status changed from Accepted to In Topic Branch Pending Review
  • Affected Puppet version set to 2.7.6
  • Branch set to https://github.com/puppetlabs/puppet/pull/192/commits

Similar issues were found with the ‘file’ face, e.g. puppet file {download|store} and when specifying a remote file source, e.g. puppet:///module/…

#3 Updated by Jacob Helwig almost 3 years ago

  • Status changed from In Topic Branch Pending Review to Merged - Pending Release
  • Target version changed from 2.7.x to 2.7.7

Merged into 2.7.x in commit:e8b9f64644eb3bbd6753e06c52258d07b63328b2 and into 2.7rc in commit:50c9394caaa1827b6e4432de9986f9004e0dc895

This moves most file operations to operate in binary mode to preserve line-endings, and avoid corrupting files as they are read/written across platforms.

#4 Updated by Matthaus Owens almost 3 years ago

  • Status changed from Merged - Pending Release to Closed
  • Target version changed from 2.7.7 to 2.7.8

Released in 2.7.8rc1

#5 Updated by Lance A almost 3 years ago

I’m witnessing corrupt files on Windows when specifying puppet:/// URLs for source and running puppet locally (puppet apply site.pp —modulepath )

Example:

# Make a local copy of the installation package
file { "c:\\temp\\packages\\product-1.0.0.msi":
  source    => "puppet:///mymodule/packages/product-1.0.0.msi",
  ensure    => present,
}

I’ve tried this for multiple files and the the results are the same. The source file is multiple megabytes in size, yet the resulting destination file is a fraction of that (40-100+KB), and it doesn’t appear at first glance to be a case of simple truncation.

Environment: Windows Server 2008 RC2 Puppet 2.7.9

#6 Updated by Josh Cooper almost 3 years ago

Filed the issue Lance reported with local file serving as #11929

Also available in: Atom PDF