The Puppet Labs Issue Tracker has Moved: https://tickets.puppetlabs.com

Bug #10544

CIFS appears to mount twice

Added by Nathan Rich over 2 years ago. Updated 11 months ago.

Status:Re-openedStart date:11/03/2011
Priority:LowDue date:
Assignee:-% Done:

0%

Category:mount
Target version:-
Affected Puppet version:2.7.5 Branch:
Keywords:mount,cifs customer

We've Moved!

Ticket tracking is now hosted in JIRA: https://tickets.puppetlabs.com

This ticket may be automatically exported to the PUP project on JIRA using the button below:


Description

package{'samba-client': }
file{'smb credentials':
path => '/etc/auto.smb.credentials',
source => '/puppet/files/etc/auto.smb.credentials',
}
mount{'home directory mount':
name => '/home/mydomainint',
atboot => 'true',
device => '//storage01.mydomainint.com/home',
ensure => 'mounted',
fstype => 'cifs',
options => 'user=samba,domain=mydomainint,credentials=/etc/auto.smb.credentials',
require => [File['smb credentials'],Package['samba-client'],File['home mount point']],
}
client:
# mount
/dev/mapper/vg_ws01-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg_ws01-lv_home on /home type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
192.168.2.111:/nfs/puppet on /puppet type nfs (rw,addr=192.168.2.111)
# puppet apply /puppet/manifests/site.pp
notice: /Stage[main]/Basics/Mount[home directory mount]/ensure: ensure changed 'unmounted' to 'mounted'
notice: /Stage[main]/Basics/Mount[home directory mount]: Triggered 'refresh' from 1 events
notice: /Stage[post]/Last_stage/Exec[update puppet last run]/returns: executed successfully
notice: Finished catalog run in 3.24 seconds
# mount
/dev/mapper/vg_ws01-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg_ws01-lv_home on /home type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
192.168.2.111:/nfs/puppet on /puppet type nfs (rw,addr=192.168.2.111)
//storage01.mydomainint.com/home/ on /home/mydomainint type cifs (rw,mand)
//storage01.mydomainint.com/home/ on /home/mydomainint type cifs (rw,mand)
# puppet --version
2.7.5

In reality the second mount seems to not be real, in that umount run twice will work once and say not mounted the second time. No idea why this happens, but it doesn’t happen on a manual mount.

puppet_apply.log (7.23 KB) Louis Bouchard, 03/04/2013 07:54 am

History

#1 Updated by Daniel Pittman over 2 years ago

  • Status changed from Unreviewed to Needs More Information

Hey. I hate to be a pain, but can you still reproduce this? If so, could you shove the content of both /etc/mtab and /proc/mounts into the ticket, to help identify where this is coming from? (It would be good to know if /etc/mtab is actually a symlink, too.)

It would also be awesome if you could run Puppet in --test --debug mode on the machine that demonstrates this, without the mount there, then dig out the bits around the SMB mount. Knowing what Puppet thought down at that log level would be helpful.

Finally, can you give us some more details of the system: distribution, and package versions for the kernel, mount, and the smb mount support bits?

Thanks.

#2 Updated by Nathan Rich over 2 years ago

Sorry this is totally changed now.. went to NFSv4. This was a CentOS 6.0 box fwiw. I was able to reproduce it on several machines at the time. I’d say if you can’t get it to reproduce under those conditions then it’s probably going to have to wait until someone else barks about it. Thanks

#3 Updated by James Turnbull over 2 years ago

  • Assignee set to Daniel Pittman

Daniel – it’s reproduced in the ticket in the support URL.

#4 Updated by James Turnbull over 2 years ago

Adding related RedHat ticket:

https://bugzilla.redhat.com/show_bug.cgi?id=770004

#5 Updated by Nathan Rich over 2 years ago

Just confirmed again it is reproducible. I will be able to provide more details if they are needed.

#6 Updated by Dominic Cleal over 2 years ago

  • Status changed from Needs More Information to Closed

This looks to be an OS problem, not Puppet’s fault. The RHEL bug is progressing, so if you’re still hitting it I’d suggest opening a support case with Red Hat to apply pressure there.

#7 Updated by John Smith almost 2 years ago

I can reproduce this on Debian, on a machine that mounts two CIFS shares. Manifest:

    mount { "/mnt/share1":
        atboot  => true,
        fstype  => "cifs",
        device  => "//fileserver.example.com/foo",
        options => "auto,_netdev,cred=/path/to/cred.txt,domain=EXAMPLE",
        ensure  => "mounted",
    }

    mount { "/mnt/share2":
        atboot  => true,
        fstype  => "cifs",
        device  => "//fileserver.example.com/bar",
        options => "auto,_netdev,cred=/path/to/cred.txt,domain=EXAMPLE",
        ensure  => "mounted",
    }

Output when running puppet when the shares are not mounted:

notice: /Stage[main]//Node[host.example.com]/Mount[/mnt/share1]/ensure: ensure changed 'unmounted' to 'mounted'
info: /Stage[main]//Node[host.example.com]/Mount[/mnt/share1]: Scheduling refresh of Mount[/mnt/share1]
info: Mount[/mnt/share1](provider=parsed): Remounting
notice: /Stage[main]//Node[host.example.com]/Mount[/mnt/share1]: Triggered 'refresh' from 1 events
info: /Stage[main]//Node[host.example.com]/Mount[/mnt/share1]: Scheduling refresh of Mount[/mnt/share1]
notice: /Stage[main]//Node[host.example.com]/Mount[/mnt/share2]/ensure: ensure changed 'unmounted' to 'mounted'
info: /Stage[main]//Node[host.example.com]/Mount[/mnt/share2]: Scheduling refresh of Mount[/mnt/share2]
info: Mount[/mnt/share1](provider=parsed): Remounting
notice: /Stage[main]//Node[host.example.com]/Mount[/mnt/share2]: Triggered 'refresh' from 1 events
info: /Stage[main]//Node[host.example.com]/Mount[/mnt/share2]: Scheduling refresh of Mount[/mnt/share2]

Contents of /etc/mtab:

//fileserver.example.com/foo /mnt/share1 cifs rw,mand 0 0
//fileserver.example.com/foo /mnt/share1 cifs rw,mand 0 0
//fileserver.example.com/bar /mnt/share2 cifs rw,mand 0 0
//fileserver.example.com/bar /mnt/share2 cifs rw,mand 0 0

Output of “mount”:

//fileserver.example.com/foo on /mnt/share1 type cifs (rw,mand)
//fileserver.example.com/foo on /mnt/share1 type cifs (rw,mand)
//fileserver.example.com/bar on /mnt/share2 type cifs (rw,mand)
//fileserver.example.com/bar on /mnt/share2 type cifs (rw,mand)

Curiously, it’s listed only once in /proc/mounts:

//fileserver.example.com/foo /mnt/share1 cifs rw,mand,relatime,unc=\\fileserver.example.com\foo,username=test,domain=EXAMPLE,uid=0,noforceuid,gid=0,noforcegid,addr=10.100.1.3,file_mode=0755,dir_mode=0755,prepath=\foo,serverino,rsize=16384,wsize=57344 0 0
//fileserver.example.com/bar /mnt/share2 cifs rw,mand,relatime,unc=\\fileserver.example.com\bar,username=test,domain=EXAMPLE,uid=0,noforceuid,gid=0,noforcegid,addr=10.100.1.3,file_mode=0755,dir_mode=0755,prepath=\bar,serverino,rsize=16384,wsize=57344 0 0

Puppet agent version 2.6.2-5+squeeze6, running on debian squeeze (kernel 2.6.32-5-686) in a VMWare VM. Let me know if you need more info.

#8 Updated by John Smith almost 2 years ago

  • Status changed from Closed to Re-opened

#9 Updated by Louis Bouchard about 1 year ago

Hi,

This issue is also reproducible on Ubuntu/Precise (https://bugs.launchpad.net/ubuntu/+source/puppet/+bug/1144612) and other versions. Reading the bug comments, I saw that you were asking for debug output. Since this is easily reproducible, here is the output of a “puppet apply —debug” (it refused to accept —test) :

notice: /Stage[main]//Package[samba-client]/ensure: ensure changed 'purged' to 'present'
debug: /Stage[main]//Package[samba-client]: The container Class[Main] will propagate my refresh event
debug: Prefetching parsed resources for mount
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount'
notice: /Stage[main]//Mount[home directory mount]/ensure: ensure changed 'unmounted' to 'mounted'
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount -o 0 /mnt/target'
debug: /Stage[main]//Mount[home directory mount]: The container Class[Main] will propagate my refresh event
info: /Stage[main]//Mount[home directory mount]: Scheduling refresh of Mount[home directory mount]
info: Mount[home directory mount](provider=parsed): Remounting
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount -o remount /mnt/target'
notice: /Stage[main]//Mount[home directory mount]: Triggered 'refresh' from 1 events
debug: /Stage[main]//Mount[home directory mount]: The container Class[Main] will propagate my refresh event
info: /Stage[main]//Mount[home directory mount]: Scheduling refresh of Mount[home directory mount]
debug: /Schedule[never]: Skipping device resources because running on a host
debug: /Schedule[weekly]: Skipping device resources because running on a host
debug: /Schedule[puppet]: Skipping device resources because running on a host
debug: Class[Main]: The container Stage[main] will propagate my refresh event
debug: Finishing transaction 69893283508100
debug: Storing state
debug: Stored state in 0.00 seconds
notice: Finished catalog run in 1.23 seconds
debug: Executing '/etc/puppet/etckeeper-commit-post'
debug: Finishing transaction 69893284093880
debug: Received report to process from precises-samba
debug: Processing report from precises-samba with processor Puppet::Reports::Store
I'm attaching the full log.
I'm more than happy to help in testing any patch that you can send my way.

Here is the content of the /etc/mtab after the mounts :

/dev/mapper/PreciseS-root / ext4 rw,errors=remount-ro 0 0
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
none /sys/fs/fuse/connections fusectl rw 0 0
none /sys/kernel/debug debugfs rw 0 0
none /sys/kernel/security securityfs rw 0 0
udev /dev devtmpfs rw,mode=0755 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
none /run/shm tmpfs rw,nosuid,nodev 0 0
/dev/vda1 /boot ext2 rw 0 0
//localhost/test /mnt/target cifs rw 0 0
//localhost/test /mnt/target cifs rw 0 0
root@PreciseS-samba:~# file /etc/mtab
/etc/mtab: ASCII text

Kind regards, …Louis

#10 Updated by Charlie Sharpsteen about 1 year ago

  • Keywords changed from mount,cifs to mount,cifs customer

#12 Updated by Steve Garf about 1 year ago

  • Support Urls deleted (https://support.puppetlabs.com/tickets/585)

Linux host 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

mount.cifs version: 5.1

Ubuntu 12.04.2 LTS precise pangolin

Puppet version: 2.7.11

My two mounts in custom module:

mount {'/home/me/d':
  device  => '//romulus/d$',
  fstype  => 'cifs',
  ensure  => 'mounted',
  options => 'rw,_netdev,user=romulus/remus,uid=1000,gid=1000',
  atboot  => 'true',
}

mount {'/home/me/e':
  device  => '//romulus/e$',
  fstype  => 'cifs',
  ensure  => 'mounted',
  options => 'rw,_netdev,user=romulus/remus,uid=1000,gid=1000',
  atboot  => 'true',
}

mount before running puppet:

$ mount
/dev/mapper/host-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)

puppet output:

# puppet agent -t --verbose --no-daemon --debug
debug: Prefetching parsed resources for mount
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount'
notice: /Stage[main]//Mount[/home/me/d]/ensure: ensure changed 'unmounted' to 'mounted'
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount -o rw,_netdev,user=romulus/remus,uid=1000,gid=1000 /home/me/d'
debug: /Stage[main]//Mount[/home/me/d]: The container Class[] will propagate my refresh event
info: /Stage[main]//Mount[/home/me/d]: Scheduling refresh of Mount[/home/me/d]
info: Mount[/home/me/d](provider=parsed): Remounting
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount -o remount /home/me/d'
notice: /Stage[main]//Mount[/home/me/d]: Triggered 'refresh' from 1 events
debug: /Stage[main]//Mount[/home/me/d]: The container Class[] will propagate my refresh event
info: /Stage[main]//Mount[/home/me/d]: Scheduling refresh of Mount[/home/me/d]
debug: file_metadata supports formats: b64_zlib_yaml pson raw yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml pson raw yaml; using pson
notice: /Stage[main]//Mount[/home/me/e]/ensure: ensure changed 'unmounted' to 'mounted'
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount -o rw,_netdev,user=romulus/remus,uid=1000,gid=1000 /home/me/e'
debug: /Stage[main]//Mount[/home/me/e]: The container Class[] will propagate my refresh event
info: /Stage[main]//Mount[/home/me/e]: Scheduling refresh of Mount[/home/me/e]
info: Mount[/home/me/e](provider=parsed): Remounting
debug: Puppet::Type::Mount::ProviderParsed: Executing '/bin/mount -o remount /home/me/e'
notice: /Stage[main]//Mount[/home/me/e]: Triggered 'refresh' from 1 events
debug: /Stage[main]//Mount[/home/me/e]: The container Class[] will propagate my refresh event
info: /Stage[main]//Mount[/home/me/e]: Scheduling refresh of Mount[/home/me/e]
debug: Class[]: The container Stage[main] will propagate my refresh event

mount after running puppet:

$ mount
/dev/mapper/host-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)
//romulus/d$ on /home/me/d type cifs (rw)
//romulus/d$ on /home/me/d type cifs (rw)
//romulus/e$ on /home/me/e type cifs (rw)
//romulus/e$ on /home/me/e type cifs (rw)

/etc/mtab after puppet run:

$ cat /etc/mtab
/dev/mapper/home-root / ext4 rw,errors=remount-ro 0 0
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
none /sys/fs/fuse/connections fusectl rw 0 0
none /sys/kernel/debug debugfs rw 0 0
none /sys/kernel/security securityfs rw 0 0
udev /dev devtmpfs rw,mode=0755 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
none /run/shm tmpfs rw,nosuid,nodev 0 0
/dev/sda1 /boot ext2 rw 0 0
//romulus/d$ /home/me/d cifs rw 0 0
//romulus/d$ /home/me/d cifs rw 0 0
//romulus/e$ /home/me/e cifs rw 0 0
//romulus/e$ /home/me/e cifs rw 0 0

/proc/mounts after puppet run:

$ cat /proc/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=1016024k,nr_inodes=254006,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,relatime,size=410228k,mode=755 0 0
/dev/mapper/host-root / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sda1 /boot ext2 rw,relatime,errors=continue 0 0
//romulus/d$ /home/me/d cifs rw,relatime,vers=1.0,sec=ntlm,cache=loose,unc=\\romulus\d$,username=remus,domain=romulus,uid=1000,forceuid,gid=1000,forcegid,addr=172.16.0.69,file_mode=0755,dir_mode=0755,nounix,serverino,rsize=61440,wsize=65536,actimeo=1 0 0
//romulus/e$ /home/me/e cifs rw,relatime,vers=1.0,sec=ntlm,cache=loose,unc=\\romulus\e$,username=remus,domain=romulus,uid=1000,forceuid,gid=1000,forcegid,addr=172.16.0.69,file_mode=0755,dir_mode=0755,nounix,serverino,rsize=61440,wsize=65536,actimeo=1 0 0

Also, /etc/mtab is not a symlink

$ ls -lah /etc/mtab
-rw-r--r-- 1 root root 788 Apr  6 14:36 /etc/mtab

#13 Updated by Louis Bouchard 12 months ago

  • Support Urls deleted (https://support.puppetlabs.com/tickets/585)

Hello, After doing an analysis of the code, most particularly lib/puppet/type/mount.rb it appears that the problem doesn’t lie with puppet but with cifs-utils.

When puppet does a ‘mount -o remount’, it triggers a bug in cifs-utils that makes a double entry in /etc/mtab. This is the commit of the fix for this bug :

http://git.samba.org/?p=cifs-utils.git;a=commit;h=f46dd7661cfb87257c95081fc2071c934bfbbb16

“During a remount of a cifs filesystem, the mtab file is not properly updated, which leads to a doubled entry of the same filesystem in the /etc/mtab file. This patch adds a new function del_mtab() which is called before the add_mtab() in case the fs is being remounted.”

A potential workaround is to use remounts => ‘false’ but a safer solution would be to get a version of cifs-utils higher than 5.3 :

https://lists.samba.org/archive/samba/2012-January/165958.html

To confirm this outside of puppet control, simply do the following :

$ mount //sambahost/test /mnt/target

$ mount -o remount /mnt/target

$ df | grep target

//sambahost/test 7033648 1278120 5402660 20% /mnt/target

//sambahost/test 7033648 1278120 5402660 20% /mnt/target

HTH,

…Louis

#14 Updated by Daniel Pittman 11 months ago

  • Assignee deleted (Daniel Pittman)

Also available in: Atom PDF