Bug #3873

Unexpected dependency created by exec

Added by Ben Beuchler almost 2 years ago. Updated 25 days ago.

Status:Accepted Start date:05/25/2010
Priority:Normal Due date:
Assignee:Randall Hansen % Done:

0%

Category:RAL
Target version:-
Affected Puppet version:0.25.4 Branch:
Keywords:
Votes: 6

Description

It looks like the Exec type is automatically creating dependencies on any files mentioned in the “onlyif” and “unless” parameters. While I understand that this is likely to be the most commonly desired behavior, it can unexpectedly break things. As an example, this is the code that triggered the behavior for me:

class syslog-ng::weblogger inherits syslog-ng {
    $alp = "/var/spool/apache2/access_log_pipe"

    file { "/var/spool/apache2":
        ensure => directory,
        owner  => www-data,
        group  => www-data
    }

    File["/etc/syslog-ng/syslog-ng.conf"] {
        require => File[ "$alp" ]
    }

    # If apache started first and created $alp as a normal file, delete
    # it.
    exec { "cleanup_alp":
        command => "/bin/rm ${alp}",
        unless  => "/usr/bin/test -p ${alp}",
    }

    # If the FIFO doesn't exist, create it.
    exec { "access_log_pipe":
        command => "/usr/bin/mkfifo ${alp}",
        unless  => "/usr/bin/test -p ${alp}",
        require => Exec["cleanup_alp"],
    }

    # Ensure permissions are correct on the pipe.
    file { "$alp":
        ensure => present,
        owner  => www-data,
        group  => www-data,
        mode   => 0770,
        require => Exec["access_log_pipe"]
    }

}

I’ll attach the relevant section of the --graph output. Note the unrequested dependency between Exec[cleanup_alp] and File[/var/spool/apache2/access_log_pipe].

Filing this as a bug as suggested by Dan Bode: http://groups.google.com/group/puppet-users/browse_thread/thread/73a98af374c3f7f2

cycle.png - Snippet of dependency graph (69.2 kB) Ben Beuchler, 05/25/2010 08:31 pm


Related issues

related to Puppet - Bug #5369: Autorequire loop resolution inadequate Accepted 11/19/2010
related to Puppet - Bug #12471: File resource type allows non-canonical paths.... Accepted 02/06/2012

History

Updated by Ben Beuchler almost 2 years ago

Updated by James Turnbull almost 2 years ago

  • Category set to RAL
  • Status changed from Unreviewed to Needs Decision
  • Assignee set to Luke Kanies

I personally think is expected, if poorly documented behaviour… Luke?

Updated by Ben Beuchler almost 2 years ago

I can buy that. It sure would be handy to be able to disable this behavior, though.

Regarding documentation: I’m pretty sure it’s completely undocumented behavior, aside from (IIRC) a single line comment in the code.

Updated by Luke Kanies almost 2 years ago

  • Status changed from Needs Decision to Accepted
  • Assignee deleted (Luke Kanies)

Seems like this should be fixed to only mention the executable that starts the command?

Updated by Ben Beuchler almost 2 years ago

It seems to me that there are two approaches that make sense:

  1. If the dependency magic is actually useful then it probably makes sense to leave it as is and add an option to disable it (magicdeps => false). Personally, I’m bothered by “magic” behavior, being firmly in the “explicit is better than implicit” camp.

  2. If it’s not useful enough to keep as is, I would suggest that it be removed outright rather than restricted to just the executable.

Updated by Eric Lindvall almost 2 years ago

Are there any work-arounds for this right now?

Updated by Mike Pountney almost 2 years ago

I can confirm that this is still current at 0.25.5, and caused a dep-cycle in our manifests that did not exist at 0.24.7.

exec {“initdb”: unless => “test -d /var/lib/postgresql/data”; before => File[“/var/lib/postgresql/data”] } file {“/var/lib/postgresql/data”: ensure => directory, mode => 0750, owner => postgres" }

.. created a dep-cycle on /var/lib/postgresql/data.

Workaround was to change the unless test to point to an unmanaged resource, eg:

exec {“initdb”: unless => “test -f /var/lib/postgresql/data/PG_VERSION”; before => File[“/var/lib/postgresql/data”] }

… but this is not going to be possible in every instance.

Updated by Alan Barrett almost 2 years ago

A workaround is to use different file names in different places. For example, if your file is /etc/foo, then in the exec that causes the trouble, use “/etc/./foo” instead of “/etc/foo”. Even if /etc/foo is a managed resource, /etc/./foo will be seen by puppet as a different resource.

Updated by Andre Nathan 7 months ago

Was any consensus reached over this? I’ve just been bitten by this bug as my onlyif property runs grep searching for a path in a file, and a dependency to that path was added, resulting in a cycle…

Updated by Nigel Kersten 4 months ago

  • Status changed from Accepted to Needs More Information

Is there another use case other than the:

unless => “test -f /var/lib/postgresql/data/PG_VERSION”

one?

That one is trivially solved by using the creates parameter.

Updated by Eric Lindvall 4 months ago

I was using it to get an md5sum of a file if it exists.

I’m sure the number of ways people will want to use unless is far beyond what we could imagine up front. Having to use the /./ hack is a pretty bad workaround — it works, but it isn’t intuitive.

Updated by Chad Metcalf 4 months ago

I’ll need to dig around to remember where but we were hit by this as well. +1 on something other then the ./ hack.

Updated by Daniel Pittman 4 months ago

Eric Lindvall wrote:

I was using it to get an md5sum of a file if it exists.

Awesome. Why were you doing that? It sounds like there is a hidden use case here that we are not meeting, and I would love to understand why that is?

Updated by Andre Nathan 4 months ago

A simple onlyif => "grep foo /some/path" also triggers this.

Updated by Daniel Pittman 4 months ago

  • Status changed from Needs More Information to Needs Decision
  • Assignee set to Randall Hansen

Randall, I have some sympathy for “this breaks things”, but I am pretty much certain this is the right default behaviour – to autorequire mentioned files, even if they are in unless or onlyif. Is this something you want your team to have a view over? If not, I am going to close it as being the “right” behaviour, more or less, for most people.

Updated by Eric Lindvall 4 months ago

Could you give some real-world scenarios where the current behavior would be the correct behavior?

I don’t see how the /./ hack is something that could honestly be argued is the right way to get out of the dependency loop.

Is someone going to document it as the proper solution for referencing files without creating dependencies?

Updated by Daniel Pittman 4 months ago

Eric Lindvall wrote:

Could you give some real-world scenarios where the current behavior would be the correct behavior?

Solid question. You have convinced me to change my mind. I thought that I could, but trying to find an example where this was the correct behaviour, and that was better than the costs of the problems, changed my view.

I don’t see how the /./ hack is something that could honestly be argued is the right way to get out of the dependency loop.

No, that is actually another bug that you are sneaking through there – we should treat the canonical, cleaned-up name of a file as the thing we demand is unique. Otherwise /foo, //foo, and ///foo can all be managed as if distinct, while actually being the same file. When that gets fixed, if it does, this loophole would go away.

Updated by Randall Hansen 3 months ago

  • Status changed from Needs Decision to Accepted

I agree with Daniel; this behavior is a bug and should go away.

Updated by Stefan Schulte 3 months ago

The example in the ticket should work if the File["$alp"] would also establish an explicit require Exec["cleanup_alp"]. This is because an explicit require always overwrites an implicit autorequire. This seems to be a far better workaround than hacking with /my/./path references.

Unfortunately an autorequire is not overwritten if the user-defined requires are implicit (a->b->c does overwrite an autorequire b->a, but not c->a).

So in my opinion this bug can also be fixed with #5369

Updated by Paul Lathrop 25 days ago

We also ran into this, in a slightly different use case. We want to trigger an exec resource but only if a file doesn’t contain the string /mnt. So (sanitized):

exec { 'update_status':
  command => '/usr/local/bin/update_status',
  unless       => '/bin/grep -E "[[:blank:]]/mnt[[:blank:]]" /var/lib/status_file',
}

This causes Puppet to fail to apply the catalog with an error message saying: ‘invalid tag: “/mnt”’, because we are managing a mount resource for /mnt elsewhere in the catalog.

I would argue that this behavior brutally violates the Principle of Least Surprise, and is the bad kind of magic (as opposed to the good kind, i.e. auto-requiring parent directory resources when they are managed).

This is on puppet version 2.6.14

Also available in: Atom PDF