Failing event_handlers and ocsp/ochp_command silently fail & not logged

[Andreas Ericsson]

Please don't top-post. It's terribly annoying not knowing what part
of the original email any part of your reply is intended for.

Brian A. Seklecki wrote:
>>From back in January -- there has been other discussion on similar
> issues, but no discussion / traction on this bug (and of course, no
> medium to report it, track it, document it, submit bugs/patches, etc.)
> 
> The problem is that the exact same code forking/exec'ing code is used
> for:
> 
>  - Service/Host Checks
>  - Event Handlers
>  - Notify Commands
>  - OCHP/OCSP Handler
>  - Performance Data Handlers
> 
> Result codes are explicitly registered with the API.
> 

That's common sense though. The drawback is that it doesn't report the
return code unscathed back to the caller, but instead crunches it to
see what happened, and it does so through the eyes of a healthcheck
executor.

> 126 and 127 are also checked for explicitly and warned/logged (but only
> in recent versions).  Of course 0,1,2,3 are evaluated as Service/Host
> check API values.
> 
>   568   /* check for possibly missing scripts/binaries/etc */
>   569   if(result==126 || result==127){
> 
> The problem is that 0,1,2,3 and != 126/127 can have different
> connotations and for non host-service check forks, but the
> method/function called, my_system(), doesn't have a way of
> distinguishing the calling method to change logging behavior, which it
> should.
> 
> The problem is further complicated by rampant use of pipes and other
> exotic Bourne-style expressions being used in command_line variables
> withing Nagios (one book in particular set this in motion), which,
> depending oh how compliant a Bourne-shell is, can behave differently on
> various systems.  Examples below, and, by no means, are they mean to be
> definitive, as how bash(1) forks may behave entirely different than
> exec().
> 

No, pipes do not behave differently depending on the shell. Pipes have
behaved in exactly the same way since day one and, barring bugs, will
continue to do so until the end of unix as we know it.

> Embedded perl could also further complicate things (but of course)
> 
> Solution 1:
>   - Teach my_system() to behave different for non-healthcheck forks
> 

Rather; Create a simple API that runs a process, storing everything
interesting in a publicly declared data structure and calls a
handler when the command is done executing.
Preferrably the API should have a shortcut to let in-core code feed
continuous input to it.


> Solution 2:
>   - Call a shell script wrapper for OCSP/OCSP/Pref/Event/Notify  
> 

That's just bloody stupid as the current Nagios solution already
scales very poorly with OCSP handlers.

> Solution 3 (added begrudgingly):
>   - Tell send_nsca and other builtins to use error codes > 3,  4->125
>     and 129->231 are available, but this doesn't fix the problems 
>     with pipes outlined below.
> 

There's no problem with pipes. You may not agree with the standard, but
that's not a problem with pipes. It's a problem with you.

> 
> ~BAS
> 
> 
> $ echo test | /doesntexist
> -bash: /doesntexist: No such file or directory
> $ echo $?
> 127
> 

Naturally. execve() returned ENOENT, so bash translates this to 127,
which is -2 in disguise. (ie, 0-ENOENT. The actual code is ~(0-ENOENT)).

Since any pipe expression must always yield the exit code of the last
entry in the pipe, the exit code you see is that of trying to run the
non-existant "/doesntexist" file, but...

> 
> $ /doesntexist | echo foo    
> foo
> -bash: /doesntexist: No such file or directory
> $ echo $?
> 0
> 

... here you have 'echo foo' at the end of the pipe, which is perfectly
alright and succeeds splendidly, so the entire pipe succeeds. This is by
no means bash-specific; It's how pipes are supposed to work.

> 
> $ echo > test.sh
> $ chmod -x test.sh
> $ /home/seklecki/test.sh
> -bash: /home/seklecki/test.sh: Permission denied
> $ echo $?
> 126
> 
> 
> $ echo test | ./test.sh
> -bash: ./test.sh: Permission denied
> $ echo $?
> 126
> 
> $ ./test.sh | echo foo
> -bash: ./test.sh: Permission denied
> foo
> $ echo $?
> 0
> 
> $ echo fuck shit ass | /usr/local/sbin/send_nsca -H cock.gobbling.asshat
> Could not open config file 'send_nsca.cfg' for reading.
> Error: Config file 'send_nsca.cfg' contained errors...
> $ echo $?
> 2
> 

What's the problem? Would you have wanted the pipe expression to return
the success code of the echo command?

You have a very serious problem in your understanding of pipes and why
it's absolutely vital that they work the way they do. The fact that
Nagios lets you specify fully-fledged shell-commands for pretty much
everything is actually a great strength, adding a huge chunk of
flexibility.


The problem (really) is this; Nagios doesn't log enough/anything when
some things fail to work properly.

The problem (really) is *NOT* this; Pipes work as they're designed to
do.

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null