Trouble with embedded perl: nagios keeps forking perl plugins (SUCCESS STORY, CHECK THIS NUMBERS!)

[Nicolás Alen]

Stanley:


Let me say:  _ Y O U _ R U L E _

Don't ask me WHAT I did, it just started to work.
I followed your steps and added some extra printfs on check.c, recompiled
nagios binary using the same ./configure line as always, and suddenly some
"embedded perl plugin output was ..." lines starting to appear on stdout.

Perl still appears when doing "ps", but after some minutes I noticed some
AMAZING performance changes:

- Nagios is running on a dedicated box. It checks 65 hosts and 512 services.
More than 60% of the checks are performed by custom perl scripts.

- Check lattency avg dropped from 59/60 SECONDS to (incredible) 0.580!!!
DAMN! I would like to write that number with a big red font if you wouldn't
piss!.

- Server load avg 5 minutes dropped from 15 to 6.5.



What can I say?!. Thanks!
If there is anything I can do, let me know!

Nicolás Alen.

 

-----Mensaje original-----
De: nagios-users-admin at lists.sourceforge.net
[mailto:nagios-users-admin at lists.sourceforge.net] En nombre de Stanley
Hopcroft
Enviado el: jueves, 17 de marzo de 2005 6:11
Para: nagios-users at lists.sourceforge.net
Asunto: [SPAM] Re: [Nagios-users] Re: Trouble with embedded perl: nagios
keeps forking perl plugins

Dear Sir,

I am writing to thank you for your letter and say,

On Thu, Mar 17, 2005 at 01:25:51AM -0300, Nicol?s Alen wrote:
> 
> 
> Nagios binary:
> 
> =================================================================
> # ./nagios -m
> 
> Nagios 1.2
> Copyright (c) 1999-2004 Ethan Galstad (nagios at nagios.org) Last 
> Modified: 02-02-2004
> License: GPL
> 
> External Data I/O
> -----------------
> Object Data:      TEMPLATE
> Status Data:      DEFAULT
> Retention Data:   DEFAULT
> Comment Data:     DEFAULT
> Downtime Data:    DEFAULT
> Performance Data: DEFAULT
> 
> Options
> -------
> * Embedded Perl compiler (With caching) 
> =================================================================
>

fine.
 
> 
> 
> An example perl command on checkcommands.cfg:
> 
> =================================================================
> define command {
> command_name                   check_ping
> command_line                   $USER1$/check_ping.pl $HOSTADDRESS$ $ARG1$
> $ARG2$ $ARG3$ $ARG4$ $ARG5$
> }
> =================================================================
> 
> 
> 
> Part of the code...
> 
> =================================================================
> #!/bin/perl -w
> 
> use strict;
> use lib "/usr/local/nagios/libexec";
> use utils qw(%ERRORS $TIMEOUT);
>
 .. good stuff omitted.
 
> >> Do your plugins contain a shebang line ?
> 
> Yes. It was #!/usr/bin/perl before you asked, but still I see Nag 
> forking and executing perl to run the plugins:
> 
> # ps axuwww | grep perl | grep ping
> nagios   21211  3.0  0.1  3008 1648 ?        R    01:21   0:00 /bin/perl
-w
> /usr/local/nagios/libexec/check_ping.pl 200.32.5.224 20 50 300.0 
> 1000.0
>

I suppose check_ping.pl _is_ a Perl plugin ?
 
>

(Is Perl in /bin on your system, or is that truncation of the ps lines
?) 

How peculiar.

Here's what I see

bash-2.05a$ let "s=0" "i=0"
bash-2.05a$ while [ 1 -eq 1 ]; do ps -auxw | grep '/nagios/libexec' | grep
perl; echo; let "i++" "s = 10 * i"; sleep 10; echo "After $s secs."; done

After 10 secs.
anwsmh 56648 38.0  2.5  7172 6540  p0  S+    7:53PM   0:00.59 
/usr/bin/perl -w /usr/local/nagios/libexec/check_internet

After 20 secs.

After 30 secs.

After 40 secs.

After 50 secs.

After 60 secs.

After 70 secs.

After 80 secs.

After 90 secs.

After 100 secs.

After 110 secs.

After 120 secs.

After 130 secs.
nagios 57354  0.0  0.2   980  576  ??  S     7:55PM   0:00.00 
/usr/local/nagios/libexec/check_by_ssh -t 60 -H asterix -C
perl/check_open_connections
nagios 57352  0.0  0.1   628  264  ??  S     7:55PM   0:00.01 sh -c 
/usr/local/nagios/libexec/check_by_ssh -t 60 -H asterix -C
'perl/check_open_connections'

After 140 secs.

After 150 secs.

After 160 secs.

After 170 secs.

After 180 secs.

After 190 secs.

After 200 secs.

After 210 secs.

After 220 secs.

After 230 secs.

After 240 secs.

After 250 secs.


The only references to Perl are either

1 one of my Perl plugins run manually to test the loop

2 check_by_ssh calling Perl code remotely.

I use a fair amount of Perl plugins (20 or more custom) with the mainly
default check_interval so I think the loop would have detected any more fork
+ exec of Perl.

What happens when you run the above bash ?


> >> The only way of nailing this, or any plugin/Nag problem, is by 
> >> building a Nag with debug options and running it in a sandpit.
> 
> Didn't find find anything useful apart from a wrong path to my 
> host-perfdata log :/
>

Did you see messages like (Nag configured with --enable-DEBUG1 and run with
a stripped config [only perl plugins running frequently])

#ifdef DEBUG1
                                        printf("embedded perl ran %s with
error %s\n",fname,SvPV(ERRSV,PL_na)); #endif
                                        }
--
                                unlink(tmpfname);    
#ifdef DEBUG1
                                printf("embedded perl plugin output was
%d,%s\n",pclose_result, plugin_output); #endif

(from checks.c) ?


 
> 
> Do you think I should try nagios 2 for a change?

No. The Perl support is basically the same (and has been stable for a long
time [with bugs unfortch]).

> This is somehow silly, but do I have to include some special code on 
> my perl plugs?

No. The ePN does that for you.

I am unlikely to be of any further help without some ideas.

Unless there is something unusual (as is obvious but what ?) I can't imagine
why you are seeing your results.

(If you are patient, I could send you some code privately to emulate the ePN
function outside of Nag but I am not sure it would help).

Yours sincerely.

--
Stanley Hopcroft

IP Australia
Ph: (02) 6283 3189  Fax: (02) 6281 1353
PO Box 200 Woden  ACT 2606
http://www.ipaustralia.gov.au



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id396&op=click
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null