big performance issue with Nagios 2.5

[Darren Dunham]

> <quote who="Daniel Meyer">
> > So, whats the issue? When i came back into the office this
> > morning i found nagios having a service check latency of about
> > 350 sec (maximum and also average). I've had this last week to,
> > and its "fixed" by simply restarting the nagios daemon.

> That is interesting.  I have 2.5 running on an Sun E250 with Sol8.
> I have a constant CPU utilization of 45% in kernel.  Constant load
> average of 2 - 2.5.  (with two CPUs)
> 
> This is for 83 hosts and 693 services, with 3 minute intervals.
> 
> Service latency will jump to 300's after about 3 days.  I have a
> cron job that restarts Nagios twice a week.  I have scheduling
> not carried across restarts.
> 
> My theory is that Solaris is having issues forking all the checks.
> Does anyone know if Solaris does "copy on write" for the process
> memory when forking?  I use a lot of ePN checks.

It does COW in a lot of situations, but most of the time you follow up
with an exec, so almost no writes occur before a new process overwrites
it anyway.  Also, there's no way that a normal fork is going to be
contributing more than an insignificant amount to a 350 second delay.

At 45% system CPU, something else is occuring.  Is the machine out of
memory and swapping?  Is one of the scripts fork bombing or respawing a
single child that exits rapidly?  (I might use 'top' and watch how fast
the 'last PID' is increasing.)


-- 
Darren Dunham                                           ddunham at taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null