Average Check latency and execution time growth - 3.2.3

[Jörg Linge]

Am 07.10.2011 um 20:52 schrieb Jim Avery:

> On 3 October 2011 04:36, Stuart Browne <stuart.browne at ausregistry.com.au> wrote:
>> Hi,
>> 
>> I know this topic has been covered many times, but I've tried those tweaks and I have the remaining issue.
>> 
>> After a few days, the latency on checks explodes.  It goes along quite happily with small values, then after (about) 3 days, the values rise quite sharply.  I've recently been graphing performance statistics (nagiostats, mrtg) and as you can see by the two attachments (day, week), it's rather surprising.
> 
> 
> I'm sorry I can't shed much light on it, but I've seen the same
> behaviour myself, albeit on my system the service check latency
> wouldn't start increasing until after a week or two but you're right -
> the rate of increase when it starts is quite alarming.  I've recently
> culled a lot of checks from the system which has ameliorated the issue
> for the time being, but it would be good to get it fixed properly.


Increasing latency  is mostly an indicator of memory leaks.

The nagios core is well tested but the embedded perl interpreter in combination with badly written per plugin might cause a memory leak.
Also some eventbrokers module lead to memory leaks. For example mk_livestatus should never be used with environment macros enabled.

You have to monitor more then just the latency to get a feeling whats going on.

Joerg  
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2dcopy2
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null