Memory leaks

[Andreas Ericsson]

Tobias Klausmann wrote:
> Hi! 
> 
> (First off: if this should also go to nagios-devel, just yell at
>  me.)
> 
> Nagios 2.6 and 2.5 have memory leaks. They are not that big that
> within hours your machine will be swapping, but they degrade
> performance in other ways.
> 
> First off, their approximate extent.
> 
> 2.5 and 2.6 without perl cache have the smallest memory leaks. A
> fairly busy Nagios server (hardware quoted below) with about 3000
> services on about 330 hosts will degrade from 330M used (that's
> *not* Nagios alone) to 368M used in about 16 hours. Or about 2.4
> MB per hour. The very same machine behaves neutral if Nagios is
> not running, so it's definitely Nagios itself.
> 
> Activating the embedded Perl interpreter and -cache will increase
> the amount of lost memory to about 5-6M per hour. In this case,
> however, sometimes the memory usage snaps back, i.e. some of the
> lost memory is collected. I've not yet found out what triggers
> the reclaim. Still, over the course of hours, more and more
> memory is lost. Still, it's roughly linear memory loss.
> 

Yes. Embedded perl is known to be leaky. It's also mentioned in various
documents around the web.


> And finally, there's the advanced permission patch. With that
> patch, memory leaking skyrockets to about 15M/hour.
> 

Yes. I pointed out where a few of those leaks where in a previous
email. I'd recommend you don't use that patch, actually. At least
not until whoever wrote it comes up with a fixed version of it.


> Unfortunately, performance degradation is not just on the memory
> used front. With increased memory usage, check latency increases.
> In the worst case, this can mean that latency increases by 120s in
> about six hours. This has the net effect that for our case, we
> have to restart Nagios every two hours. 
> 

The latency increase should only happen when the machine starts swapping.
For large networks with the access-patch thingie that could happen fairly
quickly though, I imagine.

> For the case of 2.5 and 2.6 without the permissions patch, it's
> a lot less bad, but still bad enough to require restarting Nagios
> at least every eight hours. 
> 
> Without all the fancy stuff, we get to restarting Nagios every 24
> hours, as described above.
> 

That seems a bit obsessive. Are you doing anything unusual with the system?
We have several (well over a hundred) installations where Nagios has been up
and running for several months without requiring a restart.

> 
> For vanilla Nagios, at least it's clear that in whatever way
> memory is wasted, it also slows Nagios down - a possibility would
> be a linked list that is walked and gets appended over and over.
> But I guess those with knowledge of the inner workings of Nagios
> have more clue about this than I do.
> 

Anyone wanting to look into it should probably take a look at the
event scheduling queue.

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null