Notifications severely impacting Nagios performance. Ideas?

[Marc Powell]

On Aug 24, 2009, at 9:43 PM, Ryan Bowlby wrote:

> Hello Everyone,
>
> We are experiencing issues with Nagios notifications. Apparently  
> when a large amount of services become critical and trigger  
> notifications the system all but stops processing new service check  
> results (passive). The "last check times" in the web interface stop  
> updating until notifications are disabled.
>
> Tailing the nagios.log file during this issue shows mostly  
> notifications and hardly any check results being processed. As soon  
> as I temporarily disable notifications - system wide - the passive  
> check results literally fly off the screen, and all the stale  
> results in the web interface clear up quickly.

I can confirm this behavior, at least under 2.x. I haven't had a major  
outage under 3.x yet to see if it still happens. My previous  
speculation on the issue is at the link below. I don't have enough  
knowledge of the code to be certain that's what's really happening  
though. Since then, I've come to suspect that it might actually be the  
logic to determine who gets notified rather than the actual  
notification itself.

http://www.mail-archive.com/nagios-users@lists.sourceforge.net/msg10803.html

> How does everyone handle this issue?

I tried backgrounding the notification script but that didn't help. We  
recognize that in the event of a major outage, we don't want to  
receive those notifications anyway so we disable notifications program- 
wide until the outage is over. Not a fix, per-se, but a workable  
process for us. It just doesn't happen often enough to really  
troubleshoot or spend a lot of time on.

--
Marc

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null