Check behavior during the notification event

[Mike Lindsey]

On 1/5/11 11:27 PM, Yu Watanabe wrote:
> Thank you for the reply.
>
> I understood that notification events will hang up the normal service check events.
>
> I was bit curious about your comment.
>
>> A lot of people end up writing external notification handlers to take the load off of Nagios so the scheduled checks can continue whilst the external app 
> queues and processes the notifications.
>
>   If you could share your knowledge it would be helpful. 
>   Does people create external application that scans the nagios.log without using 
>   any of Event Handlers or Notification Events of Nagios? Or perhaps event broker?
>
What I ended up doing was having notification commands that drop a spool
file that contains the Nagios environment macros, into a directory. A
second daemon reads in the spool files for the notification event,
collects all the meta data (product dependencies, runbook links, ticket
links, etc), caches it for the subsequent contacts.

The spool file write is very quick, letting Nagios get back to dealing
with check submission handling, and the secondary daemon takes a serial,
blocking process, and does all the heavy lifting for notification
generation and email in a fairly parallel process. A notification for us
will include upwards of 20 contacts (some email lists, some individuals,
some ticketing and tracking systems, and some pagers).. At the height of
the "bad times" Nagios would block for 40 or so seconds, sending out
every single notification serially. Just dumping out a spool file for
each contact happens in a small fraction of a second. Pager contacts are
still straight Nagios to postfix, because simple is better for anything
where you're actually waking someone up at 3am.

-- 
Mike Lindsey


------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and, 
should the need arise, upgrade to a full multi-node Oracle RAC database 
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null