Repeating failed notifications?

[Morrey, Steven]

I'm not sure about that but we did something similar here by creating a LUA script and using custom macros.
The script reads an email template file, does a search & replace for the relevant values, then sends it.
If the link was down it caches the contents to /tmp then each time it's called it sends the emails contained in that queue, it'll keep trying until it sends, after which each email is removed from the queue if and only if the send was successful.

You could do something very similar with post requests, just cache them all the time & only delete them when they've actually completed.

I'm looking at optimizing the process by just having the cache step during the notification process and then sending all the messages out via a daemon process that checks once per minute & cleans up the queue, this would free up resources for the rest of the system.
Not sure about perl but in lua it's just under 50 lines of code to do all this.


-----Original Message-----
From: Andrew Miklas [mailto:andrew at pagerduty.com]
Sent: Wednesday, June 02, 2010 12:52 PM
To: nagios-users at lists.sourceforge.net
Subject: [Nagios-users] Repeating failed notifications?

Hi,

Is there any way to get Nagios to repeat failed notifications?  In
other words, can Nagios be configured so that if a notification script
exits with a non-zero return value, the script will be run again after
a short interval?

For a bit of background, I'm working on a way to pass events from
Nagios to the PagerDuty alerting system (www.pagerduty.com).
PagerDuty collects events from external monitoring tools like Nagios
and sends out phone calls & SMSes based on user-provided schedules and
escalation chains.

Right now, people who want to use PagerDuty to deliver their Nagios
alerts must redirect their Nagios email to a PagerDuty-supplied email
address.  While this works reasonably well, we'd like to offer a
plugin so we can better integrate with Nagios.

I'm planning on doing this by writing a little Perl script that
invokes the PagerDuty HTTP API.  The script would be run as a Nagios
notification command.  One obvious downside to this approach is that
if there's a network problem, notifications will be lost, hence the
question above.


-- Andrew
(co-founder @ PagerDuty)


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null

Privileged/Confidential Information may be contained in this message or attachments hereto. Please advise immediately if you or your employer do not consent to Internet email for messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of this company shall be understood as neither given nor endorsed by it.

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null