Not getting email alerts
Joe Rhett
jrhett at meer.net
Wed Aug 25 05:47:55 CEST 2004
First point of review -- you won't get any more service alerts after the
host is down. That makes sense. So the problem lies in the host
notification.
On Tue, Aug 24, 2004 at 07:50:01PM -0700, Quanah Gibson-Mount wrote:
> --On Tuesday, August 24, 2004 6:55 PM -0700 Joe Rhett <jrhett at meer.net>
> wrote:
>
> >>It is willing and able to be notified 24x7. This works as expected with
> >>all service notifications.
> >
> >Yes, but the host or service notification option was set to businesshours.
> >Why don't you try making them the same across the board?
>
> Done, and Nagios restarted (I even restarted the entire system, just to
> make sure there were no renegade nagios processes lurking about). I'll
> note that is how all of our production systems are configured, and they all
> have this same issue. I also brought the host back up prior to restarting
> Nagios, so everything would be in an OK state:
>
> [1093399400] HOST ALERT: ldap-dev1;UP;HARD;1;PING OK - Packet loss = 0%,
> RTA = 2.00 ms
> [1093399400] HOST NOTIFICATION:
> test-service;ldap-dev1;UP;notify-by-email;PING OK - Packet loss = 0%, RTA =
> 2.00 ms
> [1093399401] GLOBAL HOST EVENT HANDLER: ldap-dev1;UP;HARD;1;notify-smarts
> [1093399402] SERVICE ALERT: ldap-dev1;Disk;UNKNOWN;SOFT;1;(No output!)
> [1093399402] GLOBAL SERVICE EVENT HANDLER:
> ldap-dev1;Disk;UNKNOWN;SOFT;1;notify-smarts
> [1093399410] SERVICE ALERT: ldap-dev1;Disk;OK;SOFT;2;DISK OK [3875661 kB
> (94%) free on /dev/dsk/c0t0d0s0] [0 kB (100%) free on fd] [0 kB (100%) free
> on mnttab] [26776028 kB (99%) free on /dev/dsk/c0t0d0s4] [3965584 kB (99%)
> free on swap] [34623920 kB (99%) free on /dev/dsk/c0t1d0s1] [953992 kB
> (99%) free on /dev/dsk/c0t0d0s7] [1941152 kB (97%) free on
> /dev/dsk/c0t0d0s6] [9000000 kB (100%) free on AFS
> [1093399410] GLOBAL SERVICE EVENT HANDLER:
> ldap-dev1;Disk;OK;SOFT;2;notify-smarts
> [1093400129] HOST ALERT: ldap-dev1;UP;HARD;1;(Host assumed to be up)
> [1093400129] SERVICE ALERT: ldap-dev1;Disk;OK;HARD;1;DISK OK [3875661 kB
> (94%) free on /dev/dsk/c0t0d0s0] [0 kB (100%) free on fd] [0 kB (100%) free
> on mnttab] [26776012 kB (99%) free on /dev/dsk/c0t0d0s4] [3965824 kB (99%)
> free on swap] [34623920 kB (99%) free on /dev/dsk/c0t1d0s1] [953992 kB
> (99%) free on /dev/dsk/c0t0d0s7] [1941152 kB (97%) free on
> /dev/dsk/c0t0d0s6] [9000000 kB (100%) free on AFS
>
>
> The host was then shut down. I correctly get the first notification:
>
> Notification Type: PROBLEM
>
> Service:
> Service State:
> Host: ldap-dev1
> Host State: DOWN
>
> Date/Time: Tue Aug 24 19:23:36 PDT 2004
>
> Additional Info:
>
> CRITICAL - Plugin timed out after 10 seconds
>
> Logs show:
>
> [1093400604] HOST ALERT: ldap-dev1;DOWN;SOFT;1;CRITICAL - Plugin timed out
> after 10 seconds
> [1093400604] GLOBAL HOST EVENT HANDLER: ldap-dev1;DOWN;SOFT;1;notify-smarts
> [1093400616] HOST ALERT: ldap-dev1;DOWN;HARD;2;CRITICAL - Plugin timed out
> after 10 seconds
> [1093400616] HOST NOTIFICATION:
> test-service;ldap-dev1;DOWN;notify-by-email;CRITICAL - Plugin timed out
> after 10 seconds
> [1093400616] GLOBAL HOST EVENT HANDLER: ldap-dev1;DOWN;HARD;2;notify-smarts
> [1093400617] SERVICE ALERT: ldap-dev1;Disk;UNKNOWN;HARD;1;(No output!)
>
>
> Now more than 20 minutes have passed. No further alerts have been
> received. Nothing more has been entered in nagios.log.
> status.log shows that checks are happening regularly:
>
> [1093402139]
> HOST;ldap-dev1;DOWN;1093402104;1093400616;0;408976;201490;0;1093400616;1;1;1;1;1;0;0.00;0;1;1;CRITICAL
> - Plugin timed out after 10 seconds
> [1093402139]
> SERVICE;ldap-dev1;Disk;UNKNOWN;1/3;HARD;1093401865;1093402094;ACTIVE;1;1;1;1093400617;0;UNKNOWN;408653;201561;0;0;0;0;1;21;225;1;0;0.00;0;1;1;1;(No
> output!)
>
>
> [1093402154]
> HOST;ldap-dev1;DOWN;1093402104;1093400616;0;408976;201490;0;1093400616;1;1;1;1;1;0;0.00;0;1;1;CRITICAL
> - Plugin timed out after 10 seconds
> [1093402154]
> SERVICE;ldap-dev1;Disk;UNKNOWN;1/3;HARD;1093401865;1093402094;ACTIVE;1;1;1;1093400617;0;UNKNOWN;408653;201561;0;0;0;0;1;21;225;1;0;0.00;0;1;1;1;(No
> output!)
>
>
> Updated configs:
>
> define contact {
> contact_name test-service
> alias Test Service
> service_notification_period 24x7
> host_notification_period 24x7
> service_notification_options w,u,c,r
> host_notification_options d,u,r
> service_notification_commands notify-by-email
> host_notification_commands notify-by-email
> email page-quanah
> }
>
> define contactgroup {
> contactgroup_name ldap-qsys-admins
> alias LDAP Test Host Administrators
> members test-service
> }
>
> define hostgroup {
> hostgroup_name ldap-qdev-group
> alias LDAP QDev Group
> contact_groups ldap-qsys-admins
> members ldap-dev1
> }
>
> define host {
> name ldap-qother-template
> check_command check-host-alive
> max_check_attempts 2
> notification_interval 2
> notification_period 24x7
> notification_options d,u,r
> notifications_enabled 1
> register 0
> }
>
> define host {
> use ldap-qother-template
> host_name ldap-dev1
> alias ldap-dev1
> address ldap-dev1.stanford.edu
> parents smr-a-3548
> }
>
> define contactgroup {
> contactgroup_name ldap-qapp-admins
> alias LDAP Administrators
> members test-service
> }
>
> define service {
> name ldap-qother-template
> max_check_attempts 3
> normal_check_interval 2
> retry_check_interval 2
> check_period 24x7
> notification_interval 3
> notification_options w,u,c,r
> notification_period 24x7
> contact_groups ldap-qapp-admins
> register 0
> }
>
> define service {
> use ldap-qother-template
> hostgroup_name ldap-qdev-group
> service_description Disk
> max_check_attempts 3
> check_command check_disk!20%!10%
> }
>
>
> --
> Quanah Gibson-Mount
> Principal Software Developer
> ITSS/Shared Services
> Stanford University
> GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
> 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
> Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
> http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
> _______________________________________________
> Nagios-users mailing list
> Nagios-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nagios-users
> ::: Please include Nagios version, plugin version (-v) and OS when
> reporting any issue. ::: Messages without supporting info will risk being
> sent to /dev/null
--
Joe Rhett
Senior Geek
Meer.net
-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null
More information about the Users
mailing list