Configuration changes not taking effect after restart of Nagios
Deborah Martin
Deborah.Martin at kognitio.com
Fri Apr 26 14:27:50 CEST 2013
Thanks to everyone that replied.
The problem is when I grep I will find correct occurrences of when the sms group should be used. And I want to just remove it from one serviceescalation definition for a particular service check. I certainly don't want to get rid of that group altogether.
The problem seems to be that although I've removed that group from the contacts_group of the serviceescalation definition and when I reload nagios (I've also killed nagios, deleted cache files .dat files etc etc) I can see "configuration" for service escalations, both the old entry including the sms group and the newly amended entry without the sms group.
My installation is always under /usr/local/nagios and the configuration files sit under etc/production/customer_name/hosts.cfg or services.cfg and the main config files are always under
/usr/local/nagios/etc/_systemfiles so that customer specific configs are very separate to anything that is system / global wide to nagios. I compile nagios from source rather than rpms so I can control exactly where things go etc etc. There are no config files in /usr/local/nagios/etc, just production and _systemfiles directorys
This is how it's been configured since I started using nagios back in late early 2000's. The only difference is that I recently upgraded SLES 11 SP2 and Nagios 3.4.4 as we had a problem with time drifts as the environment is a VM (And yes - I know that generally VM's are a bad idea for monitoring but I have to work with what I'm given :( ) .
I've never come across objects.precache so haven't knowingly configured nagios to use that. I can't find that file anywhere on the system.
I've just killed nagios again, removed status.dat, retention.dat and objects.cache. I've switched off the option in nagios.cfg to not look at objects.cache.
I've restarted nagios but it still shows both the old and new config for the serviceescalation. I'm completely stumped. I don't know where else to look.
A new objects.cache has been created - is that right even though I've commented that out in nagios.cfg ?
# OBJECT CACHE FILE
# This option determines where object definitions are cached when
# Nagios starts/restarts. The CGIs read object definitions from
# this cache file (rather than looking at the object config files
# directly) in order to prevent inconsistencies that can occur
# when the config files are modified after Nagios starts.
#object_cache_file=/usr/local/nagios/var/objects.cache
Thanks,
Deborah
Deborah Martin
Kognitio Analytical Platform Support
______________________________________________________________________
Direct +44 1344 788670 Switch +44 1344 300770 | Deborah.martin at kognitio.com
______________________________________________________________________
[Description: Description: cid:image006.jpg at 01CD1C86.AAF068D0]<http://www.kognitio.com/>
www.kognitio.com<http://www.kognitio.com/>
[Description: Description: Description: Description: Description: Description: Facebook]<http://www.facebook.com/pages/Kognitio/259254109883> [Description: Description: Description: Description: Description: Description: Twitter] <http://www.twitter.com/kognitio> [Description: Description: Description: Description: Description: Description: Youtube] <http://www.youtube.com/kognitiowx2> [Description: Description: Description: Description: Description: Description: LinkedIn] <http://www.linkedin.com/companies/kognitio> [Description: Description: Description: Description: Description: Description: Blog] <http://kognitio.blogspot.com/>
From: Grant Maxwell [mailto:grant.maxwell at maxan.com.au]
Sent: 25 April 2013 22:44
To: Nagios Users List
Subject: Re: [Nagios-users] Configuration changes not taking effect after restart of Nagios
Hi Deborah
Essentially it comes down to this. Somewhere you have a file with the offending definition in it. You need to find that file and either delete it or move it somewhere where it won't influence the startup.
The first thing I would check is the nagios.cfg file, in particular the lines like:
cfg_file=
cfg_dir=
Then check all of the files/directories mentioned in case there is a left-over line in one of them. You could use grep for that.
If you don't find the offending file/entry then that is most strange because those config lines determine what files and folders nagios will look in for its configurations.
Moving on and assuming that did not find it then I would search the whole system (bigger hammer).
If you are using windows then you can use the windows search facility to find it. If you are using linux (or any UNIX variant) you can use the following command to search the entire system to find it. It searches the whole system because nagios does have files in a few locations. It should only read configs from the locations in the nagios.cfg file but because you can't find the offending file -then use a slightly bigger hammer.
find / -type f -name \*.cfg -exec grep -L "smsgroupname" {} \;
replace smsgroupname with the name of the sms group you got rid of.
The command will list any files that contain the sms group name. It may print some filenames that it could not access. Thats ok just ignore them.
once you find the file .... you know what to do.
let us know how you go ....
regards
Grant
On 26/04/2013, at 2:40 AM, Deborah Martin <Deborah.Martin at kognitio.com<mailto:Deborah.Martin at kognitio.com>> wrote:
Thanks, I've just tried that and it hasn't made any difference. Somehow the sms group still creeps in. I've grepped all the config files to ensure everything is as it should be. Somehow, something is causing the configuration to pick up the old config. I've stopped nagios, deleted the suggested files, restarted nagios.
I've even taken the service escalation out of the config - now that works - I can see it's missing when I look at the configuration for service escalations for that particular node.
When I add the escalation back in, it just doesn't pick it up correctly.
Here are the definitions of sorts with customer related stuff removed :-
define host{
use generic-host
host_name hostname-a
alias hostname-a ; Lowercase
address 172.18.5.41
hostgroups APP ; Uppercase
contact_groups app-group,oracle-group,it-helpdesk-group
check_command check_ping!100.0,20%!500.0,60%
check_interval 1 ;
check_period 24x7
notification_interval 5
notification_period 24x7
check_period 24x7
}
define hostescalation{
host_name hostname-a
first_notification 2
last_notification 2
notification_interval 2
escalation_options d ; Only escalate when in DOWN state
escalation_period oncall
contact_groups apps-sms-oncall-group, app-group, oracle-group,it-helpdesk-group
}
define hostescalation{
host_name hostname-a
first_notification 3
last_notification 3
notification_interval 2
escalation_options d ; Only escalate when in DOWN state
escalation_period oncall
contact_groups app-group,oracle-group,it-helpdesk-group
}
define service{
use generic-service
active_checks_enabled 1 ; Active service check enabled
host_name hostname-a
normal_check_interval 5
service_description Oracle Data check for hostname-a ; AP ports for external app only
servicegroups apps
check_command check_nrpe_oracle_data
contact_groups app-group,oracle-group
max_check_attempts 1;
notification_interval 10;
check_interval 5;
retry_check_interval 1;
notification_period MonSun
notification_options u,c,r ; CHG2 Added warnings for this check only
}
## Escalation ONE:
define serviceescalation {
host_name hostname-a;
service_description Oracle Data check for hostname-a ;
first_notification 2
last_notification 0
notification_interval 30
escalation_options u,c,r ;
escalation_period oracle-oncall ; Only escalate during oncall periods
contact_groups app-group,oracle-group
}
define servicegroup{
servicegroup_name apps
alias 3rd Party Application monitoring
}
define contactgroup{
contactgroup_name app-group
alias Application Administrators
members app-admin
}
define contact{
contact_name app-admin
alias app-admin
contactgroups app-group
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,u,r
service_notification_commands notify-by-email
host_notification_commands host-notify-by-email
email support at kognitio.com<mailto:support at kognitio.com>
}
define contactgroup{
contactgroup_name oracle-group
alias Oracle Administrators
members person-a, person-b
}
define contact{
contact_name person-a
alias perona-a
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,u,r
service_notification_commands notify-by-email
host_notification_commands host-notify-by-email
email person-a at kognitio.com<mailto:person-a at kognitio.com>
}
define contact{
contact_name person-b
alias person-b
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,u,r
service_notification_commands notify-by-email
host_notification_commands host-notify-by-email
email person-b at kognitio.com<mailto:person-b at kognitio.com>
}
define timeperiod{
timeperiod_name oracle-oncall
alias Oracle Oncall Hours
sunday 00:00-24:00
monday 08:00-09:00,17:30-20:00
tuesday 08:00-09:00,17:30-20:00
wednesday 08:00-09:00,17:30-20:00
thursday 08:00-09:00,17:30-20:00
friday 08:00-09:00,17:30-20:00
saturday 08:00-20:00
}
Maybe I've done something with the above config that is causing this but for the life of me I can't see it!
Regards,
Deborah
Deborah Martin
Kognitio Analytical Platform Support
______________________________________________________________________
Direct +44 1344 788670 Switch +44 1344 300770 | Deborah.martin at kognitio.com<mailto:Deborah.martin at kognitio.com>
______________________________________________________________________
<image001.jpg><http://www.kognitio.com/>
www.kognitio.com<http://www.kognitio.com/>
<image002.jpg><http://www.facebook.com/pages/Kognitio/259254109883> <image003.jpg><http://www.twitter.com/kognitio> <image004.jpg><http://www.youtube.com/kognitiowx2> <image005.jpg><http://www.linkedin.com/companies/kognitio> <image006.jpg><http://kognitio.blogspot.com/>
From: davor grgicevic [mailto:dgrgicevic at gmail.com<http://gmail.com>]
Sent: 24 April 2013 17:03
To: Nagios Users List
Subject: Re: [Nagios-users] Configuration changes not taking effect after restart of Nagios
Hi Deobrah
try following:
stop nagios
erase objects.cache, status.dat and retention.dat
start nagios
Best,
On Wed, Apr 24, 2013 at 1:38 PM, Deborah Martin <Deborah.Martin at kognitio.com<mailto:Deborah.Martin at kognitio.com>> wrote:
Hi,
I'm using Nagios 3.4.4 on SLES 11 SP2. This has been running for around two months.
Recently I needed to change the configuration for a service escalation to remove a contact group that would get sms alerts. I restarted Nagios and everything ran fine.
However, it's been reported that recently, the team that shouldn't be getting the sms alerts are still getting them. When I look at the Configuration for service escalations, I can now see two entries for the same escalation, one which is the new , with the sms contact group removed and one with the old config which has the sms contact group included. But when I look at the actual .cfg file for the service escalation definition, there is not mention of the sms group anymore. I've checked, and re-checked and still I can't see why this has happened.
I've even stopped Nagios for a short while and restarted it from scratch and this still doesn't make any difference.
Has anyone else see any oddities like this where config changes don't seem to take effect after a restart of Nagios ?
Is there a "cache" file I can remove from the configuration forcing nagios to recreate it from scratch so that reads the rest of the .cfg files properly ?
Any pointers / ideas on where I go from here would be really appreciated.
Regards,
Deborah
Deborah Martin
Kognitio Analytical Platform Support
______________________________________________________________________
Direct +44 1344 788670<tel:%2B44%201344%20788670> Switch +44 1344 300770<tel:%2B44%201344%20300770> | Deborah.martin at kognitio.com<mailto:Deborah.martin at kognitio.com>
______________________________________________________________________
<image001.jpg><http://www.kognitio.com/>
www.kognitio.com<http://www.kognitio.com/>
<image002.jpg><http://www.facebook.com/pages/Kognitio/259254109883> <image003.jpg><http://www.twitter.com/kognitio> <image004.jpg><http://www.youtube.com/kognitiowx2> <image005.jpg><http://www.linkedin.com/companies/kognitio> <image006.jpg><http://kognitio.blogspot.com/>
This e-mail and any files transmitted with it are strictly confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, please delete this e-mail immediately. Any unauthorised distribution or copying is strictly prohibited.
Whilst Kognitio endeavours to prevent the transmission of viruses via e-mail, we cannot guarantee that any e-mail or attachment is free from computer viruses and you are strongly advised to undertake your own anti-virus precautions. Kognitio grants no warranties regarding performance, use or quality of any e-mail or attachment and undertakes no liability for loss or damage, howsoever caused.
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net<mailto:Nagios-users at lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null
--
Davor Grgicevic
This e-mail and any files transmitted with it are strictly confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, please delete this e-mail immediately. Any unauthorised distribution or copying is strictly prohibited.
Whilst Kognitio endeavours to prevent the transmission of viruses via e-mail, we cannot guarantee that any e-mail or attachment is free from computer viruses and you are strongly advised to undertake your own anti-virus precautions. Kognitio grants no warranties regarding performance, use or quality of any e-mail or attachment and undertakes no liability for loss or damage, howsoever caused.
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net<mailto:Nagios-users at lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null
This e-mail and any files transmitted with it are strictly confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, please delete this e-mail immediately. Any unauthorised distribution or copying is strictly prohibited.
Whilst Kognitio endeavours to prevent the transmission of viruses via e-mail, we cannot guarantee that any e-mail or attachment is free from computer viruses and you are strongly advised to undertake your own anti-virus precautions. Kognitio grants no warranties regarding performance, use or quality of any e-mail or attachment and undertakes no liability for loss or damage, howsoever caused.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2286 bytes
Desc: image001.jpg
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 521 bytes
Desc: image002.jpg
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 506 bytes
Desc: image003.jpg
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 1732 bytes
Desc: image004.jpg
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 580 bytes
Desc: image005.jpg
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 644 bytes
Desc: image006.jpg
URL: <https://www.monitoring-lists.org/archive/users/attachments/20130426/1d61916c/attachment-0005.jpg>
-------------- next part --------------
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
-------------- next part --------------
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null
More information about the Users
mailing list