Dynamically add/remove hosts on Nagios

[Mike Lindsey]

On 2/9/12 5:32 PM, Felipe Cecagno wrote:
> The problem is that I want to add and remove instances dynamically, I 
> don't want to manually modify hosts.cfg on the central each time I 
> change my infrastructure. So my idea was that when a new instance gets 
> up, it will send to Nagios something like (always using NSCA):
>
> "localhost     Server UP     0 <new instance IP>"
>
I believe XI has a feature that does some automatic adding of 
hosts/services from passive checks.

To get it working in Core you need to go about it in a different way.  
You could potentially have an event handler mechanism that works it..  
Say have a "add-host" service on your master Nagios host.  So when a new 
host is added to a cluster you trigger a passive result for that check, 
that check's event handler kicks off and adds the configuration (using 
some generous templating) for that host to your master config, 
pre-caches your object cache and restarts.  You could also have a 
"del-host" service that does the reverse if you're feeling brave.

For our environment, I took yet another route.  Our CMDB provides an API 
where I can ask for all our "production updates www" hosts.

So to get auto-updating cluster level monitoring for my environment I 
have a host entry (trimmed for brevity, and such) like:

define host {
         host_name           cluster-updates-www
         alias               cluster-updates-www
         address             cluster-updates-www
         hostgroups          All,cluster,updates,updates-www,www
         check_command       cluster_ping
         _ENVIRONMENT        prod
         _PRODUCT            updates
         _PURPOSE            www
}

It doesn't matter for this, if that hostname is in DNS, as nothing 
actually queries it.  Don't need an ip address either, because nothing 
uses it.

Check command in this instance is:
define command {
         command_name        cluster_ping
         command_line        $USER5$/cluster_check.py --product $ARG1$ 
--purpose $ARG2$ --script '$USER1$/check_icmp' --args '-H %HOST% -c 
1800.0,100% -n 2 -t 2'
}

So the cluster_check.py asks the CMDB for a list of hosts when it first 
runs, then caches that list for an hour.  It pings all the hosts in 
parallel, sums up the stats and does the right thing.  Unfortunately 
this kind of process requires some external query-able source of truth.  
Our CMDB (internally developed, not currently releasable) provides a 
JSON dump on an http port..  If you've got anything you can query or 
parse - even an rsync'd dns zone file you should be able to cobble 
something together that works similarly via bash, perl, or whatever 
works for you.  Auto-updating cloud/cluster monitoring and no 
configuration updates or restart needed.  Same script and methodology 
works for services.

Doesn't help you, however if you absolutely must have unique host and 
service entries

I've attached my script if you want to rip it apart and use it for your 
environment.  You'll have to replace any of the bits that mention 'asdb' 
with code that queries and parses your CMDB, or whatever.  It's a good 
bit of effort, but potentially worth it.. good luck.  (Or, there's XI)

-- 
Mike Lindsey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cluster_check.py.gz
Type: application/x-gzip
Size: 2760 bytes
Desc: not available
URL: <https://www.monitoring-lists.org/archive/users/attachments/20120210/5ccaf1d0/attachment.bin>
-------------- next part --------------
------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing 
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
-------------- next part --------------
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null