<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18702"></HEAD>
<BODY bgColor=#ffffff text=#000099>
<DIV dir=ltr align=left>
<DIV dir=ltr align=left><SPAN class=747544015-09062009><FONT color=#0000ff
size=2 face=Arial>Option 5: Install a local caching DNS server on your
nagios box, and put 127.0.0.1 at the top of resolv.conf.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=747544015-09062009><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=747544015-09062009><FONT color=#0000ff
size=2 face=Arial>Cheers,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=747544015-09062009><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=747544015-09062009><FONT color=#0000ff
size=2 face=Arial>Phil</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=747544015-09062009></SPAN><FONT size=2
face=Arial>--</FONT> <BR><FONT size=2 face=Arial>Phil Randal | Networks
Engineer</FONT> <BR><FONT size=2 face=Arial>Herefordshire Council | Deputy Chief
Executive's Office | I.C.T. Services Division</FONT> <BR><FONT size=2
face=Arial>Thorn Office Centre, Rotherwas, Hereford, HR2 6JT</FONT> <BR><FONT
size=2 face=Arial>Tel: 01432 260160</FONT> <BR><FONT size=2 face=Arial>email:
prandal@herefordshire.gov.uk</FONT> </DIV></DIV>
<P><FONT size=2 face=Arial>Any opinion expressed in this e-mail or any attached
files are those of the individual and not necessarily those of Herefordshire
Council.</FONT></P>
<P><FONT size=2 face=Arial>This e-mail and any attached files are confidential
and intended solely for the use of the addressee. This communication may contain
material protected by law from being passed on. If you are not the intended
recipient and have received this e-mail in error, you are advised that any use,
dissemination, forwarding, printing or copying of this e-mail is strictly
prohibited. If you have received this e-mail in error please contact the sender
immediately and destroy all copies of it.</FONT></P>
<DIV> </DIV><BR>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B> Andrew Davis [mailto:nccomp@gmail.com]
<BR><B>Sent:</B> 09 June 2009 16:19<BR><B>To:</B>
nagios-users@lists.sourceforge.net<BR><B>Subject:</B> [Nagios-users] DNS down
and false alerts...<BR></FONT><BR></DIV>
<DIV></DIV>I've observed an interesting issue with Nagios. Our environment is a
mix of UNIX, Linux, Apple, and Windows. The core of the network is Active
Directory including two AD servers that are both our primary, internal DNS
servers. All non-Windows systems have a resolv.conf that looks like:<BR>
<BLOCKQUOTE><B>nameserver 10.1.1.13<BR>nameserver 10.1.1.14<BR>domain
int.our.domain<BR>search int.our.domain</B><BR></BLOCKQUOTE>About half of the
servers have the nameserver entries inverted (ie: .14 first, .13
second).<BR><BR>The issue is that anytime one of the nameservers is rebooted (at
least once a month if staying current on patches thanks to Black Tuesdays),
whichever hosts have that nameserver listed first in its resolv.conf start
throwing the following errors:<BR>
<BLOCKQUOTE><B>CRITICAL - Plugin timed out while executing system
call.</B><BR></BLOCKQUOTE>This occurs for multiple tests for each host.
Obviously, there's a name resolution correlation here. If the nameserver with
.13 is rebooted, all hosts (about half of them) that list this IP first in their
resolve.conf then timeout for multiple tests. If the .14 server is rebooted, all
the other hosts timeout. Interestingly, none of the Windows clients issue
errors... only UNIX, Linux, and Mac's... only those with an /etc/resolv.conf.
The end result is a host of "false positives", but more importantly it looks bad
on availability reports and causes phones/pagers to go ballistic with unneeded
emails.<BR><BR>I'm trying to find a solution and I can't find one that I
like:<BR><BR>Solution 1) is to cluster the DNS servers. We have lots of clusters
here. This isn't good, though, as you don't normally cluster DNS servers...
they're meant to be redundant for a reason... one fails and it uses the next
one.<BR><BR>Solution 2) is to setup a service/host dependency. My thought would
be either a host dependency that says if either .13 or .14 are down, then don't
alert for any other host that uses them. Or a service to host dependency... if
the DNS service is down, then don't alert on any of these dependent hosts.
Honestly, I'm not sure if you can mix host and service dependencies like this...
plus... if the DNS server is actually down, then the DNS service is down, so
better to use a host dependency. The problem is that now we're not alerting on
any dependent hosts which themselves could have a legitimate issue we want to
know about. Plus, what happens if the DNS server actually dies and take a few
hours/days to rebuild/restore? At this point, the dependent hosts aren't watched
for a very long time.<BR><BR>Solution 3) is to setup a UNIX/Linux DNS server
that slaves all zones from the AD servers and have all UNIX/Linux/Apple clients
query from this server. This would work except that A) I need two of them to
keep redundancy and B) I've now added an extra layer of complication to resolve
an application (Nagios)... not exactly good practice.<BR><BR>Solution 4) is to
set the timeout value of a host querying a DNS server. Perhaps adjust the client
to timeout on the first listed nameserver after only 10 seconds, then try the
next one? Since most Nagios tests have a minimum timeout value of 30 seconds, if
the first DNS query timed out after 10 seconds, it would go to the next one
with, hopefully, enough time to respond. The downside is having to adjust every
single server.<BR><BR>Has anyone else seen this? Anyone else using Windows AD
servers to provide DNS for *nix servers? <BR><PRE class=moz-signature cols="72">--
A. Davis
Email: <A class=moz-txt-link-abbreviated href="mailto:nccomp@gmail.com">nccomp@gmail.com</A>
"There is no limit to what a man can accomplish
if he doesn't care who gets the credit." - Ronald Reagan
</PRE></BODY></HTML>