I think I found the issue. If I happen to send a reload (HUP) to nagios while a service check is in progress (fairly easy since my service check is rather long lived), the reloaded nagios doesn't seem to know about that service check and so I'll end up with another being scheduled as well as the original on its schedule. Create a dummy service check that just sleeps for 30 seconds or something and issue a reload while it is running and see if your nagios instance will start another sequence of service checks.<div class="gmail_extra">
<br><br><div class="gmail_quote">On Thu, Dec 13, 2012 at 2:37 PM, Mike Guthrie <span dir="ltr"><<a href="mailto:mguthrie@nagios.com" target="_blank">mguthrie@nagios.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><div class="im">
<div><br>
On 12/13/2012 12:38 PM, Mark Keisler wrote:<br>
</div>
<blockquote type="cite">I understand that nagios dynamically adjusts service
check times, but the puzzling thing is that there is a check that
runs every 5 minutes but then an extra or two in between. And
yes, the web interface shows the next service check as 5 mins out
and yet another runs before that time hits.</blockquote></div>
Is there any chance that there could be a second instance of Nagios
running? Look for multiple *parent* processes from the following<br>
<br>
<pre>#modify the nagios binary path to match your system</pre>
<pre>ps aux | grep /bin/nagios</pre>
<pre></pre>
<pre>/etc/init.d/nagios stop</pre>
<pre>killall -9 nagios</pre>
<pre>/etc/init.d/nagios start</pre><div><div class="h5">
<br>
<br>
<blockquote type="cite">
<div class="gmail_extra">
<br>
<br>
<div class="gmail_quote">On Thu, Dec 13, 2012 at 10:24 AM, Mike
Guthrie <span dir="ltr"><<a href="mailto:mguthrie@nagios.com" target="_blank">mguthrie@nagios.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Although some of those start times do seem close
together, it's important to know that the check_interval
in Nagios is not necessarily a hard number. Nagios is
continually adjusting and recalculating the check
schedule, so if you need a check to run on a hard 5mn
schedule, you might be better off using cron, and then
pushing the result to Nagios passively. <br>
<br>
With that said, access the service details for this
service. When new results come in does the scheduler set
the Next Check 5mn out as expected?
<div>
<div><br>
<br>
<br>
On 12/13/2012 9:43 AM, Mark Keisler wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>I'm running Nagios 3.4.1 on RHEL6. I
have an issue where I have a poller (service check)
that is running too often and I am not sure why. I
have "service_check_timeout=180" because I had
trouble with the poller running long. Relevant
settings for the service check:<br>
<br>
check_period 24x7<br>
max_check_attempts 1<br>
normal_check_interval 5<br>
retry_check_interval 5 <br>
<br>
I also set up a tracking logger in the poller to
record "timestamp PID started by PPID : Poll
[Start|End] of poller"<br>
<div>2012-12-12_12:26:38 19448 started by 19442 :
Poll Start of poller<br>
2012-12-12_12:27:13 19448 started by 19442 : Poll
End of poller<br>
2012-12-12_12:28:14 19931 started by 19930 : Poll
Start of poller<br>
2012-12-12_12:30:14 19931 started by 19930 : Poll
End of poller<br>
2012-12-12_12:31:37 20467 started by 20460 : Poll
Start of poller<br>
2012-12-12_12:33:15 20949 started by 20946 : Poll
Start of poller<br>
2012-12-12_12:33:15 20467 started by 20460 : Poll
End of poller<br>
2012-12-12_12:33:41 20949 started by 20946 : Poll
End of poller<br>
2012-12-12_12:36:38 21483 started by 21478 : Poll
Start of poller<br>
2012-12-12_12:38:14 21971 started by 21964 : Poll
Start of poller<br>
2012-12-12_12:39:17 21483 started by 21478 : Poll
End of poller<br>
2012-12-12_12:39:18 21971 started by 21964 : Poll
End of poller<br>
2012-12-12_12:41:38 22500 started by 22492 : Poll
Start of poller<br>
2012-12-12_12:42:19 22500 started by 22492 : Poll
End of poller<br>
2012-12-12_12:43:14 23003 started by 22999 : Poll
Start of poller<br>
2012-12-12_12:45:20 23003 started by 22999 : Poll
End of poller<br>
2012-12-12_12:46:37 23540 started by 23535 : Poll
Start of poller<br>
2012-12-12_12:48:14 24025 started by 24024 : Poll
Start of poller<br>
2012-12-12_12:48:20 23540 started by 23535 : Poll
End of poller<br>
2012-12-12_12:48:41 24025 started by 24024 : Poll
End of poller<br>
2012-12-12_12:51:38 24558 started by 24554 : Poll
Start of poller<br>
2012-12-12_12:53:14 25044 started by 25041 : Poll
Start of poller<br>
2012-12-12_12:54:35 25044 started by 25041 : Poll
End of poller<br>
<br>
As you can see, I start to get overlapping
pollers. I don't understand why this would happen.
Any hints or clues?<br>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
<a href="http://p.sf.net/sfu/logmein_12329d2d" target="_blank">http://p.sf.net/sfu/logmein_12329d2d</a></pre>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Nagios-users mailing list
<a href="mailto:Nagios-users@lists.sourceforge.net" target="_blank">Nagios-users@lists.sourceforge.net</a>
<a href="https://lists.sourceforge.net/lists/listinfo/nagios-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/nagios-users</a>
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null</pre>
<span><font color="#888888"> </font></span></blockquote>
<span><font color="#888888"> <br>
<br>
<pre cols="72">--
Mike Guthrie
Technical Team
___
Nagios Enterprises, LLC
Email: <a href="mailto:mguthrie@nagios.com" target="_blank">mguthrie@nagios.com</a>
Web: <a href="http://www.nagios.com" target="_blank">www.nagios.com</a>
</pre>
</font></span></div>
<br>
------------------------------------------------------------------------------<br>
LogMeIn Rescue: Anywhere, Anytime Remote support for IT.
Free Trial<br>
Remotely access PCs and mobile devices and provide instant
support<br>
Improve your efficiency, and focus on delivering more
value-add services<br>
Discover what IT Professionals Know. Rescue delivers<br>
<a href="http://p.sf.net/sfu/logmein_12329d2d" target="_blank">http://p.sf.net/sfu/logmein_12329d2d</a><br>
_______________________________________________<br>
Nagios-users mailing list<br>
<a href="mailto:Nagios-users@lists.sourceforge.net" target="_blank">Nagios-users@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/nagios-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/nagios-users</a><br>
::: Please include Nagios version, plugin version (-v) and
OS when reporting any issue.<br>
::: Messages without supporting info will risk being sent to
/dev/null<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
<a href="http://p.sf.net/sfu/logmein_12329d2d" target="_blank">http://p.sf.net/sfu/logmein_12329d2d</a></pre>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Nagios-users mailing list
<a href="mailto:Nagios-users@lists.sourceforge.net" target="_blank">Nagios-users@lists.sourceforge.net</a>
<a href="https://lists.sourceforge.net/lists/listinfo/nagios-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/nagios-users</a>
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.
::: Messages without supporting info will risk being sent to /dev/null</pre>
</blockquote>
<br>
<br>
<pre cols="72">--
Mike Guthrie
Technical Team
___
Nagios Enterprises, LLC
Email: <a href="mailto:mguthrie@nagios.com" target="_blank">mguthrie@nagios.com</a>
Web: <a href="http://www.nagios.com" target="_blank">www.nagios.com</a>
</pre>
</div></div></div>
<br>------------------------------------------------------------------------------<br>
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial<br>
Remotely access PCs and mobile devices and provide instant support<br>
Improve your efficiency, and focus on delivering more value-add services<br>
Discover what IT Professionals Know. Rescue delivers<br>
<a href="http://p.sf.net/sfu/logmein_12329d2d" target="_blank">http://p.sf.net/sfu/logmein_12329d2d</a><br>_______________________________________________<br>
Nagios-users mailing list<br>
<a href="mailto:Nagios-users@lists.sourceforge.net">Nagios-users@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/nagios-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/nagios-users</a><br>
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue.<br>
::: Messages without supporting info will risk being sent to /dev/null<br></blockquote></div><br></div>