check_web_load plugin

[Skip Montanaro]

Last week the Mojam web server got hammered by an ill-behaved crawler,
probably harvesting email addresses.  Most such crawlers don't obey
robots.txt files, and if they dive into a region of your website with lots
of dynamic content, they can wreak havoc.

I learned from another group of admins that a simple routing trick can block
such crawlers without upsetting your web server's config files:

    function disable-host () {
        route add -host $1 gw 127.0.0.1
    }

    function enable-host () {
        route delete -host $1
    }

With a way to quickly solve the problem, you now need a way to quickly
detect it.  Checking the web server's load average is one technique.
Another which seems to work reasonably well and is more specific to the
particular problem is to consider the recent access history of the top two
client IP addresses.  I wrote a simple Nagios plugin (just a shell script at
this point) which compares accesses from the two most frequent clients.  You
can get a copy here:

    http://manatee.mojam.com/~skip/check_web_load.sh

Here's the help output:

    Usage: check_web_load.sh [ -h ] [ -w M ] [ -c M ] logfile ...
      -w M - warn if most frequent client has M times more hits
             than second most frequent client (default 3)
      -c M - critical if most frequent client has M times more hits
             than second most frequent client (default 6)
      -l N - specify last N lines of logfile to check (default 1000)
      -x P - specify egrep pattern P to exclude lines (default (/images|/icons))
    The warning factor must be strictly less than the critical factor.
    If more than one logfile is given, each is considered separately,
    and the worst status (OK, WARNING, CRITICAL) is reported.

Feedback is welcome.

-- 
Skip Montanaro
Got gigs? http://www.musi-cal.com/submit.html
Got spam? http://spambayes.sf.net/
skip at pobox.com


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null