Nagios 1.2 fast IO-patch

[Andreas Ericsson]

Ethan Galstad wrote:
> This patch is a bit old, but I've just gotten around to testing it on 
> the 2.0 code.  From what I can tell, there's no measureable 
> difference with or without the patch.  I tested three different 
> reporting ranges ("today", "this year", and "last year") on two 
> separate boxes.  Both boxes are RedHat 7.2 running kernel-2.4.20-
> 20.7.
> 
> One box is a PII 400 w/ 220 MB RAM, an 8 GB SCSI drive and 45 hosts 
> in the report.  The other is a PII 300 w/ ~200 MB RAM, an 8 GB SCSI 
> drive and 15 hosts in the report.
> 
> Report generation times were within 5% of each other on both machines 
> when tested w/ and w/o the patch (sometimes the patches version was 
> actually slightly slower than the nonpatched code).  Average report 
> times for "today", "this year" and "last year" are approx: 1.25 sec, 
> 6.2 sec, and 7.7 sec.
> 
> Perhaps the SCSI drives are doing a better job at predictive reads 
> than the IDE drives you tested.  Not sure.  Or perhaps its something 
> else.  For the record, I have not tested this against 1.2, but I 
> wouldn't think the results would be much different.  Anyone else 
> tried this patch out?  If so, what were your findings?
> 

Well, I have, ofcourse. ;-)
It gives a rather noticable improvement on a system with IDE disks in 
software RAID arrays. I had a look in the kernel code, and it's quite 
obvious why this is so.
On IDE disks, the kernel doesn't implement read-ahead at all, unless it 
is explicitly told to read more of the file its pointed to. This is 
largely because of the fact that some (older) IDE disks don't have very 
large cache-buffers so the kernel doesn't want to waste memory holding 
parts of a file it may not necessarily need. The scsi standards, 
however, defines a minimum cache-buffer size so the kernel can happily 
read a larger chunk every time and let the hardware cache it until it's 
needed.
With software RAID it also does integrity checking once in a while. This 
is faster to do once on a large chunk than many times on smaller chunks.

The somewhat miscalculated measurement times in the message below was 
due to a mistake in the patch which led to the archived logfiles not 
being read at all (making state 100% undetermined), but that matched the 
original binary perfectly, so it took me a while to get to the bottom of 
the problem. This was fixed before resubmitting.

Anyways... the patch doesn't do much good on most systems (non-RAID, 
SCSI), but it does for us, so I'll keep it up to date and apply it to 
our systems. It also won't work very well unless DMA is turned on (using 
the hdparm utility).


> 
> 
> On 22 Mar 2004 at 17:07, Andreas Ericsson wrote:
> 
> 
>>Hi all.
>>
>>I wrote this patch the other day when I was juggling with automating
>>reports (avail.cgi), and I noticed that parsing the files alone took 8
>>seconds, even though it was less than 3MB of data. The solution was to
>>rewrite the read_line() function in cgiutils.c to utilize larger
>>read()-buffers and thus allow for IO bursting. With this patch applied
>>execution time went down to 0.7 seconds.
>>
>>It also solves the problem with not-newline-terminated rows not being
>>read in the configuration files (discussed earlier).
>>
>>The only drawback is that it will take forever to parse files that are
>>larger than you have physical RAM available (the kernel will utilize
>>the swap-space), and will fail completely (pretending the file is
>>empty) if the file is larger than the physical AND virtual RAM
>>combined. However, considering that it already takes forever even with
>>relatively small files, this shouldn't be a problem.
>>
>>This patch is also available for 2.0a1, but I haven't had time to test
>>it to any extent with that release.
>>
>>Tested on Openwall GNU/*/Linux, kernel 2.4.24-ow1 running on;
>>a) Celeron 2.0Ghz with 256MB RAM and a single 40GB IDE-disk.
>>b) Pentium 4 HT 2.8Ghz with 512MB RAM and dual 120GB IDE-disks with
>>software RAID-mirroring. c) Dell Inspiron PIII 866Mhz with 256MB RAM
>>and a single 20GB IDE-disk.
>>
>>Cheers,
>>Sourcerer / Andreas Ericsson
>>
> 
> 
> 
> 
> Ethan Galstad,
> Nagios Developer
> ---
> Email: nagios at nagios.org
> Website: http://www.nagios.org
> 
> 

-- 
Sourcerer / Andreas Ericsson
OP5 AB
+46 (0)733 709032
andreas.ericsson at op5.se


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click