Parsing Compressed Log Files In CGIs

[Andreas Ericsson]

Caylan Van Larson wrote:
> Our log files grow to 40M each day and compress down to 4M easily. Our  
> goal is to keep a years worth of log files for verifying uptime to our  
> clients.
> 
> Are there any plans to build compression into the archival/cgi-parsing  
> process?
> 

Nope. There's plans on creating a new GUI (well, "plans" is too weak a
word, as several are already brewing). It's likely these new gui's won't
use the logfiles at all, so you'll be able to rotate them as often as
you like and just tell mysql/postgresql/oracle/whatever to compress the
database files instead.

Otoh, trial-and-error in the git version control system indicates that
it's actually faster to read compressed data from disk and decompress
it in memory, so it might actually be a performance improvement to
store the data compressed with something like lzo (which is wicked fast
when it comes to decompressing).

> If not, what is the bare minimum logging that I can get away with and  
> still have decent trend/availability reporting?
> 

You'll need to store logs as long back as you want to be able to create
reports for, so if you want to be able to create reports for one quarter
of a year, you'll need to stash 40*90 MB of data (3.600MB).

What with disk space being so cheap now adays, I don't really see a
problem.

> log_notifications=1
> log_service_retries=1
> log_host_retries=1
> log_event_handlers=1
> log_initial_states=0
> log_external_commands=1
> log_passive_checks=1
> 
> Looking at these I'm wondering if I need any of these for the CGIs to  
> be effective in reporting...
> 

log_initial_states can be important, depending on your report settings.
Assuming you don't restart Nagios all that often, it'll barely show on
the disk usage.

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/