Really slow log2ndo

[Mikael Fridh]

On Thu, Sep 18, 2008 at 12:20 PM, Andreas Ericsson <ae at op5.se> wrote:
> Benjamin Krein wrote:
>> On Sep 18, 2008, at 4:04 AM, Andreas Ericsson wrote:
>>> Mikael Fridh wrote
>>>> LOG ROTATION: DAILY
>>> Implementation detail junk.

The log entries where just an excerpt and I wasn't putting any weight
on importance of their actual contents, but raw log messages _can_ be
interesting for some humans... I'm not saying it's always useful, but
it's there if you want it. As a user you're free to just devnull it I
guess.

>>>> I would use MyISAM MERGE tables or PARTITIONS for this if you want to
>>>> keep an "online" archive of the logs.
>>>> With the MERGE trick you can compress old tables and rejoin them in
>>>> the merge periodically if you'd like.
>>>>
>>> Or one can just print the logfiles from disk. They're already
>>> partitioned
>>> into rotated logfiles so it's not that much of a chore.

But they're not centralized which is one of the points of ndo.
And selecting a time range in SQL is pretty darn easy and fast if the
time columns are indexed.
But if you don't like the log in the database, BLACKHOLE it or something ... ;)

>>>> The event history of objects (services and hosts) is already in
>>>> nagios_statehistory, so what else is there in the log that you could
>>>> gain so much from parsing out into separate tables/fields?
>>>>
>>> Notifications. Program start and stop. Downtime start and stop.

Also available in standard ndo structure afaik:

notifications Table
"This table is used to store a historical record of host and service
notifications that have been sent out."

processevents Table
"This table is used to store a historical record of Nagios process
events (program starts, restarts, shutdowns, etc.)."

downtimehistory Table
"This table is used to store a historical record of scheduled host and
service downtime."

statehistory Table
"This table is used to store a historical record of host and service
state changes."

>> My ultimate goal for using these log entries in the DB is to compile
>> reports based on various metrics that are gathered by Nagios already.
>> The basic trend reports in Nagios & things like NagiosGrapher are ok,
>> but they aren't very flexible.  Pulling that data from the DB would be
>> far easier, but not in the format it's in now.  As it is now, it's no
>> different than just grepping files on disk.

So far you've only been talking about the logentries table and it's
format and I'm not disputing it's uselessness when it comes to
reporting but instead I'm curious about what's wrong with the other
tables in ndo and what they're missing (other than indexes for certain
use cases of course).

--
Mike

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/