Really slow log2ndo

[Benjamin Krein]

I posted the following in the Users mailing list, but got to thinking  
that maybe Developer's list was a better place for it.

Am I the only one who is experiencing very slow performance importing  
archived log files into MySQL with log2ndo?  I'm averaging ~1.5 log  
entries per second being inserted.  This seems slow to me.

I've tried dividing the parsing & MySQL service across 2 servers as  
well as move all the parsing/inserting locally on a remote (to Nagios)  
server which is much beefier than our actual Nagios server.  My  
insertion speed remains fairly consistent.

I'm currently using the Unix socket option.  Is this the wrong one?   
Would TCP be faster (doesn't seem so)?  Should I process the logs to  
flat text files first then push those to ndo2db?

At this rate it's going to take FOREVER!  I have ~10 million archived  
lines to parse and insert.  Maybe I'm expecting too much?

Benjamin Krein
Systems Administrator
AWeber Communications, Inc.




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/