Latencies with process-perfdata command

[Marc DeTrano]

Hendrik Baecker wrote:
> Hi List,
>
> I figured out, that I run into some latency problems if I use a
> process-perfdata command to handle plugin performance data with external
> scripts.
>
> So I am asking myself, what would be the best way (in thought of not
> getting into latency troube) to handle the perfdata.
>
>   
> Thanks in advance for every kind of answer (except of ndr and auto
> replies ;o) )
>
> Regards,
> Hendrik
>
>   

I am constantly working through the same kind of issues for perf2rrd.
Here's what we've done with that program so far:

perf2rrd runs as a daemon in the background, to avoid launching the same
program over and over.

Nagios sends perfdata to a pipe.

Perf2rrd has one thread dedicated to reading that pipe. As soon as it
has the data, it releases it so Nagios can continue.

A second thread is running to receive and parse the data that it gets
from the read-fifo thread. It also handles the write to rrds via a JNI
interface.

In a test environment, I've been able to get a dual-core Xeon 64-bit
machine , with plenty of RAM and a sata drive, to keep up with about
9000 items of perfdata (for 4500 service checks), updating every two
minutes. That's 75 items/sec. Latency at that level is about 2 seconds.
Not great, but maybe not so bad either. If I crank it up much higher
than that my latency starts to shoot up. However, I am finding that even
if I shut off perfdata processing, my latency and cpu load is still high
when I get up to that level.

In one production environment, I have a single-core 32-bit P4, with an
ata drive, keeping up with about 3500 items of perfdata every two
minutes (30 items/sec), but it is just about at its limit and the
slightest extra activity on the machine causes latency to rise. I cannot
disable the perfdata component on that system, however, to determine how
much it is contributing to the load.

I would be curious to know how this compares with some of the things
that have come along since perf2rrd, and some of the ideas in
development (is someone writing a NEB module to parse this data?)

Marc
Gridshield, S.A:




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642