nagios && rrd. Long and boring.

[Stanley Hopcroft]

Dear Sir,

I am writing to thank you for your letter and say,

On Wed, Apr 16, 2003 at 09:09:49AM +0200, Fred Albrecht wrote:

> 
> Ah, I've learnt something today! ;)  From my background with cricket I never really messed with the details of rrd,
> cricket simply configured the rrd for me. 

Yes, this is where Cricket and friends and value.

>From the rrdtoo create man page

NAME
       rrdtool create - Set up a new Round Robin Database
 
SYNOPSIS
       rrdtool create filename [--start|-b start time]
       [--step|-s step] [DS:ds-name:DST:heartbeat:min:max]
       [RRA:CF:xff:steps:rows]
 
DESCRIPTION
       The create function of the RRDtool lets you set up new
       Round Robin Database (RRD) files.  The file is created at
       its final, full size and filled with *UNKNOWN* data.
 
       RRA:CF:xff:steps:rows
               The purpose of an RRD is to store data in the
               round robin archives (RRA). An archive consists of
               a number of data values from all the defined data-
               sources (DS) and is defined with an RRA line.
 
               When data is entered into an RRD, it is first fit
               into time slots of the length defined with the -s
               option becoming a primary data point.
 
               The data is also consolidated with the
               consolidation function (CF) of the archive. The
               following consolidation functions are defined:
               AVERAGE, MIN, MAX, LAST.

               ... 
 
               steps defines how many of these primary data
               points are used to build a consolidated data point
               which then goes into the archive.
 
               rows defines how many generations of data values
               are kept in an RRA.

The CFs provide the means of saving a summary of the data points, when
the number of data points exceeds the numbers of rows defined for an
RRA.

The definition of an RRD typically includes a whole bunch of RRAs: this
gets to be tedious without the infrastructure provided by an RRD
framework such as Steve Shipways (? see the RRD web page), Cricket,
RRDFramework etc.

Here's an example of creating one from Perl

RRDs::create ($rrd, "--start", time() - 1, "--step",300,
                "DS:SAP:COUNTER:600:0:U",
                "DS:HTTP:COUNTER:600:0:U",
                "DS:DNS:COUNTER:600:0:U",
                "DS:Virus:COUNTER:600:0:U",
                "DS:Ntop:COUNTER:600:0:U",
                "DS:NBIOS-IP:COUNTER:600:0:U",
                "DS:Mail:COUNTER:600:0:U",
                "DS:SNMP:COUNTER:600:0:U",
                "DS:Telnet:COUNTER:600:0:U",
                "DS:Cache:COUNTER:600:0:U",
                "DS:PSCAM:COUNTER:600:0:U",
                "DS:BEA:COUNTER:600:0:U",
                "DS:SSH:COUNTER:600:0:U",
                "DS:LPD:COUNTER:600:0:U",
                "DS:PCAny:COUNTER:600:0:U",

                "RRA:AVERAGE:0.5:1:2016",
                "RRA:AVERAGE:0.5:6:672",
                "RRA:AVERAGE:0.5:24:672",

                "RRA:MAX:0.5:1:2016",
                "RRA:MAX:0.5:6:672",
                "RRA:MAX:0.5:24:672",

                "RRA:HWPREDICT:2016:0.28:0.0035:288",

                );


> The default time (I
> think) that cricket takes is to keep the records "uncompressed" for
> about two days, then averaging starts.  
>

in that case, Cricket may define the first RRA with rows == max(2 * 288,
2**n st n = int(log _base2_ 2 * 288) + 0.5 )
 
> How would rrd handle 3000+ datafiles with 32k+ rows each?  Any idea what kind of performance impact this will have?
>

I can't say. However,

. MRTG v3 will use RRDs

. rrdtool is claimed to be thread safe or nearly so

. rrd_update.c uses either fstat (Win 32) or fcntl (POSIX) to lock the
RRD prior to update. This I think allows concurrent update.

While I suspect this is the standard POSIX way of allowing concurrent
writes - APUUE may spell it out in the chapter on Databases - I don't
know how well it performs. Probably, since this is only way of
providing concurrent writes, does as well as a DB.

. Orca seems to be fast

. and one more HUGE advantage forgotten before. With COUNTER datatypes
(eg Octets from an Interface), rrdtool outputs the instantaneous rate
(ie last two values divided by step). You don't have to work out Kbp/s
or Mbp/s.

However, the main speed limiter in the example you cite is probably the
file system. You probably wouldn't keep your RRDs on NFS anymore than
your DB files (even if NFS locking works).

Some file systems perform much better than others. Maybe having a memory
mapped file system would be a big winner in the RRD case since the RRD
looks after missing values (if you crash).
 
> Thanx for clearing this up.
> 
> :)
> fred
> 

Yours sincerely.

-- 
------------------------------------------------------------------------
Stanley Hopcroft
------------------------------------------------------------------------

'...No man is an island, entire of itself; every man is a piece of the
continent, a part of the main. If a clod be washed away by the sea,
Europe is the less, as well as if a promontory were, as well as if a
manor of thy friend's or of thine own were. Any man's death diminishes
me, because I am involved in mankind; and therefore never send to know
for whom the bell tolls; it tolls for thee...'

from Meditation 17, J Donne.


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null