Memory leak

[Andreas Ericsson]

Arno Lehmann wrote:
> Sir... tststs :-)
> 
> Stanley Hopcroft wrote:
> 
>> Dear Sir,
>>
>> I am writing to thank you for your letter and say,
>>
>>>
>>> In the mailing list, I found this reported as a known bug concerning 
>>> embedded perl, and in December Andreas wrote that one ugly leak was 
>>> fixed...should be in b2, shouldn't it?
>>
>>
>>
>> There is an unfixed known bug using embedded Perl of this nature (see 
>> perldoc p1.pl).
>>
>> You may have found a new leak.
> 
> 
> Possible...
> 
>>> Now, I'm wondering if this is a new bug, or a known issue - I 
>>> couldn't find anything in the documentation or release notes... on 
>>> the other hand, this should be obvious to others.
>>>
>>> My configuration: Linux Kernel  2.6.8-24.13-default (SuSE 9.2), 
>>> nagios 2.0b2 just as downloaded, compiled without (to my 
>>> knowledge...) perl support.
>>>
>>
>>
>> I think that something like this
>>
>> tsitc> nm <path to build directory of Nag>/base/nagios | grep perl
>> 0806c29c T deinit_embedded_perl
>> 0806c1dc T init_embedded_perl
>> 080a4068 d my_perl
>>          U perl_alloc
>>          U perl_construct
>>          U perl_destruct
>>          U perl_free
>>          U perl_parse
>>          U perl_run
>> 080a406c D use_embedded_perl
> 
> 
> Ah, great... not being a programmer nm was not an idea I had.
> 
> Well, I tried it, and after seeing that the instaled binary was stripped 
> I checked the one in the build tree and found it only has the init and 
> deinit functions defined.
> 
> So, perl is not included, but the memory leak is???
> 
>>
>> will tell you if you have embedded Perl (if you see D 
>> use_embedded_perl, you have Perl compiled into your Nag).
> 
> 
> See above: No, as I thought.
> 
>>> Everything else works as expected.
>>>
>>> Unfortunately, I can't determine how the memory gets "eaten" - ps 
>>> doesn't tell me about any processes with huge RSSes. In fact, I can't 
>>> be sure that Nagios itself leaks that memory, but it happens if and 
>>> only when Nagios is running.
>>
>>
>>
>> top does on this BSD host eg
> 
> ...
> 
>> Likewise you could use check_procs from the standard plugin set to 
>> monitor vsz.
> 
> 
> Well, I tried top as well as ps and the memory isn't used by any process 
> as far as I can determine it. Logged output from ps:
> 

According to this listing Nagios doesn't seem to consume an unusual 
amount of memory.

>>              total       used       free     shared    buffers     cached
>> Mem:        515636     512884       2752          0       1800       5976
>> -/+ buffers/cache:     505108      10528
>> Swap:      2120540      47852    2072688
>>   PID USER      RSS    SZ CMD
>>  4039 ntp      5228  1308 /usr/sbin/ntpd -p 
>> /var/lib/ntp/var/run/ntp/ntpd.pid -u ntp -i /var/lib/ntp
>> 20740 wwwrun   2124  7678 /usr/sbin/httpd2-prefork -f 
>> /etc/apache2/httpd.conf -DSSL
>> 20294 wwwrun   2100  7657 /usr/sbin/httpd2-prefork -f 
>> /etc/apache2/httpd.conf -DSSL
>>  4279 root     1724  2675 /usr/sbin/smbd -D -s /etc/samba/smb.conf
>>  4286 root     1724  2683 /usr/sbin/smbd -D -s /etc/samba/smb.conf
>> 26755 cyrus    1704  7431 imapd
>> 26757 cyrus    1704  7461 imapd
>> 26126 cyrus    1700  7421 imapd
>> 26754 cyrus    1700  7425 imapd
>> 26756 cyrus    1700  7411 imapd
>>  8176 cyrus    1692  7411 imapd
>>  8177 cyrus    1692  7421 imapd
>> 20707 nagios   1596  2841 /usr/local/nagios/bin/nagios -d 
>> /usr/local/nagios/etc/nagios.cfg
>>  8635 nagios   1572  2842 /usr/local/nagios/bin/nagios -d 
>> /usr/local/nagios/etc/nagios.cfg
>>  4425 root     1568  7253 /usr/sbin/httpd2-prefork -f 
>> /etc/apache2/httpd.conf -DSSL
>>  4450 root     1544  1200 /usr/bin/fetchmail -d 912 -a -f 
>> /etc/fetchmailrc -L /var/log/fetchmail
>>  8614 postfix  1528  1550 smtpd -n smtp -t inet -u -s 2
>>  4364 postfix  1520  1400 qmgr -l -t fifo -u
>>  7831 postfix  1504  1391 pickup -l -t fifo -u
>>  8625 cyrus    1228  6983 imapd
>>  8634 root     1168  1221 ps -eopid,user,rss,sz,cmd --sort=-rss
>>  7932 man      1124  1226 /usr/bin/mandb -q -s
>> 23535 root     1092  1402 -bash
>>  4349 root     1088  1069 /usr/lib/postfix/master
>>  3632 root     1084  1701 /usr/sbin/saslauthd -a pam
>>  4440 root     1060  1161 /usr/sbin/xinetd
>>  4740 root     1044  1210 login -- root
>>  4473 root     1012  1086 /sbin/mingetty --noclear tty1
>>  4475 root     1012  1086 /sbin/mingetty tty3
>>  4476 root     1012  1086 /sbin/mingetty tty4
>>  4477 root     1012  1086 /sbin/mingetty tty5
>>  4478 root     1012  1086 /sbin/mingetty tty6
>>  4465 root      992  1046 /usr/sbin/cron
>>  7705 root      992  1047 /usr/sbin/cron
>>  3709 root      964  1357 /sbin/resmgrd
>>  4266 dhcpd     908  1139 /usr/sbin/dhcpd -chroot /var/lib/dhcp/ -lf 
>> /db/dhcpd.leases -user dhcpd -group nogroup eth0
>>  8636 nagios    860   656 sh -c sudo -u root 
>> /usr/local/nagios/libexec/check_icmp -H THIS_IS_PRIVATE -w 300.0,30% 
>> -c 500.0,70% -p 10
>>  4169 root      824   710 /usr/sbin/ypserv
>>  4399 root      660   430 /usr/sbin/rpc.mountd
>>  4722 root      528   655 /bin/bash bin/rrdloadsave.sh
>>  7713 root      488   591 /bin/bash /usr/lib/cron/run-crons
>>  7706 root      476   590 /bin/sh -c  test -x /usr/lib/cron/run-crons 
>> && /usr/lib/cron/run-crons >/dev/null 2>&1
>>  7931 root      472   590 /bin/sh /etc/cron.daily/do_mandb
>>  4481 nagios    424   419 nsca -c /usr/local/nagios/etc/nsca.cfg --inetd
>>  4482 nagios    424   419 nsca -c /usr/local/nagios/etc/nsca.cfg --inetd
>>  4570 nagios    424   419 nsca -c /usr/local/nagios/etc/nsca.cfg --inetd
>>  4571 nagios    424   419 nsca -c /usr/local/nagios/etc/nsca.cfg --inetd
>>  4658 nagios    424   419 nsca -c /usr/local/nagios/etc/nsca.cfg --inetd
>>  4659 nagios    424   419 nsca -c /usr/local/nagios/etc/nsca.cfg --inetd
>>  3840 named     412  8380 /usr/sbin/named -t /var/lib/named -u named
>>  3646 root      340  1627 /usr/sbin/nmbd -D -s /etc/samba/smb.conf
>>  3626 root      128  1596 /usr/sbin/saslauthd -a pam
>>  3631 root      128  1596 /usr/sbin/saslauthd -a pam
>>  3633 root      128  1596 /usr/sbin/saslauthd -a pam
>>  3634 root      128  1596 /usr/sbin/saslauthd -a pam
>>  3924 ldap      112 22147 /usr/lib/openldap/slapd -h ldap:/// -u ldap 
>> -g ldap -o slp=on
>>  3701 root      108  6887 /usr/sbin/bacula-fd -c 
>> /etc/bacula/bacula-fd.conf
>>  3729 cyrus     100  1426 /usr/lib/cyrus/bin/master -p 
>> /var/run/cyrus.pid -d
>>  3912 mysql      52  6199 /usr/sbin/mysqld --basedir=/usr 
>> --datadir=/var/lib/mysql --user=mysql 
>> --pid-file=/var/lib/mysql/mysqld.pid --skip-locking --port=3306 
>> --socket=/var/lib/mysql/mysql.sock
>>  3593 root       48   361 /sbin/syslogd -a /var/lib/dhcp/dev/log -a 
>> /var/lib/named/dev/log -a /var/lib/ntp/dev/log
>>  3927 cyrus      40  6795 idled
>>     1 root       24   149 init [3]
>>  3613 root       24  1770 /usr/sbin/snmpd -r -A -Lf 
>> /var/log/net-snmpd.log -p /var/run/snmpd.pid
>>  2133 root       12   340 udevd
>>  2440 root       12   358 [hwscand]
>>  3459 root       12   358 [hwscand]
>>  3598 root       12   379 /sbin/klogd -c 1 -2 -x
>>  3719 nobody     12   357 /sbin/portmap
>>  3801 root       12  1424 /usr/sbin/sshd -o 
>> PidFile=/var/run/sshd.init.pid
>>  3863 root       12   591 /bin/sh /usr/bin/mysqld_safe --user=mysql 
>> --pid-file=/var/lib/mysql/mysqld.pid 
>> --socket=/var/lib/mysql/mysql.sock --datadir=/var/lib/mysql
>>     2 root        0     0 [ksoftirqd/0]
>>     3 root        0     0 [events/0]
>>     4 root        0     0 [khelper]
>>     5 root        0     0 [netlink/0]
>>     6 root        0     0 [kblockd/0]
>>    29 root        0     0 [kapmd]
>>    31 root        0     0 [pdflush]
>>    32 root        0     0 [pdflush]
>>    34 root        0     0 [aio/0]
>>    33 root        0     0 [kswapd0]
>>   618 root        0     0 [kseriod]
>>  1585 root        0     0 [reiserfs/0]
>>  2077 root        0     0 [khubd]
>>  2465 root        0     0 [khpsbpkt]
>>  2483 root        0     0 [knodemgrd_0]
>>  4375 root        0     0 [nfsd]
>>  4376 root        0     0 [nfsd]
>>  4377 root        0     0 [nfsd]
>>  4378 root        0     0 [nfsd]
>>  4382 root        0     0 [lockd]
>>  4383 root        0     0 [rpciod]
> 
> 
> As far as I can see, the sizes don't add to the amount given as used.
> 

The kernel uses memory, and most os's implement copy-on-write with 
forked processes (Linux does this, and judging by the apps running 
that's what you're using). That means only changed frames are actually 
copied on a fork(), but the theoretical maximum consumption (as 
determined by allocated buffers in the master process) is displayed anyways.

> Any other ideas?
> 

Run it through valgrind and log everything. Post the logs on some public 
webpage so users with little or no interest doesn't have to cope with 
them on the list.

> Arno
> 

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Lead Developer


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Nagios-users mailing list
Nagios-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nagios-users
::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. 
::: Messages without supporting info will risk being sent to /dev/null