[Nagiosplug-help] check_icmp seems flapping - followon to RE: make of nagios-plugins-1.4.5 on AIX 4.3 fails

[Ralph.Grothe at itdz-berlin.de]

Hallo Andreas,

your check_icmp plugin is working fine.

All nagios induced check_host calls must have caught the response
to their single packet
because all red host lights disappeared and stayed green ever
since.

Now I think I can safely switch back on the global notification
setting.

What still annoys me a little is that the nagios daemon doesn't
seem to
reinitialize correctly when it catches a SIGHUP.
(I waited quater of an hour, which would be far too long anyway)
Although it correctly reports the signal handling in its logfile
when I try to access the webinterface I get that Woops page as if
nagios hasn't started at all.
Because so far I have been using an early 2.0 beta version of
nagios
I thought this minor flaw was due to the beta stage.
Somehow for my new nagios 2.5 AIX build it is still prevelant.

$ ps -fp $(cat /opt/sw/nagios/var/nagios.lock)
     UID   PID  PPID   C    STIME    TTY  TIME CMD
  nagios 22302     1   0 10:39:26      -  3:57
/opt/sw/nagios/bin/nagios -d /opt/sw/nagios/etc/
 
$ kill -1 $(cat /opt/sw/nagios/var/nagios.lock) && tail -2
/opt/sw/nagios/var/nagios.log
[1164991166] Auto-save of retention data completed successfully.
[1164993038] Caught SIGHUP, restarting...


$ ps -fp $(cat /opt/sw/nagios/var/nagios.lock)
     UID   PID  PPID   C    STIME    TTY  TIME CMD
  nagios 22302     1  60 10:39:26      -  6:45
/opt/sw/nagios/bin/nagios -d /opt/sw/nagios/etc/



$ check_http -H thishost.thisdomain.tld -a nagiosadmin:secret -u
/nagios/cgi/tac.cgi
 -s 'Error: Could not read host and service status information!'
HTTP OK HTTP/1.1 200 OK - 0.197 second response time
|time=0.197270s;;;0.000000 size=2057B;;;0



> -----Original Message-----
> From: Grothe, Ralph 
> Sent: Friday, December 01, 2006 10:06 AM
> To: 'Andreas Ericsson'
> Cc: nagiosplug-help at lists.sourceforge.net
> Subject: RE: [Nagiosplug-help] check_icmp seems flapping -
followon to
> RE: make of nagios-plugins-1.4.5 on AIX 4.3 fails
> 
> 
> > The relevant pieces can be gleaned by
> > 	$ grep -B1 'pid & 0xffff' check_icmp.c
> 
> I was wondering why I didn't know the --before-context switch 
> as it seems to be a GNU grep only goody I suppose.
> AIX's grep is too dumb, but I found the line nevertheless.
> 
> > Attached is the version we use. You should be able to compile
it 
> > stand-alone on any sane system with
> > 	$ gcc check_icmp.c -o check_icmp -O2
> > 
> 
> produced just one warning which I assume can be neglected
> 
> $ gcc -O2 -o check_icmp check_icmp.c 
> check_icmp.c: In function `recvfrom_wto':
> check_icmp.c:975: warning: passing arg 6 of `nrecvfrom' from 
> incompatible pointer type
> 
> 
> $ ls -l check_icmp
> -rwxr-xr-x   1 nagios   nagios     86628 Dec 01 09:39
check_icmp
> 
> though this beggars description to any security concerned
admin...
> 
> $ su root -c 'chown root check_icmp && chmod 04555 check_icmp'
> root's Password: 
> $ ls -l check_icmp
> -r-sr-xr-x   1 root     nagios     86628 Dec 01 09:39
check_icmp
> 
> $ ./check_icmp -H 127.0.0.1 -p 5
> OK - 127.0.0.1: rta 0.155ms, lost 
> 0%|rta=0.155ms;200.000;500.000;0; pl=0%;40;80;; 
> 
> From this unconscientious and very superficial test it seems to
work.
> 
> I will place it in $USER1$ and have my nagios implementation 
> exercise it for a while
> to see if the ruddy host state fields disappear one by one 
> and remain in a sustained green...
> 
> 
> 
> > -----Original Message-----
> > From: Andreas Ericsson [mailto:ae at op5.se]
> > Sent: Thursday, November 30, 2006 5:00 PM
> > To: Grothe, Ralph
> > Cc: nagiosplug-help at lists.sourceforge.net
> > Subject: Re: [Nagiosplug-help] check_icmp seems flapping - 
> followon to
> > RE: make of nagios-plugins-1.4.5 on AIX 4.3 fails
> > 
> > 
> > Ralph.Grothe at itdz-berlin.de wrote:
> > >> -----Original Message-----
> > >> From: Andreas Ericsson [mailto:ae at op5.se]
> > >> Sent: Thursday, November 30, 2006 2:01 PM
> > >> To: Grothe, Ralph
> > >> Cc: nagiosplug-help at lists.sourceforge.net
> > >> Subject: Re: [Nagiosplug-help] check_icmp seems flapping -
> > > followon to
> > >> RE: make of nagios-plugins-1.4.5 on AIX 4.3 fails
> > >>
> > >>
> > >> Ralph.Grothe at itdz-berlin.de wrote:
> > >>> I am still seem to have some serious trouble with my
build of
> > > the
> > >>> check_icmp plugin.
> > >>>
> > >>>
> > >>> Now I am curiuous whether my compilation of check_icmp is
ok?
> > >>>
> > >> You'd get this problem if you use an old check_icmp on a
system
> > > that 
> > >> handles process id's > 65535. In the old version,
check_icmp
> > > didn't 
> > >> recognize valid ICMP responses because the id-field used
in the
> > > icmp 
> > >> header is only 16 bits wide, so a 32-bit pid doesn't fit
in it.
> > > This 
> > >> would typically only happen when the pid of check_icmp is
> > > larger than 
> > >> 65535, which would explain the checks hopping between OK
for 
> > >> a while and 
> > >> non-OK for a while. Judging by "mode: 1" above, I'd say
your 
> > >> check_icmp 
> > >> is fairly old and needs to be upgraded. What version of
the 
> > >> plugins are 
> > >> you using?
> > > 
> > > Oh, I thought I downloaded the latest nagios-plugins
tarball
> > > (viz. 1.4.5),
> > > of which I guess check_icmp is part of.
> > > Wait, there should be a revision header in the sources,
> > > or maybe check_icmp is revealing its age?
> > > 
> > > $ /opt/sw/nagios/libexec/check_icmp -V
> > > 
> > > no it's dumb as a grave
> > > 
> > > $ /opt/sw/nagios/libexec/check_icmp --version
> > > /opt/sw/nagios/libexec/check_icmp: illegal option -- -
> > > /opt/sw/nagios/libexec/check_icmp: illegal option -- e
> > > /opt/sw/nagios/libexec/check_icmp: illegal option -- r
> > > /opt/sw/nagios/libexec/check_icmp: illegal option -- s
> > > check_icmp: No hosts to check
> > > 
> > > 
> > > Oops, it isn't behaving according to its own help screen.
> > > But you are the author, then you should know better ;-)
> > > 
> > > $ /opt/sw/nagios/libexec/check_icmp -h|head -15
> > > Copyright (c) 2005 Andreas Ericsson <ae at op5.se>
> > > Copyright (c) 2005-2006 Nagios Plugin Development Team
> > >         <nagiosplug-devel at lists.sourceforge.net>
> > > 
> > > 
> > > 
> > > Usage: check_icmp [options] [-H] host1 host2 hostn
> > > 
> > > Options:
> > >  -h, --help
> > >     Print detailed help screen
> > >  -V, --version
> > >     Print version information
> > >  -H
> > >     specify a target
> > > 
> > > 
> > > Luckily we've got the source
> > > 
> > > 
> > > $ grep -i \$ID
> > >
/opt/sw/nagios/build/nagios-plugins-1.4.5/plugins-root/check_icmp
> > > .c 
> > > * $Id: check_icmp.c,v 1.8 2006/10/24 21:54:06 opensides Exp
$
> > > 
> > > 
> > > So is this recent enough?
> > > 
> > 
> > Well, that depends, really. If the patches I sent in (way 
> back) when 
> > this problem was first encountered (iirc someone had upgraded
from 
> > FreeBSD 4.x to 5.x and ran into this exact problem) were 
> > incorporated in 
> > the official release, then this should indeed be recent 
> enough. It's 
> > possible they didn't, or that I forgot to send that patch 
> > (although it 
> > was definitely circulated around the list a time or two) so 
> that the 
> > official and my version might differ.
> > 
> > Attached is the version we use. You should be able to compile
it 
> > stand-alone on any sane system with
> > 	$ gcc check_icmp.c -o check_icmp -O2
> > 
> > or by replacing check_icmp.c in your plugins directory with 
> this one.
> > 
> > The relevant pieces can be gleaned by
> > 	$ grep -B1 'pid & 0xffff' check_icmp.c
> > 
> > if anyone's interested.
> > 
> > Btw, I think this one still doesn't support "--version", 
> although it 
> > pretends to in the help output ;-)
> > 
> > -- 
> > Andreas Ericsson                   andreas.ericsson at op5.se
> > OP5 AB                             www.op5.se
> > Tel: +46 8-230225                  Fax: +46 8-230231
> > 
>