NAGIOS Service not able to open port on AIX 7.1

Tags
aix 7.1, nagios, port

 
Thread Tools Search this Thread
# 1  
Old 09-04-2018
NAGIOS Service not able to open port on AIX 7.1

I have an AIX 7.1 LPAR where Nagios agent was installed for monitoring. The issue is that when I start the nagios service (ncpa_listener), it starts but does not open the 5693 port it requires for communication. On all other LPARs the service opens the port and is listening. I tried reinstalling the agent. Sometimes, when I try to stop the service it goes into a stopping state and has to be killed forcefully. The oslevel is 7100-05-01-1731. The LPAR has about 120GB of memory and 6 CPUs as entitled capacity. This is an Oracle DB DR server. Please let me know if more details are required.

I tried to see if there is anything using the port using netstat -an | grep 5693. I also checked the services file to see if any service is already using this port, but there was nothing in both cases.

------ Post updated at 12:22 PM ------

I just wanted to add one more thing. If I do a "telnet localhost 5693", I don't get a telnet prompt. But on all other LPARs, the telnet connects and gives a prompt. I can't seem to think of a reason why the service should not be able to open an unused port.
# 2  
Old 09-05-2018
What do your Nagios agent startup logs say?


Maybe you can post the recent output of this logfile:

Code:
/usr/local/nagios/var/nagios.log

# 3  
Old 09-07-2018
Below are the logs. I don't see anything which can help me with the port issue.

Code:
/usr/local/ncpa/var/log#tail -50 ncpa_listener.log
2018-07-29 19:35:38,563 62456054 INFO 172.23.103.81 - - [2018-07-29 19:35:38] "GET /api/disk/logical/%7CARCH_MYMONEY_DR/used_percent/?token=XhRJ5
PTsL&warning=70&critical=90&check=1 HTTP/1.1" 200 242 0.082072
2018-07-29 19:35:41,910 62456054 INFO 172.23.103.81 - - [2018-07-29 19:35:41] "GET /api/disk/logical/%7CARCH_REPORT_DR/used_percent/?token=XhRJ5S
TsL&warning=70&critical=90&check=1 HTTP/1.1" 200 242 0.050659
2018-07-29 19:35:44,978 62456054 INFO 172.23.103.81 - - [2018-07-29 19:35:44] "GET /api/disk/logical/%7Cor11_6/used_percent/?token=XhRJ5ScGPTsL&w
ing=70&critical=90&check=1 HTTP/1.1" 200 242 0.115249
2018-07-29 19:35:51,887 62456054 INFO 172.23.103.81 - - [2018-07-29 19:35:51] "GET /api/disk/logical/%7CARCH_TPPPROD_DR/used_percent/?token=XhRJ5
PTsL&warning=70&critical=90&check=1 HTTP/1.1" 200 242 0.056481
2018-07-29 19:36:02,089 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:02] "GET /api/disk/logical/%7CCTM/used_percent/?token=XhRJ5ScGPTsL&warn
=70&critical=90&check=1 HTTP/1.1" 200 239 0.051827
2018-07-29 19:36:10,775 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:10] "GET /api/disk/logical/%7Cdbfs/used_percent/?token=XhRJ5ScGPTsL&war
g=70&critical=90&check=1 HTTP/1.1" 200 242 0.186102
2018-07-29 19:36:11,828 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:11] "GET /api/disk/logical/%7CCDCSOFTMMMA/used_percent/?token=XhRJ5ScGP
&warning=70&critical=90&check=1 HTTP/1.1" 200 242 0.112948
2018-07-29 19:36:21,192 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:21] "GET /api/disk/logical/%7CDBScripts/used_percent/?token=XhRJ5ScGPTs
arning=70&critical=90&check=1 HTTP/1.1" 200 242 0.074064
2018-07-29 19:36:27,303 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:27] "GET /api/disk/logical/%7Cvar/used_percent/?token=XhRJ5ScGPTsL&warn
=70&critical=90&check=1 HTTP/1.1" 200 247 0.045942
2018-07-29 19:36:30,496 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:30] "GET /api/disk/logical/%7Cexport/used_percent/?token=XhRJ5ScGPTsL&w
ing=70&critical=90&check=1 HTTP/1.1" 200 242 0.172986
2018-07-29 19:36:40,537 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:40] "GET /api/disk/logical/%7Chome/used_percent/?token=XhRJ5ScGPTsL&war
g=70&critical=90&check=1 HTTP/1.1" 200 242 0.155030
2018-07-29 19:36:50,784 62456054 INFO 172.23.103.81 - - [2018-07-29 19:36:50] "GET /api/disk/logical/%7Copt/used_percent/?token=XhRJ5ScGPTsL&warn
=70&critical=90&check=1 HTTP/1.1" 200 242 0.124295
2018-07-29 19:37:01,410 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:01] "GET /api/disk/logical/%7Cadmin/used_percent/?token=XhRJ5ScGPTsL&wa
ng=70&critical=90&check=1 HTTP/1.1" 200 239 0.056164
2018-07-29 19:37:10,224 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:10] "GET /api/disk/logical/%7Cusr/used_percent/?token=XhRJ5ScGPTsL&warn
=70&critical=90&check=1 HTTP/1.1" 200 242 0.054711
2018-07-29 19:37:20,011 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:20] "GET /api/disk/logical/%7Ctmp/used_percent/?token=XhRJ5ScGPTsL&warn
=70&critical=90&check=1 HTTP/1.1" 200 242 0.051173
2018-07-29 19:37:26,371 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:26] "GET /api/disk/logical/%7Cvar/used_percent/?token=XhRJ5ScGPTsL&warn
=70&critical=90&check=1 HTTP/1.1" 200 247 0.071834
2018-07-29 19:37:28,804 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:28] "GET /api/disk/logical/%7C/used_percent/?token=XhRJ5ScGPTsL&warning
&critical=90&check=1 HTTP/1.1" 200 242 0.288115
2018-07-29 19:37:46,452 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:46] "GET /api/disk/logical/%7Cproc/used_percent/?token=XhRJ5ScGPTsL&war
g=70&critical=90&check=1 HTTP/1.1" 200 250 0.043516
2018-07-29 19:37:50,129 62456054 INFO 172.23.103.81 - - [2018-07-29 19:37:50] "GET /api/disk/logical/%7Cibmperf/used_percent/?token=XhRJ5ScGPTsL&
ning=70&critical=90&check=1 HTTP/1.1" 200 242 0.162522
2018-07-29 19:38:05,755 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:05] "GET /api/disk/logical/%7Cor11_163/used_percent/?token=XhRJ5ScGPTsL
rning=70&critical=90&check=1 HTTP/1.1" 200 242 0.123206
2018-07-29 19:38:22,439 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:22] "GET /api/disk/logical/%7Cor11_5/used_percent/?token=XhRJ5ScGPTsL&w
ing=70&critical=90&check=1 HTTP/1.1" 200 242 0.042097
2018-07-29 19:38:31,503 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:31] "GET /api/disk/logical/%7Corasr5/used_percent/?token=XhRJ5ScGPTsL&w
ing=70&critical=90&check=1 HTTP/1.1" 200 242 0.047661
2018-07-29 19:38:38,962 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:38] "GET /api/disk/logical/%7CARCH_STAGCDC_DR/used_percent/?token=XhRJ5
PTsL&warning=70&critical=90&check=1 HTTP/1.1" 200 239 0.066742
2018-07-29 19:38:42,470 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:42] "GET /api/disk/logical/%7CCDCSOFTSTAG/used_percent/?token=XhRJ5ScGP
&warning=70&critical=90&check=1 HTTP/1.1" 200 247 0.068118
2018-07-29 19:38:57,421 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:57] "GET /api/memory/virtual/?token=XhRJ5ScGPTsL&warning=85&critical=90
its=Gi&check=1 HTTP/1.1" 200 387 0.079366
2018-07-29 19:38:57,932 62456054 INFO 172.23.103.81 - - [2018-07-29 19:38:57] "GET /api/cpu/percent/?token=XhRJ5ScGPTsL&warning=85&critical=90&ch
=1&aggregate=avg HTTP/1.1" 200 229 0.676725
2018-07-29 19:39:07,876 62456054 INFO 172.23.103.81 - - [2018-07-29 19:39:07] "GET /api/disk/logical/%7CCDCDATAMMMA/used_percent/?token=XhRJ5ScGP
&warning=70&critical=90&check=1 HTTP/1.1" 200 239 0.084986
2018-07-29 19:39:08,859 62456054 INFO 172.23.103.81 - - [2018-07-29 19:39:08] "GET /api/memory/swap/?token=XhRJ5ScGPTsL&warning=30&critical=40&un
=Gi&check=1 HTTP/1.1" 200 330 0.082764
2018-07-29 19:39:12,904 62456054 INFO 172.23.103.81 - - [2018-07-29 19:39:12] "GET /api/disk/logical/%7Cvar%7Cadm%7Cras%7Clivedump/used_percent/?
en=XhRJ5ScGPTsL&warning=70&critical=90&check=1 HTTP/1.1" 200 239 0.048158
2018-07-29 19:39:17,578 62456054 INFO 172.23.103.81 - - [2018-07-29 19:39:17] "GET /api/disk/logical/%7CCDCDATASTAG/used_percent/?token=XhRJ5ScGP
&warning=70&critical=90&check=1 HTTP/1.1" 200 239 0.150568
2018-07-29 19:39:34,103 62456054 INFO 172.23.103.81 - - [2018-07-29 19:39:34] "GET /api/disk/logical/%7CCDCDATATPPPROD/used_percent/?token=XhRJ5S
TsL&warning=70&critical=90&check=1 HTTP/1.1" 200 239 0.098944
2018-07-29 19:39:56,727 62456054 INFO 172.23.103.81 - - [2018-07-29 19:39:56] "GET /api/disk/logical/%7CCDCSOFTTPPPROD/used_percent/?token=XhRJ5S
TsL&warning=70&critical=90&check=1 HTTP/1.1" 200 242 0.114090
2018-08-09 22:11:06,089 47448130 INFO started
2018-08-09 22:14:39,071 47448138 INFO started
2018-08-09 22:52:45,887 47448138 INFO stopped
2018-08-09 22:59:59,896 15270034 INFO started
2018-08-16 14:46:18,123 15270034 INFO stopped
2018-08-16 15:36:39,075 49414344 INFO started
2018-08-16 15:36:54,761 49414344 INFO stopped
2018-08-16 15:38:47,656 46727418 INFO started
2018-08-16 15:40:06,202 51904760 INFO started
2018-08-16 15:40:47,708 46727212 INFO started
2018-08-16 15:42:48,639 44761154 INFO started
2018-08-16 15:47:01,135 62980292 INFO started
2018-08-16 15:53:15,527 58458112 INFO started
2018-08-16 15:54:18,140 58458112 INFO stopped
2018-08-20 22:24:18,533 58458136 INFO started
2018-08-28 22:13:33,386 58458136 INFO stopped
2018-08-31 07:47:07,234 2490720 INFO started

This User Gave Thanks to wibhore For This Post:
Neo (09-08-2018)
# 4  
Old 09-08-2018
Maybe you need to change the logging level to get more information?

You should be able to get a more verbose logging level and then you can attempt to connect to the port and see the errors.

There is always a log file somewhere that can help you sort this out.

You just need to find the right log and the right logging level.
# 5  
Old 09-09-2018
Some other debugging tips:
If the telnet terminates with "connection refused" then there is no listener.
The listener and a connection to it is shown with netstat.
You can start the service daemon with truss -f to see what it does.

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Ping the service with out port number userexperience UNIX for Beginners Questions & Answers 1 04-24-2017 09:37 AM
Nagios for AIX 7.1 bsivavani AIX 2 10-15-2015 01:12 AM
Nagios binaries for AIX 7.1 bsivavani Infrastructure Monitoring 1 10-14-2015 05:10 AM
Cabling and adapters to communicate to service processor serial port from Windows PC with USB port. joboy Solaris 5 02-23-2015 11:08 AM
Tcp ip port open but no such process (merged: Release A Port) alinamadchian IP Networking 30 03-05-2013 02:56 PM
Nagios on AIX asyed123 AIX 2 07-29-2012 03:18 PM
Using Different port for Daytime Service manisum Programming 2 02-25-2012 08:09 PM
How to open port 1001 on AIX 6.1 arm_naja AIX 5 10-05-2011 11:15 PM
Parsing Nagios service config files with awk vinbob Shell Programming and Scripting 2 12-21-2010 07:39 AM
Need to run Nagios Web Interface on a different port jacki Web Programming 1 12-18-2010 09:26 PM
Using SMF to register & start a (Nagios) service lyle Infrastructure Monitoring 0 09-21-2010 02:07 PM
Finding service based on port number manoj.solaris AIX 2 01-11-2010 10:38 AM
list service with port number sharkux Solaris 5 09-24-2009 01:12 AM
Unknown open port: "6881/tcp open bittorrent-tracker" found with nmap Rledley IP Networking 0 03-19-2009 05:57 PM
How to open a port in AIX sanju_d1231 AIX 2 05-07-2008 05:38 AM