I agree with vbe, we should move this thread to the AIX forum. You need an AIX expert.

Never heared from SSH Tectia. Just was at their site...
Maybe it is no big help, but why not use the ssh/sshd that comes with AIX 5.3 for free???

Else you might check if the sshd_config or if something similar exists with that kind of sshd contains something awkward compared to those of the boxes that work.
Does this Tectia sshd write some log? Something in your errpt?

Just a question: do you have checked /dev/random? The availability of it is key to secure-layer protocols. It would explain the sshd running wild.

I hope this helps.

bakunin

Well Tectia SSH was allready in use when I arrived but I believe it offers a higher security level. I've also checked a couple of config files such as sshd2_config and compared them with other boxes, however everything seems to be correct, including the permissions.

I've also checked the /dev/random and also there the permissions are correct. I've also checked the .profile of the root user on the problemserver and nothing seems wrong with it, but still root cannot logon to that server. It's like he doesn't have a shell to work on it. The strange thing is that if I try the ssh command followed by another command such as 'ls', it works... But login into the server is impossible.
I can however use the root user on the server, but first I have to logon to the server with another user and afterwards login in as root.

When I do ssh -d 5 server1 (server1 is the problemserver) this is a part of the output:
16/10/2008 09:07:22:744 SecShBrokerCom/secshbrokercom.c:608: Failed to connect to broker socket `/tmp/ssh-root/ssh-broker'.
16/10/2008 09:07:22:744 SecShBrokerCom/secshbrokercom.c:710: Error in creating connection to broker.
16/10/2008 09:07:22:744 SecShBrokerCom/secshbrokercom.c:1065: Shutting down, status 5.
16/10/2008 09:07:22:744 SshSecShBroker/secsh_broker.c:1927: com_create status: error: 2, com err: 2.
16/10/2008 09:07:22:744 SshSecShBroker/secsh_broker.c:1933: Broker is not running.
16/10/2008 09:07:22:745 SecShBrokerCom/secshbrokercom.c:608: Failed to connect to broker socket `/tmp/ssh-root/ssh-broker'.
16/10/2008 09:07:22:745 SecShBrokerCom/secshbrokercom.c:710: Error in creating connection to broker.
16/10/2008 09:07:22:745 SecShBrokerCom/secshbrokercom.c:1065: Shutting down, status 5.
16/10/2008 09:07:22:745 SshSecShBroker/secsh_broker.c:1843: com_create status: error: 2, com err: 2.
16/10/2008 09:07:22:745 SshSecShBroker/secsh_broker.c:1693: Starting broker.
16/10/2008 09:07:22:745 SecShUserProcess/secsh_user_process_unix.c:1026: Executing command `/opt/tectia/libexec/ssh-broker-cli -D "5" --slave --run-on-demand --check-accession --no-gui': process 20125288 (params: allocate-pty: FALSE, support-handle-passing: TRUE, dont-inherit-handles: FALSE, force-hide-application: FALSE, chroot=(null), ulimit=(null), umask=(null), no-path-expand=TRUE, use-sigterm-instead-of-sigint=TRUE, let-live=TRUE, new-pgrp=TRUE, exec-directly=TRUE,use-shell-shell-exe=FALSE).
16/10/2008 09:07:22:745 SecShUserProcess/secsh_user_process_unix.c:1049: No environment given -> passing parent process environment.
16/10/2008 09:07:22:745 SecShUserProcess/secsh_user_process_unix.c:433: argv[0] = /opt/tectia/libexec/ssh-broker-cli.
16/10/2008 09:07:22:745 SecShUserProcess/secsh_user_process_unix.c:433: argv[1] = -D.
16/10/2008 09:07:22:746 SecShUserProcess/secsh_user_process_unix.c:433: argv[2] = 5.
16/10/2008 09:07:22:746 SecShUserProcess/secsh_user_process_unix.c:433: argv[3] = --slave.
16/10/2008 09:07:22:746 SecShUserProcess/secsh_user_process_unix.c:433: argv[4] = --run-on-demand.
16/10/2008 09:07:22:746 SecShUserProcess/secsh_user_process_unix.c:433: argv[5] = --check-accession.
16/10/2008 09:07:22:746 SecShUserProcess/secsh_user_process_unix.c:433: argv[6] = --no-gui.
debug: 16/10/2008 09:07:22:764 SshNioDispatcher/sshnio_dispatcher_unix.c:1363: Creating 4 threads.
16/10/2008 09:07:22:765 SecShBrokerCom/secshbrokercom.c:661: Verifying broker saneness.debug: 16/10/2008 09:07:22:766 Broker/broker.c:3501: Broker address: /tmp/ssh-root/ssh-broker
debug: 16/10/2008 09:07:22:766 SecshUserFiles/secsh_user_files.c:227: real path: root
debug: LOG EVENT (discard,notice): 6100 Broker_starting, Local username: root

debug: 16/10/2008 09:07:22:810 SshEKSoft/softprovider.c:4269: softkey; init-string use_proxy(), directory(path(//.ssh2/)) passphrase_timeout(0) passphrase_idle_timeout(0)
debug: 16/10/2008 09:07:22:811 SshEKSoft/softprovider.c:2850: Hard passphrase timeout 0 seconds.
debug: 16/10/2008 09:07:22:811 SshEKSoft/softprovider.c:2868: Idle passphrase timeout 0 seconds.
debug: 16/10/2008 09:07:22:811 SecShKeyStore/secsh_keystore.c:1619: Provider software://0/ added.
debug: 16/10/2008 09:07:22:815 SshUserFiles/sshkeyblob2.c:391: Failed to match header.
debug: 16/10/2008 09:07:22:815 SshUserFiles/sshkeyblob2.c:391: Failed to match header.
debug: 16/10/2008 09:07:22:815 SshPKB/openssh2pubkey.c:135: SSH1 public key decode failed: Key format was corrupted.
debug: 16/10/2008 09:07:22:815 SecShKeyStore/secsh_keystore.c:574: Waiting for provider software://0/ to scan all keys..
debug: 16/10/2008 09:07:22:815 SshUserFiles/sshkeyblob2.c:391: Failed to match header.
debug: 16/10/2008 09:07:22:815 SshPKB/openssh2pubkey.c:135: SSH1 public key decode failed: Key format was corrupted.

Thank you!

Could you please post the output of
# lsuser root
from the so called problemserver.

This is the output of lsuser root.

root id=0 pgrp=system groups=system,bin,sys,security,cron,audit,lp,exploit home=/ shell=/usr/bin/ksh auditclasses=general login=true su=true rlogin=true daemon=true admin=true sugroups=ALL admgroups=<all groups> tpath=nosak ttys=ALL expires=0 auth1=SYSTEM auth2=NONE umask=22 registry=files SYSTEM=compat logintimes= loginretries=0 pwdwarntime=0 account_locked=false minage=0 maxage=0 maxexpired=-1 minalpha=0 minother=0 mindiff=0 maxrepeats=8 minlen=0 histexpire=0 histsize=0 pwdchecks= dictionlist= fsize=-1 cpu=-1 data=-1 stack=-1 core=4194302 rss=65536 nofiles=20000 fsize_hard=-1 cpu_hard=-1 data_hard=-1 stack_hard=-1 time_last_login=1223971121 time_last_unsuccessful_login=1224082704 tty_last_login=/dev/pts/11 tty_last_unsuccessful_login=/dev/pts/0 host_last_login=<all servers> host_last_unsuccessful_login=server2 unsuccessful_login_count=13 roles=


The 'host_last_login' is kind of strange though because every server is listed, normally there is only 1 or so? It's like every server tried to connect at the same time, not certain though if it has anything to do with the problem.

To cut a long story short: if you did not change anything with root environment settings after Tue Oct 14 09:58:41 2008 (your localtime) there is nothing wrong with this user or the operating system. I'd suggest you uninstall that ssh software completely and install from scratch.
Somewhere up in the thread you mention that that Tectia ssh software is being used because it was somewhat more secure than OpenSSH. If nobody at your datacenter can name exactly where this additional security compared to the current version of OpenSSH is you might consider using OpenSSH instead. Reason is that you will get faster and better response to questions related to a product that is widely used compared to some niche product.