Solaris

Hi does anyone got idea on this x4600 machine error?

WARNING: rpcmod:svc_default_stksize is set more than once in /etc/system. "set r
pcmod:svc_default_stksize = 0x6000" applied as the current setting.

SunOS Release 5.10 Version Generic_125101-02 64-bit
Copyright 1983-2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
WARNING: md: d20: (Unavailable) needs maintenance
WARNING: Last shutdown is later than time on time-of-day chip; check date.
WARNING: BIOS microcode patch for AMD Athlon(tm) 64/Opteron(tm) processor
erratum 131 was not detected; updating your system's BIOS to a version
containing this microcode patch is HIGHLY recommended or erroneous system
operation may occur.

Hostname: xxxxxxxx
NOTICE: pciex8086,105e - e1000g[2] : Adapter 1000Mbps full duplex copper link is
up.
NOTICE: pci8086,1011 - e1000g[5] : Adapter 1000Mbps full duplex copper link is u
p.
NOTICE: VxVM vxdmp V-5-0-34 added disk array DISKS, datype = Disk

WARNING: md: d21: (Unavailable) needs maintenance
WARNING: md: d23: (Unavailable) needs maintenance
NIS domain name is XXXXXXXXXX
Sat Jul 19 22:19:16.4974 Disabling keephot for cache passwd since ttl is
less than 600 seconds
Sat Jul 19 22:19:16.4975 Disabling keephot for cache group since ttl is l
ess than 600 seconds
Sat Jul 19 22:19:16.4975 Disabling keephot for cache hosts since ttl is l
ess than 600 seconds
Sat Jul 19 22:19:16.4976 Disabling keephot for cache ipnodes since ttl is
less than 600 seconds
LLT INFO V-14-1-10009 LLT Protocol available
GAB INFO V-15-1-20021 GAB available
WARNING: md: d25: (Unavailable) needs maintenance
WARNING: md: d26: (Unavailable) needs maintenance
WARNING: md: d24: (Unavailable) needs maintenance
checking ufs filesystems
/dev/md/rdsk/d4: is logging.
/dev/md/rdsk/d6: is logging.
/dev/md/rdsk/d5: is logging.
checking vxfs filesystems
Running parallel replay fsck ...
/dev/vx/rdsk/ovsgxdg/lvxollogo:log replay in progress
/dev/vx/rdsk/ovsgxdg/lvxoladm:log replay in progress
/dev/vx/rdsk/ovsgxdg/lvxollogo:replay complete - marking super-block as CLEAN
/dev/vx/rdsk/ovsgxdg/lvxoladm:replay complete - marking super-block as CLEAN
/dev/vx/rdsk/ovsgxdg/lvovsgx:log replay in progress
/dev/vx/rdsk/ovsgxdg/lvxolxolo:log replay in progress
/dev/vx/rdsk/ovsgxdg/lvovsgx:replay complete - marking super-block as CLEAN
/dev/vx/rdsk/ovsgxdg/lvxolxolo:replay complete - marking super-block as CLEAN
Jul 19 22:20:06 in.mpathd[284]: NIC failure detected on e1000g2 of group xola_ne
t

Access to this computer is prohibited unless authorised
Accessing programs or data unrelated to your job is prohibited

xxxxxx console login: root

Enter Kerberos password for root/xxxxxx.com:

so you can't log in as root with kerberos?

regardless, can you get in with another username?
if so, check /etc/default/login.

look for this

Code:

# If CONSOLE is set, root can only login on that device.
# Comment this line out to allow remote login by root.
#
CONSOLE=/dev/console

Its not the issue. kerberos is another layer of problem. the mirrored disk synchronization caused the passwd and shadow files to get corrupted and caused inconsistency in the filesystem. This server is a GeoCluster with zones in it. Its an urgent relo... and Im up for almost 24hrs.. shagged.. still resolving network issues and other complications..

That looks nasty...

The only thing that I can suggest here is that you boot from media, mount the root disk and edit /etc/vfstab and /etc/system so that you boot from the slice instead of the md. While you're there you can also put together an /etc/passwd and /etc/shadow and nsswitch / pam configuration that will work to get you logged in. Once rebooted and logged in you should be able to cleanup the meta devices and remirror / metaroot the system if it is recoverable.

hi guys,
there were a total of 5 cluster pairs.. each had it own issues.
one of the pairs showed similar symptoms. the culprit was /etc/inet/ipnodes entry which I realised much later. In Solaris 10, im shocked that this file is configured..
The kerberos authentication via the console and remoted ssh failed cos of this.
We had configured IPMP on the public LAN. I noticed that the virtual IP was pointing to the old IP (previous location) and was not changed.
somehow or rather, this had caused the passwd/shadow files to get corrupted and the system will dis-allow you to login as root even after you amend the changes in the shadow file. after the reboot, the editted shadow file will disappear.. i was not able to login as root at all.If issued root id at the login and hit enter (even before typing the password/empty password), the system panics..I was not even able to ping the gateway. When i unplumbed and re-plumbed the IP, it worked. After reboot, the ping to gateway will fail and ssh will not come into action. The main culprit caused the delay was this ipnode file that we overlooked and was unexpected...phwoooooo....its a pain... but I quite really dont understand the linkage between the kerberos, console root login , network issues and how they come into play together..SUN x86 is sucks.. If you've knowledge and idea of what this is all about, pls contribute. Though the issue is solved, I would like to know the "extremes" of kerberos and ipnodes stuff.. Thx

And yes, before all this happened, I had to detach/pull out the root mirror disk... followed by the above troubleshooting tasks mentioned..

I don't see anything here which looks to be x86 specific.

The solaris behaviour with respect to ipnodes is a little odd prior to update 4. IPv4 addresses are looked up in ipnodes before hosts, so if there is an entry in there then you really need to be sure it is correct.

Name resolution order if you had the following in nsswitch.conf

Code:

ipnodes:    files dns
hosts:      files dns

Would be: ipnodes, then dns, then hosts, then dns.

On my server I use this for nsswitch.conf:

Code:

ipnodes:    files
hosts:      files dns

And only keep the localhost in the ipnodes file.

After update 4 ipnodes is a symlink to hosts, but if the symlink is replaced by a file you will still see the old behavior.

In relation to the kerberos, what was the passwd & shadow order in nsswitch.conf? There is a possibility that VCS was causing the panic if it could not locate the primary naming service.

So far I can't see any reason for the edits to disappear on the root disk for the passwd and shadow.

Also you mentioned that this is a GeoCluster. Just to clarify, is this a stretch cluster or a true geo cluster with VVR replication to the DR site, and if so is this the primary or secondary site?