Login or Register to Ask a Question and Join Our Community


AIX

Misconfiguration detected Adapter interface name en 3 Adapter offset 0

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems AIX Misconfiguration detected Adapter interface name en 3 Adapter offset 0
# 1  
Old 06-06-2019
Misconfiguration detected Adapter interface name en 3 Adapter offset 0
Hi,
We had a hardware problem with an IBM System p5 server, with AIX 5.2
We restore from a tape the last backup we had, but the server does not boot up as expected.

The server try to mount some directories from a storage, but could not comunicate with them, we check the FC and everything is fine.
A second alert is a misconfiguration in an ethernet adapter.
After this alerts the server starts a loop trying to find the storage.

Im stuck with this problem, any clues ?

Thank you in advance .

Here is the log.

Moderator's Comments:
Mod Comment
Please wrap all code, files, input & output/errors in CODE tags.
It makes it far easier to read and preserves spaces for indentation or fixed-width data.

Code:
Welcome to AIX.
                       boot image timestamp: 22:18 05/29
                 The current time and date: 18:02:15 05/31/2019
                number of processors: 2    size of memory: 1904Mb
boot device: /pci@800000020000003/pci@2,4/pci1069,b166@1/scsi@0/sd@5:2
closing stdin and stdout...
-------------------------------------------------------------------------------

Saving Base Customize Data to boot disk
Starting the sync daemon
Mounting the platform dump file system, /var/adm/ras/platform
Starting the error daemon
System initialization completed.
Setting tunable parameters...complete
Starting Multi-user Initialization
 Performing auto-varyon of Volume Groups
 Activating all paging spaces
0517-075 swapon: Paging device /dev/hd6 is already active.
/dev/rhd1 (/users): ** Unmounted cleanly - Check suppressed
/dev/rhd10opt (/opt): ** Unmounted cleanly - Check suppressed
 Performing all automatic mounts
mount: 0506-324 Cannot mount /dev/fslv00 on /home: A file or directory in the path name does not exist.
Replaying log for /dev/fslv01.
mount: 0506-324 Cannot mount /dev/fslv03 on /home/monitor_logs: A file or directory in the path name does not exist.
mount: 0506-324 Cannot mount /dev/archdwdg_S_01 on /usr/hacmp_fs/archdw_dg: A file or directory in the path name does not exist.
mount: 0506-324 Cannot mount /dev/archdg_S_01 on /usr/hacmp_fs/arch_dg: A file or directory in the path name does not exist.
Multi-user initialization completed
nsmb0 Available
Checking for srcmstr active...complete
Starting tcpip daemons:
0513-059 The syslogd Subsystem has been started. Subsystem PID is 27108.
0513-059 The portmap Subsystem has been started. Subsystem PID is 27546.
0513-059 The inetd Subsystem has been started. Subsystem PID is 24010.
0513-059 The snmpd Subsystem has been started. Subsystem PID is 24672.
May 31 14:03:57 localhost snmpd[24672]: EXCEPTIONS: open_device: Unable to connect to device driver.
May 31 14:03:57 localhost last message repeated 2 times
0513-059 The dpid2 Subsystem has been started. Subsystem PID is 24938.
0513-059 The hostmibd Subsystem has been started. Subsystem PID is 26714.
0513-059 The aixmibd Subsystem has been started. Subsystem PID is 25218.
0513-059 The muxatmd Subsystem has been started. Subsystem PID is 27162.
Finished starting tcpip daemons.
Starting NFS services:
May 31 14:04:09 localhost syslog: /usr/sbin/ifconfig -l
0513-059 The biod Subsystem has been started. Subsystem PID is 25658.
0513-059 The nfsd Subsystem has been started. Subsystem PID is 29758.
0513-059 The rpc.mountd Subsystem has been started. Subsystem PID is 28002.
0513-059 The rpc.lockd Subsystem has been started. Subsystem PID is 22564.
May 31 14:04:13 localhost syslog: /usr/sbin/ifconfig -l
May 31 14:04:13 localhost automountd[25508]: svc_create: no well known address for autofs on transport udp
May 31 14:04:13 localhost syslog: dlopen(/usr/ldap/lib/libibmldapn.a) failed: A file or directory in the path name does not exist.
May 31 14:04:13 localhost syslog: WARNING: ldap is not loaded
May 31 14:04:13 localhost syslog: WARNING: ldap is not configured
May 31 14:04:13 localhost unix:
May 31 14:04:13 localhost unix:
May 31 14:04:13 localhost unix:
May 31 14:04:13 localhost unix:
Completed NFS services.
May 31 14:04:20 localhost no[30460]: Network option tcp_keepinit was set to the value 40
May 31 14:04:20 localhost no[30462]: Network option tcp_keepidle was set to the value 20
May 31 14:04:23 localhost no[30208]: Network option tcp_keepintvl was set to the value 15
May 31 14:04:26 localhost no[30210]: Network option tcp_sendspace was set to the value 262144
May 31 14:04:29 localhost no[30212]: Network option tcp_recvspace was set to the value 262144
May 31 14:04:32 localhost no[30214]: Network option udp_sendspace was set to the value 65536
May 31 14:04:35 localhost no[30216]: Network option udp_recvspace was set to the value 262144
May 31 14:04:38 localhost no[30218]: Network option rfc1323 was set to the value 1
May 31 14:04:41 localhost no[30220]: Network option udp_sendspace was set to the value 65536
May 31 14:04:41 localhost no[30222]: Network option udp_recvspace was set to the value 262144
May 31 14:04:42 localhost su: from root to imdba at /dev/tty??
0513-059 The clcomdES Subsystem has been started. Subsystem PID is 31510.
0513-059 The nmbd Subsystem has been started. Subsystem PID is 31274.
0513-059 The smbd Subsystem has been started. Subsystem PID is 30226.
May 31 14:04:42 localhost last message repeated 6 times
May 31 14:04:50 localhost no[33086]: Network option routerevalidate was set to the value 1
May 31 14:05:03 localhost no[33864]: Network option nonlocsrcroute was set to the value 1
May 31 14:05:03 localhost no[33866]: Network option ipsrcroutesend was set to the value 1
May 31 14:05:03 localhost no[33866]: Network option ipsrcrouterecv was set to the value 1
May 31 14:05:03 localhost no[33866]: Network option ipsrcrouteforward was set to the value 1
May 31 14:05:06 localhost topsvcs[24034]: (Recorded using libct_ffdc.a cv 2):::Error ID: 6OP0ZW0GnKwQ/fhG./...                                                                                              z/...................:::Reference ID: :::Template ID: a29426da::: Details File:  :::Location: rsct,nim_control.                                                                                             
C,1.39.1.2,4359              :::TS_MISCFG_ER Local adapter misconfiguration detected Adapter interface name en 3 Adapter offset 0 Adapter expected IP address 10.189.125.27
May 31 14:05:06 localhost topsvcs[24034]: (Recorded using libct_ffdc.a cv 2):::Error ID: 6OP0ZW0GnKwQ/p/I./...                                                                                              z/...................:::Reference ID: :::Template ID: a29426da::: Details File:  :::Location: rsct,nim_control.                                                                                              
C,1.39.1.2,4359              :::TS_MISCFG_ER Local adapter misconfiguration detected Adapter interface name en 1 Adapter offset 1 Adapter expected IP address 10.189.126.27
May 31 14:05:06 localhost topsvcs[24034]: (Recorded using libct_ffdc.a cv 2):::Error ID: 6FYVDG0GnKwQ/GlJ./...                                                                                              z/...................:::Reference ID: :::Template ID: 923e1911::: Details File:  :::Location: rsct,nim_control.                                                                                             
 C,1.39.1.2,4428              :::TS_NIM_OPEN_ERROR_ER Failed to open NIM connection Interface name rhdisk29 Description 1 SYSCALL Description 2 openx() Value 1 19 Value 2 0
May 31 14:05:19 localhost clstrmgrES[36390]: Fri May 31 14:05:19 HACMP/ES Cluster Manager Started
May 31 14:05:19 localhost clstrmgrES[36390]: Fri May 31 14:05:19 IpcInit: called
0513-029 The ctrmc Subsystem is already active.
Multiple instances are not supported.
May 31 14:05:53 localhost HACMP for AIX: EVENT START: node_up hd2z
May 31 14:05:55 localhost HACMP for AIX: EVENT FAILED: 1: node_up hd2z 1
May 31 14:05:55 localhost HACMP for AIX: EVENT START: event_error 1 TE_JOIN_NODE
WARNING: Cluster his_cluster Failed while running event [JOIN], exit status was 1
                                                                                 
May 31 14:05:55 localhost HAC  MP for AIX: EVENT FAILED: -1: event_error 1 TE_JOIN_NODE -1

Fri May 31 14:10:39 AST 2019
                   Automatic Error Log Analysis for sysplanar0 has detected a problem.
                  The Service Request Number is B7006970: I/O subsystem (hub, bridge, bus) Unrecovered Error, general. Refer to the system service documentation for more information.
                   Additional Words: 2-00000062 3-00010002 4-24030 230 5-00000000 6-00001140 7-00020000 8-00000000 9-00000000.

May 31 14:11:50 localhost HACMP for AIX: EVENT START: config_too_long 360 /usr/es/sbin/cluster/events/node_up.rp
WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 360 seconds. Please check cluster status.
WARNING: Cluster his_cluster has been running recovery program '/usr/es /sbin/cluster/events/node_up.rp' for 390 seconds. Please check cluster status.
WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 420 seconds. Please check cluster status.
WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp'  for 450 seconds. Please check cluster status.
 WARNING: Cluster his_cluster has been running recovery program ' /usr/es/sbin/cluster/events/node_up.rp' for 480 seconds. Please check cluster status.

 Fri May 31 14:13:54 AST 2  019
   Automatic Error Log Analysis for sysplanar0 has detected a problem.
  The Service Request Number is BA210000 : Platform Firmware Unrecovered Error, general. Refer to the system  service documentation for more information.
        Additional Words: 2-20202020 3-20202020 4-20202020 5-20202020 6-20202020 7-20202020 8-20202020 9-20202020.

  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 540 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 600 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 660 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 720 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 780 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 900 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 1020 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 1140 seconds. Please check cluster status.

Last edited by rbatte1; 06-06-2019 at 12:36 PM..
# 2  
Old 06-07-2019
Firstly, let me say that I'm not familiar with your specific hardware platform so this is a generic answer.

What kind of hardware problem did you have? Did it result in changing the motherboard (mobo)?

If so, did you put the add-on adapters into the same slots as on the old one?

Is there a BIOS or other configuration program that allows you to set the bus address and IRQs for those slots?

If so, and they're now set differently then that's probably your issue.
Last edited by hicksd8; 06-07-2019 at 07:50 AM..
# 3  
Old 06-10-2019
Thank you hicksd8 for your answer

Basically the main hard disk failed (with the OS), so we changed for a new one, same model and type, and restore the info from a backup.
The backup was supossed to work, because we change only the hard disk.
We dont know why this isnt working.
Thanks again
trev
# 4  
Old 06-10-2019
First off AIX 5.2 is NOT my cup of tea.
I'm limited to 6.1 and 7.1 so there might be some differences and I can't state all commands will work as expected on AIX 5.2.

Let's try to dive in to the first issue. Missing mount points:
Code:
 Performing all automatic mounts
mount: 0506-324 Cannot mount /dev/fslv00 on /home: A file or directory in the path name does not exist.
Replaying log for /dev/fslv01.
mount: 0506-324 Cannot mount /dev/fslv03 on /home/monitor_logs: A file or directory in the path name does not exist.
mount: 0506-324 Cannot mount /dev/archdwdg_S_01 on /usr/hacmp_fs/archdw_dg: A file or directory in the path name does not exist.
mount: 0506-324 Cannot mount /dev/archdg_S_01 on /usr/hacmp_fs/arch_dg: A file or directory in the path name does not exist.

So, you posted above that you had a disk failure. How many disks are attached to this system?
Looking above its unable to find some of your logical volumes/filesystems. Do you know where those underlying physical disks that supported those filesystems reside?

What product or how did you backup the system to tape? Are there any exclude statements?
Lets take a look at the output of the following commands:
lspv
lsvg

For each volume group defined in lsvg, lets do a lsvg <volume group name>, followed up by a lsvg -l <volume groupname>

lsfs

And lastly, lets see if there's an exclude file for your rootvg
Code:
cat /etc/exclude.rootvg

If all those happy logical volumes lived on SAN and everything is zoned correctly, there's a very good chance that they are there and fine, you just need to use importvg.

The adapter configuration errors appear to be related to the RSCT (HACMP) issues, and that might be related to your missing logical volumes.
Last edited by RecoveryOne; 06-10-2019 at 01:38 PM..
This User Gave Thanks to RecoveryOne For This Post:
hicksd8
# 5  
Old 06-10-2019
RecoveryOne, Hi, and thank you.

How many disks ? 4
The filesystems resides in a storage. This storage is connected to a fiber optic switch, and connected to the server through fiber optic patch cords.
What product used to backup ? mksysb (Dont know if the people that did the bakcup used exlude statements)

Right know, Im using putty and connected to the console port of the server, and working from there. I dont have image in the monitor.

After this part

Code:
WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 540 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 600 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 660 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 720 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 780 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 900 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 1020 seconds. Please check cluster status.
  WARNING: Cluster his_cluster has been running recovery program '/usr/es/sbin/cluster/events/node_up.rp' for 1140 seconds. Please check cluster status.

I dont know how to stop it and let the OS continue and do the lspv or lsvg commands

Thanks
Last edited by trevian3969; 06-10-2019 at 07:29 PM.. Reason: Some part is not clear
# 6  
Old 06-10-2019
Oh. Not good.
So, looks like TCPIP is up and running so you are pretty far along in the boot. A control-c/spam enter a few times might get you to a login prompt, if you time it between the messages printed to console.

If you do manage to get in, you will likely be spammed with all the happy fun AIX stuff it prints to console. I think in 5.3 TL7?? There's the swcons command, you could do a swcons /tmp/console.out to temporally send all messages printed to console to file. To revert it back just typing swcons.
If you have telnet/ssh it might be responsive on the network. Not sure however.
So, was this lpar part of a cluster?

Next lets go talk about single user mode!
Also, do you have any sort of HMC or is this a standalone frame?

If standalone, this should work (again 5.2 not my cup of tea). Lets bring this lpar up in single user mode. With your console still attached, bounce the lpar (nicely if hmc, or well...press the magical power button if you must). You should see the system start to IPL, and if you are sitting near it, it should beep on the frame (if so equipped) and beep/flash your console. Alternate, if you have a monitor/kbd hooked up it would likely flash/beep there so yeah.
Now, you should see an option of choosing another boot list during the time of the beeping/flashing. Should be boot selection. An option for 'Press 6 to start diagnostics'.
Normal AIX boot header of 'Welcome to AIX' will display, and you will be presented with a diagnostic screen which you can enter past. The next screen should have an option for Start Shell, or Single User mode.


IBM has some good docs about what can and cannot be done in that mode and how to exit single user and get into multiuser. Here's one now! IBM Booting AIX in Single-User Mode
From there, I can't say which commands will work and which wont. I only had to go into single user mode once for a really messed up inittab issue that a vendor left me with!


Again I could be way off the processes as I have 6.1 and 7.1 under my belt. The 5.3 systems and our lowly 4.3.3 box was retired as I came in.

Anyways, do you have other backups of this lpar?
Good news, it sounds like your filesystems are protected if they are on SAN, you just need to get to a point where you can bring them in.
Bad news, depending on if any excludes were done you may end up with a painful rebuild, if other data wasn't backed up via other means.

If you have the knowledge base of 5.2, I'd really look at setting up a mirrored rootvg once you are up and running. I've done both a local hdisk0 and a SAN provided hdisk1 as a mirror before with 6.1 TL7. Not sure if applicable with 5.2, I know a lot of things have changed.

Unfortunately a lot of information regarding 5.2 and even 5.3 is lost in the mix, so not to be 'you should look at updating guy', but....should look at updating Smilie A small Power9, or Power8, heck P7 lines will run rings around a p5. In fact, I'm pushing for that now, we have a few p5's still and they are dropping like flies. Lost two raid controllers (drives ok, but sissas0 AND sissas1 on same frame went belly up.) Another had sysplanar0 errors. And a 3rd I keep under my desk in the office as a heater in the winter months has been reporting predictive failure on the service processor for at least a year now if not longer.


Sorry trevian, I may not be much help in the long run.
Feel free to post what you can, and I'll try to assist when I can, but just all before my time. Perhaps one of the mods here who've worked with AIX for a really long time may have some other ideas?
# 7  
Old 06-11-2019
First off: AIX 5.2 is at least 10 years out of support, the support for its successor 5.3 ended 2012. The support for some Power5 systems (if they have been upgraded with Power6- and Power6plus-processors) also ended beginning this year, even the Power7-support will end in September.

So, whatever you do, you should URGENTLY consider updating to anything recent.

You said the LUNs you are missing come from a FC-connected storage: did you make sure the zoning is correct? If you restored the system from an mksysb image then chances are you use another FC-adapter with a different WWPN and hence you need to modify the zones accordingly. You *do* know what "zoning" means, yes? Everything else is, i suppose, just coming from the not-found disks.

By the way: you should always give your LVs, VGs and FSes meaningful names: names like /dev/fslv00 are not meaningful and will over time contribute to the overall confusion.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
hicksd8
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Ethernet Adapter not detected.

Hi, I have installed Oracle Enterprise Linux 5 in my PC. After the installation the Ethernet Card is not got detected. But I could able to get the details using the command lspci -a: 01:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit Ethernet Adapter (rev b0). But I could... (4 Replies)
Discussion started by: chidhu.anu
4 Replies

2. AIX

Vscsi and npiv on same adapter

Hi, I want to change from vscsi to npiv. Is it possible to use both on the same adapter, so we can change the systems one by one, or must we place a second FC adapter in the VIO servers? Thanks, Ronald (2 Replies)
Discussion started by: ronaldm60
2 Replies

3. AIX

Dedicate graphics adapter ?

Hello, Runing VIO and couple lpars on POWER blade. I want to dedicate graphics adapter(ati0) to one lpar. Is that possible ? I tried to do that using IVM (I/O Adapter Management->View/Modify Physical Adapters->Modify partition assignment): Failed to remove adapter U78A6.001.WIH4088-P2-C8... (1 Reply)
Discussion started by: vilius
1 Replies

4. Hardware

SCSI Ultra320 80->68 adapter

I need to connect 80pin SCA Ultra320 SCSI hdd to 68pin. There are rumors that some bad "passive" adapters with problematic termination can fry the HDD. The adapter we got is Assmann ABSCA-2. Hard drive is 'Fujitsu MAW3073NC' Do we need to set any jumpers before turning it on? We already had... (0 Replies)
Discussion started by: orange47
0 Replies

5. SCO

Installing LAN adapter

I inserted a PCI Realtek RTL8139 network adapter on a PC running SCO Openserver 5.0.6. I successfully loaded drivers and I'm trying to install the card with netconfig command, unfortunately without success. Here are the steps I follow to install the card: I run netconfig, from menu 'hardware'... (2 Replies)
Discussion started by: kampus
2 Replies

6. UNIX for Dummies Questions & Answers

Simulated driver for Network Interface Adapter

Hi all, I got sort of a task to do. I have to write in C "simulated network driver". What does it mean? - It has to run on all network adapters - It has to be as simple as it can be - It has to run on all linux distributions (or at least most of them) - It does not have to run a network... (4 Replies)
Discussion started by: Chrisdot
4 Replies

7. HP-UX

what is an adapter?

Hi , Hi what is an adapter?what is its use ? How to log into a tibco server from unix home directory. (1 Reply)
Discussion started by: megh
1 Replies

8. Solaris

Creator 3D Video Adapter

Hi everyone, it's me the Solaris novice once again. Here's my situation. - I bought a Sun Ultra 10 Creator 3D Workstation and Sony 19" SVGA Monitor package deal. - The system has the Creator 3D Video Adapter installed but I could not use it because the adapter has a 13w3 connection and my... (5 Replies)
Discussion started by: jbarbuto
5 Replies

9. Solaris

Which SCSI Adapter?

Hi. I need to find a simple SCSI adapter card for a Sun Ultra 5 workstation. I don't need the type w/ ethernet attached (e.g. X1032) as I already have ethernet onboard & they are more spendy. I know Adaptec is out, but what unit would work? I am attaching an external SCSI tape unit to this... (4 Replies)
Discussion started by: Plain Person
4 Replies

10. IP Networking

LAN Adapter

Im newest in unix, Please, give me solution for installing lan adapter if driver not found in the list of ethernet drivers ihave unix driver in a dos diskete please give me the other solution and tell me where the domain of unix class online, so ican read from it. Thank's very much (3 Replies)
Discussion started by: ade_muhdiat
3 Replies
Login or Register to Ask a Question