AIX

Good evening,

Thanks a lot for your help, we appreciate it a bunch. Heres basically a summary of what we have (sorry Im adding some extra history as well):

We have 2 Ibm Servers (Hag and Aki) that are running AIX Version 5. Each connect to a SAN device via 2 separate fiber paths. Hag seems to have problems running varyonvg for 3 of the 5 logical volumes, while Aki can varyonvg all of them without a problem. My goal is to get Hag to be able to varyonvg all of the 5 logical volumes. Here is the history:

- Initial Problem (Battery Replacement - Resolved!)
Initially, one of our software GUIs displayed that a battery needed to be replaced in the Fiber adapter on the SAN device. I shutdown the entire system and safely replaced the battery according to the manuals. Upon restarting the system, the LED was green for that device. Upon booting up the servers (Hag and Aki), I noticed...

- Second Problem (Fiber Controller Card Bad, Replaced - Resolved?)
The back of the Hag chassis, has a 2 PCI cards (Fiber Adapters). One of them was bad. I shutdown the machine and replaced the fiber card with a spare. Upon booting it up, the LEDs turned Green like the other Fiber card, so Im assuming that it is connected again. I also used the manual for replacing it (i.e. running `cfgmgr` after replacement). Upon mounting the varyonvg volume groups, I found the current problem...

The SAN has 2 controller cards (controller_A [dac0], controller_B [dac1]) that are used when trying to mount these partitions.

Hag can map ALL virtual partitions if they are mapped through only controller_A (controller_B is disabled). However upon enabling controller_B, some partitions are moved by ADT only controller_A's partitions can be mapped.

Smitty is a centralized administrative application for running all sorts scripts to turn on and off fiber (HBA) devices, hard disks, LVM, and so forth. Do you have any ideas on how I may be able to perform additional tests to switch the mapping? I want to run a command that allows me to test the connectivity to Hag's dac1 (Disk Array Controller).

Here are the outputs of some commands on Hag:
#lsdev -C | grep fcs
fcs0 Available 1j-08 FC Adapter
fcs1 Available 14-08 FC Adapter ***
*** This FC Adapter is the one associated with the fiber connection that was down. (I.e., when I ran this command before replacing the Fiber Adapter on Hag, it was "Defined" instead of "Available").

#lsdev -C | grep dac
dac0 Available 1j-08-01 1722-600 Disk Array Controller
dac1 Available 14-08-01 1722-600 Disk Array Controller

#lsdev -C | grep dar
dar0 Available 1722-600 (600) Disk Array Router

#fget_config -A
--dar0--
dac0 ACTIVE dac1 ACTIVE
hdisk2 dac0
hdisk3 dac0
hdisk4 dac0
--dar1--

*** However on Aki, the results are different since dac1 is used and load distributed (ADT)
#fget_config -A
--dar0--
dac0 ACTIVE dac1 ACTIVE
hdisk2 dac1
hdisk3 dac0
hdisk4 dac1
hdisk5 dac1
hdisk6 dac0
hdisk7 dac1

*** Here are all of the physical volumes:
#lspv #FROM HAG MACHINE (All available vg's are varied on and active).
hdisk0 xxxxxxxxxxxxxxxx rootvg active
hdisk1 xxxxxxxxxxxxxxxx db01vg active
hdisk2 xxxxxxxxxxxxxxxx vg00 active
hdisk3 xxxxxxxxxxxxxxxx vg02 active
hdisk4 xxxxxxxxxxxxxxxx vg03 active
hdisk5 xxxxxxxxxxxxxxxx vg05 active

#lspv #FROM AKI MACHINE
hdisk0 xxxxxxxxxxxxxxxx rootvg active
hdisk1 xxxxxxxxxxxxxxxx None
hdisk2 xxxxxxxxxxxxxxxx vg00
hdisk3 xxxxxxxxxxxxxxxx vg01
hdisk4 xxxxxxxxxxxxxxxx vg02
hdisk5 xxxxxxxxxxxxxxxx vg03
hdisk6 xxxxxxxxxxxxxxxx vg04
hdisk7 xxxxxxxxxxxxxxxx vg05

When I run `varyonvg` on haguroyama, it lets varyonvg: vg00, vg02, vg03. It will not allow me to varyonvg volumes: vg01 (hdisk3), vg04 (hdisk6), and vg05 (hdisk7).

This is all the information I have at the moment. So basically to sum it up:

1) I want fget_config on Hag to have the same hdisks(2-7) as Aki.
2) I want to be able to varyonvg and be load distributed on Hag.

Thank you very much for your help!
Regards,
-Bill

When you replace an HBA the world wide name (WWN) will change so the SAN probably isn't communicating on that port. You'll have to change the config/setup at the SAN side I believe.

homeyjoe is right. You'll need to update the zoning on your SAN switches. You will also need to update the SAN device, and it's wwpn mappings. Each LUN is "mapped" to particular fiber adapter addresses called wwpn's (world wide port name). SAN switches have zones which are part of zonesets. At it's simplest, zones specify which fiber adapters can communicate. zones are grouped together in zonesets.

You can determine what the wwpn of the new fiber card is using

Code:

lscfg -vpl fcs1 | grep Network

Beyond that, your SAN vendor should be able to walk you through the re-mapping.

Altering zones and zonesets are potentially very hazardous tasks. If you have multiple devices, you can affect them all. Update/change these with extreme caution and only if you are completely sure of what you are doing.

Hi Guys... This is an update.

QUESTION:

How can I get my HAGUROYAMA machine to mount vg00-vg05 volume groups. It cant seem to talk to the SAN, but I made the WPPN access allowed. How can I make it so when I type 'lspv' I see the same output as my working server AKINOUMI. Both are AIX


DETAILS:



Thanks for your help. We appreciate it very much. Today we did a few tests on the machine which may have put Haguroyama closer (or further) away to the working machine Akinoumi.

We noticed that the Haguroyama had 2 dar devices (dar0, dar1) when running your command, while Akinoumi only had 1 dar device (dar0). Thanks for showing us this command, it seemed to have helped resolve the problem a bit. I then decided to use the below steps to make Haguroyama only have 1 dar device:

rmdev -R -dl dar0 #also removed: hdisk2,3,4
rmdev -R -dl dar1 #also removed: hdisk5,6,7
rmdev -dl dac0
rmdev -dl dac1
rmdev -R -dl fcs0
rmdev -R -dl fcs1
cfgmgr -v

After cfgmgr finished running, there was now only 1 dar device, and the mapping seemed to be closer between the two machines. The only difference is two things:

1. Each Hdisk has "none None" as PVid (when viewing with lspv)
2. Each Hdisk on Haguroyama uses the opposite fcs device from Akinoumi.

Im thinking that if we can fix the above 2 problems, we may be able restore the ability for hacmp to varyonvg the volume groups (such as vg00, vg01 and so forth). So here is the question:

"How can we make the `lspv` command produce the same mapping on both Haguroyama and Akinoumi(working)"?

Below are the results of the commands of the current state of the systems.

lspv
getlvodm -C
fget_config -A
lsdev -C | grep dar
lsdev -C | grep dac
lsdev -C | grep fcs
lsdev -Cc disk -F 'name location'
lsdev -Cc driver -F 'name location'


*As you look at the results below you will notice a lot of differences between the machine! Even the fget_config is weird!*
/_______________________\
|Results are as follows:|
\_______________________/

------------------------------------
------------------------------------
-Haguroyama (Not able to varyonvg)--
------------------------------------
------------------------------------

1) lspv
hdisk0 000268fceda1da0 rootvg active
hdisk1 000268fdfe25883 db01vg active
hdisk2 none None
hdisk3 none None
hdisk4 none None
hdisk5 none None
hdisk6 none None
hdisk7 none None

2) getlvodm -C
hdisk0
hdisk1

3) fget_config -A
--- dar0 ---

dac0 ACTIVE dac1 ACTIVE
hdisk2 dac0
hdisk3 dac1
hdisk4 dac0
hdisk5 dac0
hdisk6 dac1
hdisk7 dac1

4) lsdev -C | grep dar
dar0 Available 1722-600 (600) Disk Array Router

5) lsdev -C | grep dac
dac0 Available 1j-08-01 1722-600 (600) Disk Array Controller
dac1 Available 14-08-01 1722-600 (600) Disk Array Controller

6) lsdev -C | grep fcs
fcs0 Available 1j-08 FC Adapter
fcs1 Available 14-08 FC Adapter

7) lsdev -Cc disk -F 'name location'
hdisk0 1A-08-00-8,0
hdisk1 1A-08-01-10,0
hdisk2 1j-08-01
hdisk3 14-08-01
hdisk4 1j-08-01
hdisk5 1j-08-01
hdisk6 14-08-01
hdisk7 14-08-01

8) lsdev -Cc driver -F 'name location'
dar0
fcnet0 1j-08-02
fcnet1 14-08-02
fscsi0 1j-08-01
fscsi1 14-08-01
scsi2 1A-08-00
scsi3 1A-08-01



--------------------------------------------------
______The Other Machine...________________________
--------------------------------------------------


------------------------------------
------------------------------------
---Akinoumi (Works Perfectly)--
------------------------------------
------------------------------------

1) lspv
hdisk0 000b80bfce436320 rootvg active
hdisk1 000b80bf05467e80 None
hdisk2 000b80bf27ceb498 vg00 active
hdisk3 000b80bf6c9b41f8 vg01 active
hdisk4 000b80bf6c9a3a8d vg02 active
hdisk5 000b80bffbb9f2af vg03 active
hdisk6 000b80bffbb6bf7b vg04 active
hdisk7 000b80bf27dc7320 vg05 active

2) getlvodm -C
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
hdisk6
hdisk7

3) fget_config -A
--- dar0 ---

User array name = "Production"
dac0 ACTIVE dac1 ACTIVE

Disk DAC LUN Logical Drive
hdisk2 dac1 1 AIX_1
hdisk3 dac0 2 AIX_2
hdisk4 dac1 3 AIX_3
hdisk5 dac1 4 AIX_4
hdisk6 dac0 5 AIX_5
hdisk7 dac0 6 AIX_6

4) lsdev -C | grep dar
dar0 Available 1722-600 (600) Disk Array Router

5) lsdev -C | grep dac
dac0 Available 1j-08-01 1722-600 (600) Disk Array Controller
dac1 Available 14-08-01 1722-600 (600) Disk Array Controller

6) lsdev -C | grep fcs
fcs0 Available 1j-08 FC Adapter
fcs1 Available 14-08 FC Adapter

7) lsdev -Cc disk -F 'name location'
hdisk0 1A-08-00-8,0
hdisk1 1A-08-01-10,0
hdisk2 14-08-01
hdisk3 1j-08-01
hdisk4 14-08-01
hdisk5 14-08-01
hdisk6 1j-08-01
hdisk7 1j-08-01

8) lsdev -Cc driver -F 'name location'
dar0
fcnet0 1j-08-02
fcnet1 14-08-02
fscsi0 1j-08-01
fscsi1 14-08-01
scsi2 1A-08-00
scsi3 1A-08-01

simple solution:

You have two servers and want these two servers to see the same SAN STORAGE ?

1) In the SAN STORAGE make sure you assign the logical disk to both AIX machines

2) run cfgmgr on AIX machines

3) Check the pvid id of the hdisks on both servers, it should be same
if there is no pvid then add one with the command ;chdev -l hdiskXX -a pv=yes

4) Check the path of the SAN HDISK by typing
lspath

Hello Sir,

Thanks for your help!

It seems like it might work. I was a bit scared before running the command because it seems like the chdev command will affect the hdisk device on both machines. (In other words my AKI machine will already have it varyonvg, then should I run this command?)

I was scared that running this command on HAG would ruin the ability for AKI to varyonvg the volume group.

The lspath commands on both machines are slightly different orderings also. Here is the output:

HAG> lspath
Enabled hdisk1 scsi3
Enabled hdisk0 scsi2

AKI> lspath
Enabled hdisk0 scsi2
Enabled hdisk1 scsi3


So... is it safe? Is there anything i should know before running your sequence of commands:

1. cfgmgr on HAG
2. cfgmgr on AKI
3. varyoffvg vg00~vg05 on AKI (to make sure no IO is happening to the hdisks)
4. lspath (to make sure the paths to the hdisk0/1 are enabled.... *** Note that hdisk2-7(which are the ones that map to vg00-vg05, are NOT LISTED here. Is that bad?) ***
5. Take a few minutes to relax nerves...
6. Run command:
chdev -l hdisk3 -a pv=yes; lspv ; (then check to see if lspv for hdisk3 has a value instead of "none None").
If safe... then run:
chdev -l hdisk4 -a pv=yes; chdev -l hdisk5 -a pv=yes; chdev -l hdisk6 -a pv=yes; chdev -l hdisk7 -a pv=yes; lspv


Is this safe? Will I be OK?

Thank you for your help!

---------- Post updated 07-02-09 at 07:48 AM ---------- Previous update was 07-01-09 at 08:43 PM ----------

Hello,

How are you doing... I tried the command you told me but it did not work.

When I ran chdev -l it said...:

$ chdev -l hdisk3 -a pv=yes

Method error (/etc/methods/chgfcparray):
0514-047: Cannot access a device: pv

Is this because the volumes are varied on my other server? Should they be varied off before this command?

Thank you very much for your help!

You have to configure the VG for "concurrent access" before it can be accessed from more than one system simultaneously. Its the "-C" flag to "chvg", but if you are unsure about how to do that use "smitty chvg" and follow the panels there - its pretty self-explanatory.

You will have to break the vg-lock afterwards using "chvg -u" to access the VG from another system.

I hope this helps.

bakunin