Need Help with SDD / SDDPCM / MPIO


 
Thread Tools Search this Thread
Operating Systems AIX Need Help with SDD / SDDPCM / MPIO
# 1  
Old 05-07-2016
Need Help with SDD / SDDPCM / MPIO

This is getting very confusing for me, and appreciate if someone can help.

Platform: Power VM ( Virtual I/O Server)
ioslevel 2.1.3.10-FP23
# oslevel -s
6100-05-00-0000
Storage: IBM DS4300
Two HBAs - Dual Port Fibre Adapter Channels
Each card has two ports , so a total of 4 ports going through two San Switches into one SAN box

Support Matrix

Code:
# pcmpath
ksh: pcmpath:  not found.

# lsvpcfg
ksh: lsvpcfg:  not found.


# manage_disk_drivers -l
Device           Present Driver     Driver Options
2810XIV          AIX_AAPCM          AIX_AAPCM,AIX_non_MPIO
DS4100           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4200           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4300           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4500           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4700           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4800           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS3950           AIX_APPCM          AIX_APPCM
DS5020           AIX_APPCM          AIX_APPCM
DS5100           AIX_APPCM          AIX_APPCM
DS3500           AIX_APPCM          AIX_APPCM


# lslpp -l | grep mpio
  devices.common.IBM.mpio.rte
  devices.common.IBM.mpio.rte


# lslpp -Lc | grep -E "sdd|fcp.disk.ibm"
#


# lspath -H
status  name   parent
Enabled dac0   fscsi0
Enabled dac1   fscsi0
Enabled dac0   fscsi1
Enabled dac1   fscsi1
Enabled dac0   fscsi2
Enabled dac1   fscsi2
Enabled dac0   fscsi3
Enabled dac1   fscsi3
Enabled hdisk8 fscsi0
Enabled hdisk9 fscsi0
Enabled dac1   fscsi2
Enabled hdisk8 fscsi1
Enabled hdisk9 fscsi1
Enabled hdisk8 fscsi2
Enabled hdisk9 fscsi2
Enabled hdisk8 fscsi3
Enabled hdisk9 fscsi3
#

# lspv
hdisk8          00c7780e8979b683                    None
hdisk9          00c7780e8979b683                    None
#


# lsdev -Cc disk
hdisk8 Available 0A-08-02     MPIO Other DS4K Array Disk
hdisk9 Available 0A-08-02     MPIO Other DS4K Array Disk

# lsattr -El fscsi0
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10200      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True
#

# lsattr -El fscsi1
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10200      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True

# lsattr -El fscsi2
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10300      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True


# lsattr -El fscsi3
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10300      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True
#


The confusing part for me, i read somewhere
Quote:
However, the SDDPCM fileset might be required on a VIOS partition for attached DS8000 and/or SVC devices and you might want to attach also DS4000/DS5000 devices to the same VIOS. A supported configuration with SDDPCM on VIOS2.1 for DS8000 and/or SVC devices and native AIX MPIO for DS4k/DS5k devices might be achived by using SDDPCM 2.5.1.0 on VIOS 2.1.2.10-FP-22.1 (or higher) and manually deselecting the SDDPCM driver option for the DS4k/DS5k devics with the manage_disk_drivers command. Such a configuration would be based on:
VIOS 2.1.2.10-FP-22.1 (or higher) required by SDDPCM 2.5.1.0 and including the new manage_disk_drivers command
SDDPCM 2.5.1.0 (or higher) fileset with appropriate SDDPCM prerequisites (e.g. devices.fcp.disk.ibm.mpio.rte, V1.0.0.20)
selecting AIX_APPCM as present driver for attached DS5k/DS4k subsystems with manage_disk_drivers command (i.e. deselect AIX_SDDAPPCM)
AIX MPIO using SDDPCM for DS8000 / SVC devices (i.e. only the DS8000/SVC devices are managed by SDDPCM)
AIX MPIO using native AIX PCM for all DS4k/DS5k devices (i.e. all DS4k/DS5k devices are managed by the default AIX MPIO driver with the native AIX PCM)


How can I solve this issue of Same Disk appearing Twice (see hdisk8 and hdisk9 have the same PVIDs)

How should I proceed ?



---------- Post updated 05-07-16 at 12:35 PM ---------- Previous update was 05-06-16 at 01:06 PM ----------

Code:
+-----------------+
|  host           | 
+-----------------+
   |       |
   | 3     | 4
   |       |
+-----------------+
|  FC-Switch      |
+-----------------+
   |       |
   | 1     | 2
   |       |
+-----------------+
|  Storage        |
+-----------------+

After a little more study, I have come to known that it is important to have the right zoning, on this forum, I have read
Quote:
In your case, if your AIX system sees extra "dacX" devices or extra "hdiskX" devices, there's an error in your config & you'll need to go back & debug your installation.

I have 2 x Dual Port HBA

HBA 1
FCS0
FCS1

HBA 2
FCS2
FCS3

What is the best zoning practice ?

fcs0 (HBA 1) -----> switch A <------ controller A

fcs3 (HBA 2) -----> switch B <------ controller B

or could i make it like this

fcs0 (HBA 1) -----> switch A <------ controller A
fcs1 (HBA 1) -----> switch B <------ controller B

fcs3 (HBA 2) -----> switch B <------ controller B
fcs4 (HBA 2) -----> switch A <------ controller A





Explanation from Bakunin
Quote:
It has nothing to do with how many ports one adapter has. A "path" is just a way of looking at one disk. The short answer is, a "path" is a certain connection from the system to the disk, using a specific way.

Here is the long version: consider the following setup. You have one FC-Adapter (two ports, FCa, FCb), which is connected to two FC-switches (SWa, SWb), which both connect to a storage system. The switches are themselves connected via a crossover-link. You create a single disk on this storage system. This disk will be seen over different paths: FCa-SWa-Disk, FCa-SWa-SWb-Disk, FCb-SWb-Disk, FCb-SWb-SWa-Disk. AIX will therefore create 4 different hdisk-devices when you run cfgmgr.

Now, this is not very comfortable, especially when you create several disks. Each disk will be seen via 4 paths (if you have a fabric more complex than the one i sketched above you might have even more paths) and soon you will have trouble to remember that hdisk137 is in fact hdisk49 and also hdisk87 and hdisk3.

This is where MPIO comes into play: it takes all the hdisk-devices which represent in fact one and only one disk (its like taking a picture of a building from several angles - regardless of how many pictures you take it is always the same one building) and builds a "super-device" above them. You can use this super-device to build VGs, LVs and all other sorts of things you use normally disks for. The MPIO (Multipath-I/O) driver will take care about the relationship of the different hdisk devices and do the bookkeeping for you (and the OS as well).

And there is an additional advantage to it: suppose Switch A in the above picture would fail. Two of the four hdisks (the two paths involving SWa) would be inaccessible, because these pathes would not be usable, but the super-device still works, because it uses ALL paths and if two fail there are still two more to use. In fact this is the main reason one builds such complex fabrics with parallel FC-switches, crossover-links and the like.
and old post
Quote:
Volume groups other than rootvg normally use virtual SAN disks called vpaths.
The system-side SAN attachment is done through 2 Fibre Channel PCI adapters, connected to 2 independent SAN fabrics. The logical disks on the SAN are attached to each fabric via 2 storage paths, totaling to 4 paths from the host to each LUN.
Each path to a LUN is seen by AIX as a fiber channel disk device
The Subsystem Device Driver (SDD) groups all paths to the same LUN into a multipath virtual device (vpath) which is accessible for IO operations [MPIO] as long as at least one of the component paths is available.

My only concern is not to see the HdiskX as double, with same PVID? any idea folks ?

Last edited by filosophizer; 05-10-2016 at 06:04 AM..
# 2  
Old 05-08-2016
Further developments

Present Config
Code:
# lspcmcfg
hdisk8 (Avail pv ) 600A0B800019B057000010A3572E3AC5 = path0 (Enabled) path1 (Enabled) path2 (Ena
# pcmpath query device

Total Active/Passive Devices : 1


DEV#:   8  DEVICE NAME: hdisk8  TYPE: 1742-900  ALGORITHM:  Load Balance
SERIAL: 600A0B800019B057000010A3572E3AC5
==========================================================================
Path#      Adapter/Path Name          State     Mode     Select     Errors
    0           fscsi0/path0          CLOSE   NORMAL          0          0
    1           fscsi1/path1          CLOSE   NORMAL          0          0
    2           fscsi2/path2          CLOSE   NORMAL          0          0
    3           fscsi3/path3          CLOSE   NORMAL          0          0
#
#
#
#       lspath
Enabled hdisk0 scsi2
Enabled hdisk1 scsi2
Enabled hdisk2 scsi2
Enabled hdisk3 scsi2
Enabled hdisk4 scsi3
Enabled hdisk5 scsi3
Enabled hdisk6 scsi3
Enabled hdisk7 scsi3
Enabled hdisk8 fscsi0
Enabled dac0   fscsi0
Enabled hdisk9 fscsi0
Enabled dac1   fscsi0
Enabled hdisk8 fscsi1
Enabled dac0   fscsi1
Enabled hdisk9 fscsi1
Enabled dac1   fscsi1
Enabled hdisk8 fscsi2
Enabled dac0   fscsi2
Enabled hdisk9 fscsi2
Enabled dac1   fscsi2
Enabled hdisk8 fscsi3
Enabled dac0   fscsi3
Enabled hdisk9 fscsi3
Enabled dac1   fscsi3
#
#

# manage_disk_drivers -l
Device           Present Driver     Driver Options
2810XIV          AIX_AAPCM          AIX_AAPCM,AIX_non_MPIO
DS4100           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4200           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4300           AIX_APPCM          AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4500           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4700           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4800           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS3950           AIX_SDDAPPCM       AIX_APPCM,AIX_SDDAPPCM
DS5020           AIX_SDDAPPCM       AIX_APPCM,AIX_SDDAPPCM
DS5100           AIX_SDDAPPCM       AIX_APPCM,AIX_SDDAPPCM
DS3500           AIX_APPCM          AIX_APPCM
#
#
#
# lspv
hdisk0          00c7780e79838606                    rootvg          active
hdisk1          00c7780e2e21ec86                    diskpool_4      active
hdisk2          00c7780ea5bd16bb                    diskpool_4      active
hdisk3          00c7780ee224f286                    disk_pool_5     active
hdisk4          00c7780e1b75933b                    diskpool_3      active
hdisk5          00c7780ece91bde2                    diskpool_2      active
hdisk6          00c7780ec2b65f4d                    diskpool_1      active
hdisk7          00c7780e5293914b                    None
hdisk8          00c7780e9119f6d7                    None
hdisk9          none                                None
# lsdev -Cc disk
hdisk0 Available 09-08-00-3,0 16 Bit LVD SCSI Disk Drive
hdisk1 Available 09-08-00-4,0 16 Bit LVD SCSI Disk Drive
hdisk2 Available 09-08-00-5,0 16 Bit LVD SCSI Disk Drive
hdisk3 Available 09-08-00-8,0 16 Bit LVD SCSI Disk Drive
hdisk4 Available 09-08-01-3,0 16 Bit LVD SCSI Disk Drive
hdisk5 Available 09-08-01-4,0 16 Bit LVD SCSI Disk Drive
hdisk6 Available 09-08-01-5,0 16 Bit LVD SCSI Disk Drive
hdisk7 Available 09-08-01-8,0 16 Bit LVD SCSI Disk Drive
hdisk8 Available 0A-08-02     IBM MPIO DS4300 Array Disk
hdisk9 Available 0B-08-02     IBM MPIO DS4500 Array Disk

# lspcmcfg
hdisk8 (Avail pv ) 600A0B800019B057000010A3572E3AC5 = path0 (Enabled)
hdisk9 (Avail ) 600A0B800019B057000010A3572E3AC5 = path0 (Enabled)


Both hdisk8 and hdisk9 are same disks but one says IBM MPIO DS4300 and the other IBM MPIO DS4500 apart from showing it double.


So to make it simple:

Direct Connection (removed the san switch)
HBA 1 -- Port 1 -- To Controller A (SAN Storage )
HBA 2 -- Port 1 -- To Controller B (SAN Storage )


Code:
# lsdev -Cc disk

hdisk8 Available 0A-08-02     IBM MPIO DS4300 Array Disk
hdisk9 Available 0B-08-02     IBM MPIO DS4500 Array Disk

again seeing two disks

any solution / suggestion ?

Last edited by filosophizer; 05-10-2016 at 06:06 AM..
# 3  
Old 05-14-2016
What is firmware level on Ds4300 ?
Please check compatible OS & multipath with that Firmware on IBM SSIC website.

Sent from my Le X507 using Tapatalk
# 4  
Old 05-21-2016
Firmware on DS4300 is 6.0.3.x
ALready checked the compatibility matrix

I am trying to upgrade vios to the latest version and will check what happens and post here so everyone can benefit

upgraded ios

Code:
# oslevel -s
6100-09-06-1543

$ ioslevel
2.2.4.10

Code:
manage_disk_drivers -d DS4500 -o AIX_APPCM
manage_disk_drivers -d DS4300 -o AIX_APPCM

did not work

---------- Post updated at 11:44 AM ---------- Previous update was at 12:36 AM ----------

Problem resolved

Code:
# fget_config -Av

---dar0---

User array name = ''
dac0 ACTIVE dac2 ACTIVE

Disk     DAC   LUN Logical Drive
utm             31
hdisk8   dac0    0 1
hdisk9   dac2    1 2
# lsdev -p fcs0
fcnet0 Defined   0A-08-01 Fibre Channel Network Protocol Device
fscsi0 Available 0A-08-02 FC SCSI I/O Controller Protocol Device
# lsdev -p fscsi0
dac0     Available 0A-08-02    1722-600 (600) Disk Array Controller
sfwcomm0 Available 0A-08-02-FF Fibre Channel Storage Framework Comm
#

# manage_disk_drivers -l
Device              Present Driver        Driver Options
2810XIV             AIX_AAPCM             AIX_AAPCM,AIX_non_MPIO
DS4100              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4200              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4300              AIX_fcparray          AIX_APPCM,AIX_fcparray
DS4500              AIX_fcparray          AIX_APPCM,AIX_fcparray
DS4700              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4800              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS3950              AIX_APPCM             AIX_APPCM
DS5020              AIX_APPCM             AIX_APPCM
DCS3700             AIX_APPCM             AIX_APPCM
DS5100/DS5300       AIX_APPCM             AIX_APPCM
DS3500              AIX_APPCM             AIX_APPCM
XIVCTRL             MPIO_XIVCTRL          MPIO_XIVCTRL,nonMPIO_XIVCTRL
2107DS8K            NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
IBMFlash            NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
IBMSVC              NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
#

# lsdev -Cc disk
hdisk0 Available 09-08-00-3,0 16 Bit LVD SCSI Disk Drive
hdisk1 Available 09-08-00-4,0 16 Bit LVD SCSI Disk Drive
hdisk2 Available 09-08-00-5,0 16 Bit LVD SCSI Disk Drive
hdisk3 Available 09-08-00-8,0 16 Bit LVD SCSI Disk Drive
hdisk4 Available 09-08-01-3,0 16 Bit LVD SCSI Disk Drive
hdisk5 Available 09-08-01-4,0 16 Bit LVD SCSI Disk Drive
hdisk6 Available 09-08-01-5,0 16 Bit LVD SCSI Disk Drive
hdisk7 Available 09-08-01-8,0 16 Bit LVD SCSI Disk Drive
hdisk8 Available 0A-08-02     1722-600 (600) Disk Array Device
hdisk9 Available 0B-08-02     1722-600 (600) Disk Array Device

manage_disk_drivers -d DS4300 -o AIX_fcparray

if DS4300 doesn;t work


manage_disk_drivers -d DS4500 -o AIX_fcparray


Last edited by filosophizer; 05-30-2016 at 02:43 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

hdisk status in sdd volume groups

hi all we recently upgraded our san switches. after that, i found i got many hdisks showing in lsdev with status "Defined" and "Available". the number of "Defined" is equal to "Available". the time stamp shows all "Available" disks created at our downtime night. "datapath query device"... (0 Replies)
Discussion started by: rs6000er
0 Replies

2. AIX

sddpcm on vio

I have same lun (hdisk3) assigned to 2 vios server (dual vio server) and this is mapped to the client lpar. I have sddpcm on the VIO. On the VIOA, the attributes for fscsi1 and fscsi3 are set to fast_fail and dyntrk is set to yes Now, if I do a lqueryvg -Atp on the hdisk3, I get a "DISK... (4 Replies)
Discussion started by: mk8570
4 Replies

3. AIX

Upgrading from native MPIO to SDDPCM AIX 6.1

We have AIX 6.1 system attached to SAN disks (DS4700 and DS8100) thru SVC. Initially when the system was I forgot to install sddpcm drivers. and wanted to know how can i go with the installation of the sddpcm drivers. My understandin going thru the manual ... (3 Replies)
Discussion started by: mk8570
3 Replies

4. AIX

AIX SDD

I have sdd driver installed correctly but when I run the command: datapath query adapter No device file found what does it mean.. it is not using vpath or whats wrong? (2 Replies)
Discussion started by: Vit0_Corleone
2 Replies

5. AIX

AIX with SDD to VIOed SDDPCM?

Hi folks. We want to move a "physical" AIX box to a VIOed LPAR. The physical box is running SDD with disks from a DS8300. I want to be able to reattach those LUNs from the DS8300 to my VIO server which is running SDDPCM and then assigned them back to my AIX LPAR. Any thoughts about this? I... (1 Reply)
Discussion started by: Stephan
1 Replies

6. Red Hat

IBM SDD driver

I was wondering if there is a way of auto-detecting a new vpath, using IBM's SDD driver, on a Linux platform. ON AIX it is a simple as running cfgmgr. Alan (0 Replies)
Discussion started by: alanp
0 Replies

7. AIX

SDD SDDPCM MPIO lspath Jargon

Can anyone recommend me some reading material surrounding how AIX handles LUNs: - with and without MPIO installed - with and without SDD or SDDPCM installed Where does lspath sit in all of this (MPIO layer?). Can a system be built with just MPIO software? Is MPIO software even needed? I guess... (0 Replies)
Discussion started by: apra143
0 Replies

8. Filesystems, Disks and Memory

SDD for SVC on HP-UX and LINUX

Hello Unix Admins: Has any one in this distribution list implemented SVC (SAN Volume Contoller - A solution from IBM) in your environment? If you have, do you have any do's / don'ts? Anything which you like sharing would be really appreciated. For example, my inputs: I tried implementing... (1 Reply)
Discussion started by: kdossjojo
1 Replies

9. UNIX for Advanced & Expert Users

Help with SDD, SAN ESS and AIX 5.3 Host

Hi all, Sorry if this is in the wrong place but needed to make sure lots of people saw this so that hopefully someone will be able to help. Basically i've upgraded a test server from 4.3 to 5.3 TL04. The server has hdisk0 and 1 as rootvg locally but then has another vg setup on our ESS... (1 Reply)
Discussion started by: djdavies
1 Replies

10. Solaris

IBM SDD on Sun

I just installed IBMsddsun64_........... on my sun server. I untarred it (tar xvf) & I tried to install the paxkage. When I did pkgadd -d ./IBMsdd..... I get a message Error, no packages were found. Any idea Thanks, (1 Reply)
Discussion started by: Remi
1 Replies
Login or Register to Ask a Question