DISK and MPIO


 
Thread Tools Search this Thread
Operating Systems AIX DISK and MPIO
# 1  
Old 04-01-2019
DISK and MPIO

Hello,
I have some concerns over the disk management of my AIX system.
For example server1
Code:
[root@server1] / > lspv
hdisk0          00fa6d1288c820aa                    rootvg          active
hdisk1          00fa6d1288c8213c                    vg_2           active
hdisk2          00c1cc14d6de272b                    vg_3            active
hdisk3          00c1cc14d6de27e5                    vg_3            active
hdisk4          00c1cc14d6de2898                    vg_3            active
hdisk5          00c1cc14d6de2958                    vg_3            active
hdisk6          00c1cc14d6de2a07                    vg_3            active
hdisk7          00c1cc14716beb5c                    vg_3            active
hdisk8          00c1cc14816d37d0                    vg_2           active
hdisk9          00c1cc14816d38a0                    vg_3            active
hdisk10         00c1cc14816d396c                    vg_3            active
hdisk11         00c1cc14716beb98                    vg_3            active
hdisk12         00c1cc14716bebd9                    vg_3            active
hdisk13         none                                None

[root@server1] / > mpio_get_config -A
    Storage Subsystem worldwide name: 600a0b80006e87a4000000004f59530d
    Storage Subsystem Name = 'SGDS5KSTG'
        hdisk#           LUN #   Ownership          User Label
        hdisk0               0   B (preferred)      OS_01
        hdisk1               1   B (preferred)       OS_02
        hdisk2               2   B (preferred)      VG_01
        hdisk3               3   B (preferred)      VG_02
        hdisk4               4   B (preferred)      SVG_03
        hdisk5               5   B (preferred)      VG_04
        hdisk6               6   B (preferred)      VG_05
        hdisk7               7   B (preferred)      VG_10
        hdisk8               8   A (preferred)      VG_07
        hdisk9               9   A (preferred)      VG_08
        hdisk10             10   A (preferred)      VG_09
        hdisk11             11   B (preferred)      VG_11
        hdisk12             12   B (preferred)      VG_12

[root@server1] / > lsdev -Cc disk
hdisk0  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk1  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk2  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk3  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk4  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk5  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk6  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk7  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk8  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk9  Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk10 Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk11 Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk12 Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk13 Available 23-T1-01 MPIO FC 2145

So my questions are:
1. I can see there is 14 hdisks (from hdisk#0 to hdisk#13, come from DS5K) but in the mpio_get_config I only see 13 hdisks (from hdisk#0 to hdisk#12), the hdisk#13 is the new one from different storage V7000.
From my point of view, the mpio_get_config command should display the mpio info of hdisk#13 as well?

2. What difference between this? It should be show the same drivers as the driver is AIX PCM?
Code:
hdisk12 Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk13 Available 23-T1-01 MPIO FC 2145

3. And in the server2, the mpio_get_config command even returns no results
Code:
[root@server2] / > mpio_get_config -A
Error: Unable to find devices.

Can we know in which context mpio_get_config command is applicable?
# 2  
Old 04-01-2019
Quote:
Originally Posted by Phat
1. I can see there is 14 hdisks (from hdisk#0 to hdisk#13, come from DS5K) but in the mpio_get_config I only see 13 hdisks (from hdisk#0 to hdisk#12), the hdisk#13 is the new one from different storage V7000.
From my point of view, the mpio_get_config command should display the mpio info of hdisk#13 as well?
I don't know this command, but probably not:
Quote:
Purpose

Displays information about the DS3000 or the DS4000® subsystem that is based on multiple path I/O (MPIO) and the hdisks that are associated with the subsystem.
It may be that the command is able to list DS5k LUNs as well (like in your case) and maybe even the Storwize system but the way these IBM commands usually work is that they filter their information from the ODM. Remove the disk(s) and its associated adapter (rmdev -Rdl, see its man page for details) to remove the ODM information then run cfgmgr again to re-populate the ODM-entries.

Quote:
Originally Posted by Phat
2. What difference between this? It should be show the same drivers as the driver is AIX PCM?
Code:
hdisk12 Available 23-T1-01 MPIO DS5100/5300 Disk
hdisk13 Available 23-T1-01 MPIO FC 2145

See above: maybe just missing ODM information which a rebuild may fix, maybe the command just is not built to display subsystems like the V7000. You also may try this link from the NetApp manual and look if it helps your situation.

Judging from your last posts i'd suggest you take a look (again) at the zoning too and make sure that the system can indeed see the disks. Regardless of mpio_get_config working or not the system should be able to access the LUNs so that lspv, lsdev -Cc disk and similar commands should show them. Run cfgmgr and see if the LUNs show up.

I hope this helps.

bakunin
# 3  
Old 04-01-2019
Hi,

I have another questions

Code:
[root@server2] / > lsdev -Cc disk
hdisk0  Available 13-T1-01 IBM MPIO FC 2107
hdisk1  Available 13-T1-01 IBM MPIO FC 2107
hdisk2  Available 13-T1-01 IBM MPIO FC 2107
hdisk3  Available 13-T1-01 IBM MPIO FC 2107
hdisk4  Available 13-T1-01 IBM MPIO FC 2107
hdisk5  Available 13-T1-01 IBM MPIO FC 2107
hdisk6  Available 13-T1-01 IBM MPIO FC 2107
hdisk7  Available 13-T1-01 IBM MPIO FC 2107
hdisk8  Available 13-T1-01 IBM MPIO FC 2107
hdisk9  Available 13-T1-01 IBM MPIO FC 2107
hdisk10 Available 13-T1-01 IBM MPIO FC 2107
hdisk11 Available 13-T1-01 IBM MPIO FC 2107
hdisk12 Available 13-T1-01 IBM MPIO FC 2107
hdisk13 Available 13-T1-01 IBM MPIO FC 2107
hdisk14 Available 13-T1-01 IBM MPIO FC 2107
hdisk15 Available 13-T1-01 IBM MPIO FC 2107
hdisk16 Available 13-T1-01 IBM MPIO FC 2107
hdisk17 Available 13-T1-01 IBM MPIO FC 2107
hdisk18 Available 13-T1-01 MPIO Other FC SCSI Disk Drive
hdisk19 Defined   13-T1-01 Other FC SCSI Disk Drive
hdisk20 Defined   13-T1-01 Other FC SCSI Disk Drive
hdisk21 Defined   13-T1-01 Other FC SCSI Disk Drive
hdisk22 Available 13-T1-01 IBM MPIO FC 2107
hdisk23 Available 13-T1-01 IBM MPIO FC 2107

--> the disks marked with Defined mean that they are not available?

on hdisk0 it shows it's using sddpcm, but on hdisk18 it uses PCM/friend/fcpother (/usr/lib/drivers/aixdiskpcmke)

Code:
[root@server2] / > lsattr -El hdisk0
PCM             PCM/friend/sddpcm                PCM                                     True
PCM             PCM/friend/sddpcm                PCM                                     True
PR_key_value    none                             Reserve Key                             True
PR_key_value    none                             Reserve Key                             True
algorithm       load_balance                     Algorithm                               True
clr_q           no                               Device CLEARS its Queue on error        True
clr_q           no                               Device CLEARS its Queue on error        True
dist_err_pcnt   0                                Distributed Error Percentage            True
dist_tw_width   50                               Distributed Error Sample Time           True
hcheck_interval 60                               Health Check Interval                   True
hcheck_mode     nonactive                        Health Check Mode                       True
location                                         Location Label                          True
location                                         Location Label                          True
lun_id          0x4091400000000000               Logical Unit Number ID                  False
lun_id          0x4091400000000000               Logical Unit Number ID                  False
lun_reset_spt   yes                              Support SCSI LUN reset                  True
lun_reset_spt   yes                              Support SCSI LUN reset                  True
max_transfer    0x100000                         Maximum TRANSFER Size                   True
max_transfer    0x100000                         Maximum TRANSFER Size                   True
node_name       0x5005076304ffd345               FC Node Name                            False
node_name       0x5005076304ffd345               FC Node Name                            False
pvid            00c528f76cff27e10000000000000000 Physical volume identifier              False
pvid            00c528f76cff27e10000000000000000 Physical volume identifier              False
q_err           yes                              Use QERR bit                            True
q_err           yes                              Use QERR bit                            True
q_type          simple                           Queuing TYPE                            True
q_type          simple                           Queuing TYPE                            True
qfull_dly       2                                delay in seconds for SCSI TASK SET FULL True
qfull_dly       2                                delay in seconds for SCSI TASK SET FULL True
queue_depth     20                               Queue DEPTH                             True
queue_depth     20                               Queue DEPTH                             True
reserve_policy  no_reserve                       Reserve Policy                          True
reserve_policy  no_reserve                       Reserve Policy                          True
retry_timeout   120                              Retry Timeout                           True
rw_timeout      60                               READ/WRITE time out value               True
rw_timeout      60                               READ/WRITE time out value               True
scbsy_dly       20                               delay in seconds for SCSI BUSY          True
scbsy_dly       20                               delay in seconds for SCSI BUSY          True
scsi_id         0x146e00                         SCSI ID                                 False
scsi_id         0x146e00                         SCSI ID                                 False
start_timeout   180                              START unit time out value               True
start_timeout   180                              START unit time out value               True
unique_id       200B75BGT11910007210790003IBMfcp Device Unique Identification            False
unique_id       200B75BGT11910007210790003IBMfcp Device Unique Identification            False
ww_name         0x500507630438d345               FC World Wide Name                      False
ww_name         0x500507630438d345               FC World Wide Name                      False

[root@server2] / > lsattr -El hdisk18
PCM             PCM/friend/fcpother                   Path Control Module              False
PR_key_value    none                                  Persistant Reserve Key Value     True+
algorithm       fail_over                             Algorithm                        True+
clr_q           no                                    Device CLEARS its Queue on error True
dist_err_pcnt   0                                     Distributed Error Percentage     True
dist_tw_width   50                                    Distributed Error Sample Time    True
hcheck_cmd      test_unit_rdy                         Health Check Command             True+
hcheck_interval 60                                    Health Check Interval            True+
hcheck_mode     nonactive                             Health Check Mode                True+
location                                              Location Label                   True+
lun_id          0x0                                   Logical Unit Number ID           False
lun_reset_spt   yes                                   LUN Reset Supported              True
max_coalesce    0x40000                               Maximum Coalesce Size            True
max_retry_delay 60                                    Maximum Quiesce Time             True
max_transfer    0x80000                               Maximum TRANSFER Size            True
node_name       0x2ff70002ac0109f9                    FC Node Name                     False
pvid            00c528f77c1379a90000000000000000      Physical volume identifier       False
q_err           yes                                   Use QERR bit                     True
q_type          simple                                Queuing TYPE                     True
queue_depth     16                                    Queue DEPTH                      True+
reassign_to     120                                   REASSIGN time out value          True
reserve_policy  single_path                           Reserve Policy                   True+
rw_timeout      30                                    READ/WRITE time out value        True
scsi_id         0x144c00                              SCSI ID                          False
start_timeout   60                                    START unit time out value        True
timeout_policy  fail_path                             Timeout Policy                   True+
unique_id       2510002A000109F9000002VV083PARdatafcp Unique device identifier         False
ww_name         0x20120002ac0109f9                    FC World Wide Name               False

We can opt to select the mpio method sddpcm, AIX PCM for whatever disk we want?
# 4  
Old 04-01-2019
Quote:
Originally Posted by Phat
--> the disks marked with Defined mean that they are not available?
Exactly. AIX stores information about devices generally in the ODM. Since this is updated only by a run of cfgmgr (in fact that is the very purpose of this command) it might be that a device is still defined in the ODM (because once it was available but is not any more now) describes a device that is already removed (or not working for some other reason). These devices will be in state "Defined". Re-plung them in and they become "Available" automatically. Delete them from the ODM (via rmdev and they will be gone forever - only to return after being replugged and cfgmgr is run again.

Quote:
Originally Posted by Phat
We can opt to select the mpio method sddpcm, AIX PCM for whatever disk we want?
You need to use a driver that works (duh! ;-)) ). It does not only depend on a disk if a driver works or not but also on the nature of the connection LUN<->host system, the storage subsystem involved the type of fabric you use and so on.

I hope this helps.

bakunin
# 5  
Old 04-01-2019
Quote:
You need to use a driver that works (duh! ;-)) ).
Yes I know Smilie but can we pickup whatever 1 disk and make it use AIX PCM or vice versa? Just want to know the possibility.

But I wonder why how they can have the 2 different drivers. I detect hdisk18 belongs to IBMSVC which is using AIX PCM whereas the others are using SDDPCM.

Code:
[root@server2] / > manage_disk_drivers -l
Device              Present Driver        Driver Options
2810XIV             AIX_AAPCM             AIX_AAPCM,AIX_non_MPIO
DS4100              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4200              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4300              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4500              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4700              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4800              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS3950              AIX_APPCM             AIX_APPCM
DS5020              AIX_APPCM             AIX_APPCM
DCS3700             AIX_APPCM             AIX_APPCM
DCS3860             AIX_APPCM             AIX_APPCM
DS5100/DS5300       AIX_APPCM             AIX_APPCM
DS3500              AIX_APPCM             AIX_APPCM
XIVCTRL             MPIO_XIVCTRL          MPIO_XIVCTRL,nonMPIO_XIVCTRL
2107DS8K            NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
IBMFlash            NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
IBMSVC              NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO

So what is IBMSVC? can we move disk from IBMSVC to 2017DS8K? I can see that all the LUNs comes from 1 source storage and visible to all 4 FCs adapters.
# 6  
Old 04-01-2019
Quote:
Originally Posted by Phat
Yes I know Smilie but can we pickup whatever 1 disk and make it use AIX PCM or vice versa? Just want to know the possibility.
Well, maybe i didn't phrase that clearly enough: that depends. It depends on a lot of things i don't know because i do not know your environment and - with all due respect - you don't seem to know either, so you can't tell me what i would need to know to explain it to you.

Quote:
Originally Posted by Phat
But I wonder why how they can have the 2 different drivers. I detect hdisk18 belongs to IBMSVC which is using AIX PCM whereas the others are using SDDPCM.
SDDPCM and MPIO are two different multipathing drivers, both for AIX. SDDPCM is a bit older than MPIO. You cannot just switch between them because you will need to meet certain requirements to use either of them. But this would be better discussed with someone knowing your complete setup: fabric, storage boxes, zoning, FC-switches, virtualiser, .... I would be of limited help in this capacity (and everybody else here too) because we will not have the intimate knowledge that would be required. My suggestion: go to the storage people and ask one of them to explain storage basics and their application in your specific surroundings to you.

I am terribly sorry but this is something i cannot tell you over the net with just a few forum posts. The matter is just too complex (and the single pieces too interdependent) for that.

Quote:
Originally Posted by Phat
So what is IBMSVC? can we move disk from IBMSVC to 2017DS8K? I can see that all the LUNs comes from 1 source storage and visible to all 4 FCs adapters.
IBM SAN Volume Controller is a storage virtualiser. It is similar in scope to i.e. EMCs VPlex product. The DS8K is a (high-end) storage box itself, which is probably managed (or maybe not - i don't know your environment) by the SVC. In principle the SVC sits between (all) the storage boxes on one side and (all) the systems using their LUNs on the other side. So, the answer to your question wether you can move the LUN from one to the other: maybe. It depends on wether the DS8k is configured to let the SVC manage the LUNs it exports or not and, again, it depends on the layout of your FC fabric and it depends on your FC switches and how they are connected and zoned and, .... You see the problem?

I hope this helps.

bakunin
# 7  
Old 04-02-2019
Quote:
it depends on the layout of your FC fabric and it depends on your FC switches and how they are connected and zoned and, .... You see the problem?
Yes I see it and understand what you want to convey. By some reasons, I have not enough info to assess.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

Need Help with SDD / SDDPCM / MPIO

This is getting very confusing for me, and appreciate if someone can help. Platform: Power VM ( Virtual I/O Server) ioslevel 2.1.3.10-FP23 # oslevel -s 6100-05-00-0000 Storage: IBM DS4300 Two HBAs - Dual Port Fibre Adapter Channels Each card has two ports , so a total of 4 ports going... (3 Replies)
Discussion started by: filosophizer
3 Replies

2. AIX

Failed mpio path on AIX5.3

I have found failed mpio path on AIX and enabled that failed path as below failed path on AIX bash-3.2# lspath -l hdisk10 Enabled hdisk10 fscsi0 Enabled hdisk10 fscsi0 Failed hdisk10 fscsi3 Enabled hdisk10 fscsi3 Enabled failed path as below chpath -l hdisk10 -p fscsi3 -s enable... (13 Replies)
Discussion started by: murali969
13 Replies

3. AIX

Upgrading from native MPIO to SDDPCM AIX 6.1

We have AIX 6.1 system attached to SAN disks (DS4700 and DS8100) thru SVC. Initially when the system was I forgot to install sddpcm drivers. and wanted to know how can i go with the installation of the sddpcm drivers. My understandin going thru the manual ... (3 Replies)
Discussion started by: mk8570
3 Replies

4. AIX

MPIO - list of supported arrays

Hello, we are planning to deploy some of our databases on AIX/LPAR based servers (we didn't bought it yet ...). IBM's engineers says that if we want to boot them from SAN the hardware array has to be compatible with MPIO but they don't want to deliver any document with list of arrays supported... (5 Replies)
Discussion started by: andy55
5 Replies

5. AIX

MPIO reliability

Hi, we have a vew boxes using MPIO and they are connected to some virtualization software managing some disk subsystems, offering volumes to the AIX boxes. Sometimes when a cable has been plugged out for a test or when a real problem occurs, using lspath to show the state of the paths shows... (8 Replies)
Discussion started by: zaxxon
8 Replies

6. AIX

MPIO Driver

On a particular LPAR, I was running AIX 5.3 TL 3. On Monday I did an update of the LPAR to 5.3 TL 9 SP2. The install was smooth, but then I ran into a problem. The MPIO driver does not work with LSI's StoreAge (SVM4). I did some looking, and looks like 5.3 TL3 = IBM.MPIO 5.3.0.30 5.3... (0 Replies)
Discussion started by: clking
0 Replies

7. AIX

SDD SDDPCM MPIO lspath Jargon

Can anyone recommend me some reading material surrounding how AIX handles LUNs: - with and without MPIO installed - with and without SDD or SDDPCM installed Where does lspath sit in all of this (MPIO layer?). Can a system be built with just MPIO software? Is MPIO software even needed? I guess... (0 Replies)
Discussion started by: apra143
0 Replies

8. AIX

AIX native MPIO

Hi folks, does anybody have a link to a documentation how to implement native MPIO on AIX? We are using EMC PowerPath and Datacore SanSymphony/Cambex for this so far and I wasn't able to find a good description on that topic. All I know so far is that mkpath, chpath and lspath are used to... (3 Replies)
Discussion started by: zaxxon
3 Replies

9. AIX

AIX 5.3 MPIO vs RDAC on DS4000 range

Hi I would like to ask what the benefits are of changing from RDAC to MPIO when connecting to a DS4000 on AIX 5.3? I have heard that IBM MPIO "might" support more than 1 active path to a LUN when connecting to a DS4800 through more than 1 host connection on the same AIX client. I understand that... (8 Replies)
Discussion started by: kimyo
8 Replies

10. UNIX for Advanced & Expert Users

AIX MPIO and EMC

We are looking at running MPIO for it's redundancy and load balancing benefits. Does anyone know what pieces of software or modules are needed on the VIO server to get load balancing to work. Remember we are using EMC's DMX3500 storage system. We no longer want to use Powerpath. :rolleyes: ... (2 Replies)
Discussion started by: vxg0wa3
2 Replies
Login or Register to Ask a Question