AIX

My company have encountered similar problems, and we have seen that some settings need to be set.

Here are the settings which have to be implemented.

Each child fibre device (fscsiX) has to have the following two modes set:

Code:

chdev -l fscsiX -a dyntrk=yes -a fc_err_recov=fast_fail

Additionally, every hdisk device needs to be changed (which I didn't see mentioned in the post).

Code:

chdev -l hdiskX -a reserve_policy=no_reserve

Lastly, you may want to check that the hcheck_interval is NOT set to 0, as then it won't check at all. Usual recommendation is to set to 30 (but 10 should be sufficient).

Code:

chdev -l hdiskX -a hcheck_interval=10

Moderator's Comments:

Use code tags please.

UPDATE: Sorry: The hcheck_interval idea was already mentioned by smurphy. I should have moved on to page 2.

One other thing to check is your "hcheck_interval" which is set at the disk level. The hcheck_interval tells your system how often to check, or re-check, FAILED paths and inactive ENABLED paths (in the case of "algorithm" being set to "fail_over") to ensure they are still connected and functioning. I suggest setting your hcheck_interval to 3600 (once an hour). You'll have to set this on all your disks individually. If the hcheck_interval is set to "0", then this disables it and the disk will never automatically change out of a FAILED or MISSING state.

Remember that MPIO is not like etherchannels, where it automatically re-enables all the paths as soon as the plug is back in. Something has to occur on the disk side to make it recheck them. Either the hcheck_interval comes around again, or you unplug your secondary fiber car which will cause AIX to suddenly start sending checks for all your disks down all the paths, FAILED or MISSING, and try to find a path that is working and it will set it back to ENABLED if it finds one.

Code:

hostname:/:$ lsattr -El hdisk0 | egrep "hcheck_interval"
hcheck_interval 3600                             Health Check Interval      True
hostname:/:$

Also, you can re-enable the paths manually by doing a chdev on it:

Code:

chdev -l hdisk0 -p vscsi0 -s enable

You can also see which path is being used by watching for numbers increasing in the output of "iostat -m":

Code:

hostname:/:$ iostat -m hdisk0

System configuration: lcpu=4 drives=7 ent=0.20 paths=10 vdisks=2

tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc
          0.0         10.6                0.9   0.5   98.3      0.3   0.0    1.6

Disks:        % tm_act     Kbps      tps    Kb_read   Kb_wrtn
hdisk0           0.3      46.3       3.7   180755051  55682968

Paths:        % tm_act     Kbps      tps    Kb_read   Kb_wrtn
Path1            0.0       0.0       0.0          0         0
Path0            0.3      46.3       3.7   180755051  55682968
hostname:/:$