10-11-2012
8,
0
Join Date: Oct 2012
Last Activity: 11 March 2013, 1:13 AM EDT
Posts: 8
Thanks Given: 1
Thanked 0 Times in 0 Posts
Sun Storage Tek 6140 compatible with RHEL5.7?
Issue Description:
================
There are of 4 servers (SunFire X440) :
siman7tdw: SunFire X440 (affected server)
siman8tdw:SunFire X440 (affected server)
siman9tdw:SunFire X440
siman10tdw:SunFire X440
Storage Server: Sun Storage Tek 6140 (Name: simantdw_disk_bak) and Sun Storage Tek 6780
I) siman7tdw: it is a SunFire X4440 server with following software:
#########################################
[root@siman7tdw ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)
[root@siman7tdw ~]# uname -a
Linux siman7tdw 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@siman7tdw ~]#
#########################################
II) siman8tdw: it is a SunFire X4440 server with following software:
#########################################
[root@siman8tdw ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)
[root@siman8tdw ~]# uname -a
Linux siman8tdw 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@siman8tdw ~]#
#########################################
III) siman9tdw: it is a SunFire X4470 server with following software:
#########################################
[root@siman9tdw ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)
[root@siman9tdw ~]# uname -a
Linux siman9tdw 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@siman9tdw ~]#
#########################################
IV) siman10tdw: it is a SunFire X4470 server with following software:
#########################################
[root@siman10tdw ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)
[root@siman10tdw ~]# uname -a
Linux siman10tdw 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@siman10tdw ~]#
#########################################
All these servers are included in Oracle cluster (RAC) and connected to 2 Oracle storage cabinets (Sun Storage Tek 6140 and Sun Storage Tek 6780) configured with multipath (RDAC).
Since these servers were updated to RHEL Server release 5.7, we observed one of the storage cabinet (Sun Storage Tek 6140, named as "simantdw_disk_bak") is switching between controller A and B quite frequently (many times per day) which sometime even provoked server reboot (file system is ocfs2).
To avoid these reboots all LUNs in storage cabinet "simantdw_disk_bak" were unmounted for all the servers, but controllers continue doing failover for siman7tdw and siman8tdw.
Oracle support confirmed multipathing software does failover between controllers due to timeout and has discarded any problem with storage cabinet.
RDAC driver has been updated to a newer version in every server, but behaviour has not changed.
These are the messages displayed in the affected servers, we are concerned about "mpp" messages and also "lpfc_sci" ones:
#########################################
[root@siman7tdw ~]# tail -f /var/log/messages
Oct 10 08:47:53 siman7tdw kernel: 122 [RAIDarray.mpp]simantdw_disk_bak:0:0:12 Controller IO time expired. Delta 400 secs
Oct 10 08:47:53 siman7tdw kernel: 497 [RAIDarray.mpp]simantdw_disk_bak:0:0:12 Failed controller to 1. retry. vcmnd SN 76094 pdev H3:C0:T1:L12 0x00/0x00/0x00 0x06000000 mpp_status:8
Oct 10 08:47:53 siman7tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 128
Oct 10 08:47:53 siman7tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 128
Oct 10 08:47:53 siman7tdw kernel: 10 [RAIDarray.mpp]simantdw_disk_bak:1 Failover command issued
Oct 10 08:47:53 siman7tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 128
Oct 10 08:48:24 siman7tdw last message repeated 3363 times
Oct 10 08:49:25 siman7tdw last message repeated 8460 times
Oct 10 08:50:26 siman7tdw last message repeated 10103 times
Oct 10 08:51:27 siman7tdw last message repeated 10124 times
#########################################
#########################################
[root@siman8tdw ~]# tail -f /var/log/messages
Oct 10 08:53:28 siman8tdw kernel: 122 [RAIDarray.mpp]simantdw_disk_bak:1:0:10 Controller IO time expired. Delta 400 secs
Oct 10 08:53:28 siman8tdw kernel: 497 [RAIDarray.mpp]simantdw_disk_bak:1:0:10 Failed controller to 0. retry. vcmnd SN 27060 pdev H3:C0:T0:L10 0x00/0x00/0x00 0x06000000 mpp_status:8
Oct 10 08:53:28 siman8tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 124
Oct 10 08:53:28 siman8tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 124
Oct 10 08:53:28 siman8tdw kernel: 10 [RAIDarray.mpp]simantdw_disk_bak:0 Failover command issued
Oct 10 08:53:28 siman8tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 124
Oct 10 08:53:30 siman8tdw last message repeated 2 times
Oct 10 08:53:30 siman8tdw kernel: 801 [RAIDarray.mpp]Failover succeeded to simantdw_disk_bak:0
Oct 10 08:53:30 siman8tdw kernel: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg. Config 64, seg_cnt 124
Oct 10 08:54:01 siman8tdw last message repeated 1678 times
#########################################
Was this problem happened because of RHEL upgrade to 5.7?