Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

R5 zpool in corrupted state

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users R5 zpool in corrupted state
# 1  
Old 05-06-2013
R5 zpool in corrupted state
Hi All,

I am getting zpool corrupted message under zpool status command.What could be the reason for this.I had observed this zpool was full this morning after this i am seeing this error.

Code:
[Sun1 root ] $PWD>zpool status -xv R5
  pool: R5
 state: DEGRADED
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://www.sun.com/msg/ZFS-8000-8A
 scan: scrub in progress since Mon May  6 06:21:11 2013
    358G scanned out of 1.26T at 27.8M/s, 9h30m to go
    358G scanned out of 1.26T at 27.8M/s, 9h30m to go
    0 repaired, 27.81% done
config:

        NAME                                       STATE     READ WRITE CKSUM
        R5                                         DEGRADED     0     0 3.56M
          raidz1-0                                 DEGRADED     0     0 21.1M
            c3t600A0B80006872FF000003104B8BE555d0  DEGRADED     0     0     0  too many errors
            c3t600A0B80006872FF000003134B8BE5C3d0  DEGRADED     0     0     0  too many errors
            c3t600A0B800068729A000002DC4B8BE524d0  DEGRADED     0     0     0  too many errors
            c3t600A0B800068729A000002DF4B8BE59Ed0  DEGRADED     0     0     0  too many errors
            c3t600A0B800068729A000002E24B8BE618d0  DEGRADED     0     0     0  too many errors

errors: Permanent errors have been detected in the following files:

        <metadata>:<0x37>
        <metadata>:<0x38>
        <metadata>:<0xa5>
        <metadata>:<0x10c>
        <metadata>:<0x192>
        R5/zones:<0x0>
        R5/cat-test-db/projects:<0x0>
        R5/cat-trn-db/export:<0x0>
        R5/cat-trn-oas/export:<0x0>
        /R5/cat-11gr2-db/app/oracle/oradata/testdb2/datafile/undotbs1_01.dbf
        R5/was801/export:<0x0>
        /R5/oas11g-eval/export/app/oracle/weblogic/10.3.5/user_projects/domains/MST/servers/AdminServer/logs/access.log
        <0x289>:<0x0>

I had a look to check if any of disk has failed.But nothing seems to appear under format/iostat commands.

Format
Code:
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@0,0
       1. c0t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@1,0
       2. c3t600A0B80006872FF000003104B8BE555d0 <SUN-LCSM100_F-0735-278.90GB>
          /scsi_vhci/ssd@g600a0b80006872ff000003104b8be555
       3. c3t600A0B80006872FF000003134B8BE5C3d0 <SUN-LCSM100_F-0735-278.90GB>
          /scsi_vhci/ssd@g600a0b80006872ff000003134b8be5c3
       4. c3t600A0B800068729A000002AB4B87E71Dd0 <SUN-LCSM100_F-0735 cyl 65533 alt 2 hd 128 sec 139>
          /scsi_vhci/ssd@g600a0b800068729a000002ab4b87e71d
       5. c3t600A0B800068729A000002DC4B8BE524d0 <SUN-LCSM100_F-0735-278.90GB>
          /scsi_vhci/ssd@g600a0b800068729a000002dc4b8be524
       6. c3t600A0B800068729A000002DF4B8BE59Ed0 <SUN-LCSM100_F-0735-278.90GB>
          /scsi_vhci/ssd@g600a0b800068729a000002df4b8be59e
       7. c3t600A0B800068729A000002E24B8BE618d0 <SUN-LCSM100_F-0735-278.90GB>
          /scsi_vhci/ssd@g600a0b800068729a000002e24b8be618
       8. c3t600000E00D0000000001277400020000d0 <FUJITSU-ETERNUS_DXL-0000-1.79TB>
          /scsi_vhci/ssd@g600000e00d0000000001277400020000
       9. c3t600000E00D0000000001277400000000d0 <FUJITSU-ETERNUS_DXL-0000-272.50GB>
          /scsi_vhci/ssd@g600000e00d0000000001277400000000
      10. c3t600000E00D0000000001277400010000d0 <FUJITSU-ETERNUS_DXL-0000-1.86TB>
          /scsi_vhci/ssd@g600000e00d0000000001277400010000
      11. c6t2d31 <SUN-UniversalXport-0735 cyl 8 alt 2 hd 64 sec 64>
          /pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w202500a0b86872ff,1f
      12. c7t0d31 <SUN-UniversalXport-0735 cyl 8 alt 2 hd 64 sec 64>
          /pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w202400a0b86872ff,1f
Specify disk (enter its number): Specify disk (enter its number):

Iostat output
Code:
c0t1d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE  Product: ST973402SSUN72G  Revision: 0603 Serial No: 080522G5KP
Size: 73.41GB <73407865856 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 3 Predictive Failure Analysis: 0
c0t0d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE  Product: ST973402SSUN72G  Revision: 0603 Serial No: 080522FVZH
Size: 73.41GB <73407865856 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 3 Predictive Failure Analysis: 0
c0t3d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: TSSTcorp Product: CD/DVDW TS-L632D Revision: SR02 Serial No:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
c6t2d31          Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: Universal Xport  Revision: 0735 Serial No:
Size: 0.02GB <20971520 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 53 Predictive Failure Analysis: 0
c7t0d31          Soft Errors: 0 Hard Errors: 1 Transport Errors: 0
Vendor: SUN      Product: Universal Xport  Revision: 0735 Serial No:
Size: 0.02GB <20971520 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 5 Predictive Failure Analysis: 0
c3t600A0B800068729A000002AB4B87E71Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: LCSM100_F        Revision: 0735 Serial No:
Size: 598.93GB <598925639680 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
c3t600A0B800068729A000002E24B8BE618d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: LCSM100_F        Revision: 0735 Serial No:
Size: 299.46GB <299462819840 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
c3t600A0B80006872FF000003134B8BE5C3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: LCSM100_F        Revision: 0735 Serial No:
Size: 299.46GB <299462819840 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c3t600A0B800068729A000002DF4B8BE59Ed0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: LCSM100_F        Revision: 0735 Serial No:
Size: 299.46GB <299462819840 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
c3t600A0B80006872FF000003104B8BE555d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: LCSM100_F        Revision: 0735 Serial No:
Size: 299.46GB <299462819840 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c3t600A0B800068729A000002DC4B8BE524d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SUN      Product: LCSM100_F        Revision: 0735 Serial No:
Size: 299.46GB <299462819840 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
c3t600000E00D0000000001277400000000d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: FUJITSU  Product: ETERNUS_DXL      Revision: 0000 Serial No:
Size: 292.59GB <292594647040 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 1 Predictive Failure Analysis: 0
c3t600000E00D0000000001277400010000d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: FUJITSU  Product: ETERNUS_DXL      Revision: 0000 Serial No:
Size: 2048.16GB <2048162529280 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 1 Predictive Failure Analysis: 0
c3t600000E00D0000000001277400020000d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: FUJITSU  Product: ETERNUS_DXL      Revision: 0000 Serial No:
Size: 1966.56GB <1966558150656 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
rmt/1            Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HP       Product: Ultrium 3-SCSI   Revision: G57Z Serial No:
rmt/0            Soft Errors: 137 Hard Errors: 46 Transport Errors: 141118
Vendor: HP       Product: Ultrium 3-SCSI   Revision: G56Z Serial No:

Dmesg logs

Code:
May  4 09:10:09 sun        mpt_handle_event: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event_sync: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event_sync: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event_sync: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):
May  4 09:10:09 sun        mpt_handle_event_sync: IOCLogInfo=0x110a0000
May  4 09:10:09 sun scsi: [ID 243001 kern.info] /pci@3,700000/pci@0/scsi@8 (mpt1):

Last edited by sahil_shine; 05-06-2013 at 06:56 AM.. Reason: added dmesg logs
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

Zpool mirroring

Now this doesnt look right to me. All of these disks are 100Gb LUNS so total zpool size is 300Gb. Am I right in saying that this zpool consists of two disks plus one more disk that is 6 way mirrored? So a bit pointless because only one of the three is mirrored (and 6 way is a bit of overkill... (3 Replies)
Discussion started by: psychocandy
3 Replies

2. Solaris

Replace zpool with another disk

issue, I had a zpool which was full pool_temp1 199G 197G 1.56G 99% ONLINE - pool_temp2 199G 196G 3.09G 98% ONLINE - as you can see, full so I replaced with a larger disk. zpool replace pool_temp1 c3t600144F0FF8BA036000058CC1DB80008d0s0... (2 Replies)
Discussion started by: rrodgers
2 Replies

3. BSD

Zpool problem

Hi I have a problem with size on zfs filesystem on FreeBSD 9.2-RELEASE-p3. When I do this: free01# df -Th Filesystem Type Size Used Avail Capacity Mounted on /dev/ufs/FreeNASdde ufs 926M 826M 26M 97% / devfs devfs ... (1 Reply)
Discussion started by: primo102
1 Replies

4. Solaris

Shrinking zpool

Hello experts, I have a solaris 10 (SunOS 5.10 Generic_148888-05 sun4u sparc SUNW,SPARC-Enterprise) that by mistake I added a second san space of 700g to the pool. the whole pool is now 1.2T and, I need to take the space away from the pool and, make the pool 700g total. this is live oracle... (7 Replies)
Discussion started by: afadaghi
7 Replies

5. AIX

Open firmware state to running state

Hi Admins, I am having a whole system lpar in open firmware state on HMC. How can I bring it to running state ? Let me know. Thanks. (2 Replies)
Discussion started by: snchaudhari2
2 Replies

6. Solaris

How to tell what disks are used for a zpool?

Hello, Does anyone know how I can tell what disk are being not being used by a zpool? For example in Veritas Volume manager, I can run a "vxdisk list" and disks that are marked as "online invalid" are disk that are not used. I'm looking for a similar command in ZFS which will easily show... (5 Replies)
Discussion started by: robertinoau
5 Replies

7. Solaris

Zpool query

Hi, I have an X86pc with Solaris 10 and ZFS system. It has 8 similar disks. I need help in creating some zpools and changing the mount-point of a slice. Currently, the zpool in my system is like this: root@abcxxx>zpool status pool: rpool state: ONLINE scrub: none requested... (4 Replies)
Discussion started by: mystition
4 Replies

8. Solaris

zpool hasn't expanded

Hi Guys, I have a raidz zpool that consists of four disks. 2x2TB, 1x1TB and 1x0.75TB. Originally it was only 1x1TB, 3x0.75TB, and I had around 1.7TB of storage capacity. I've just switched out two of the 0.75TB disks for the 2x2TB ones. I did this one at a time and now the resilvering is... (2 Replies)
Discussion started by: rudigarude
2 Replies

9. Solaris

ZPOOL help..

hi ... i have added a physical disk to the pool with ""zpool add <poolname> diskname"""... after that i realized that i have to mirror it instead..then i tried to take that disk out of the pool but i m not able to do that.. i have gone through many unix help sites , nothing worked , so please... (6 Replies)
Discussion started by: yrajendergoud
6 Replies

10. Solaris

need zpool to revert...

hi i have created a pool using zpool command for my /dev/dsk/c1d0s3 disk. The poolname is qwertyuiopasdfghjklmnbvcxzzxcvbnmasdfghjklqwertyuiopoiuytrewqasdfghjklkjhgfdsazxcvbnmmnbnbcxczxzassd ddddvfhfghgjjgjhgkhkljfjlhohihiuyuioyguioyguiowyuiogwyuigwrigywuigyguiyuiogyugiyguioyuyguiowygiuygui... (1 Reply)
Discussion started by: SankarV
1 Replies
Login or Register to Ask a Question