Solaris

Running Solaris 9 with SVM. I'm not that familiar with it, but metastat output gives "needs maintenance" message on 2 of the mirrors. There are no errors in /var/adm/messages. What do I need to do to fix this error? Thanks.

just try the command metasync(1M)....

This may be caused by the "metasync -r" command not getting executed when the system boots, or if the system boots up only to single-user mode.

This metasync command is normally executed in one of the startup scripts run at boot time.

For Online: DiskSuite[TM] 1.0, the metasync command is located in the /etc/rc.local script. This entry is placed in that file by the metarc command.

For Solstice DiskSuite versions between 3.x and 4.2, inclusive, the metasync command is located in the /etc/rc2.d/S95SUNWmd.sync file.

For Solstice DiskSuite version 4.2.1 and above, the metasync command is located in the file /etc/rc2.d/S95lvm.sync.

In all cases, because this script is not run until the system transitions into run state 3 (multi-user mode), it is to be expected to have both submirrors in a "Needs maintenance" state until the command is run. I/O to these metadevices works just fine while in this state, so there is no need to worry.

if that doesn't help, you be in the situation discribed in bug 82642

When trying to run the metasync command, the c1t0d0s0 device reported errors in /var/adm/messages:

Sep 15 09:11:17 bobbob scsi: WARNING: /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0 (ssd1):
Sep 15 09:11:17 bobbob Error for Command: read(10) Error Level: Retryable
Sep 15 09:11:17 bobbob scsi: Requested Block: 4057844 Error Block: 4057969
Sep 15 09:11:17 bobbob scsi: Vendor: SEAGATE Serial Number: 0107D1MVCF
Sep 15 09:11:17 bobbob scsi: Sense Key: Media Error
Sep 15 09:11:17 bobbob scsi: ASC: 0x11 (unrecovered read error), ASCQ: 0x0, FRU: 0xe4
Sep 15 09:11:19 bobbob scsi: WARNING: /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0 (ssd1):
Sep 15 09:11:19 bobbob Error for Command: read(10) Error Level: Retryable
Sep 15 09:11:19 bobbob scsi: Requested Block: 4057844 Error Block: 4057969
Sep 15 09:11:19 bobbob scsi: Vendor: SEAGATE Serial Number: 0107D1MVCF
Sep 15 09:11:19 bobbob scsi: Sense Key: Media Error
Sep 15 09:11:19 bobbob scsi: ASC: 0x11 (unrecovered read error), ASCQ: 0x0, FRU: 0xe4


In this case, the same block is being reported as having problems.

Resolution:

The bad block can be fixed by running format --> analyze --> read on the c1t0d0 disk.

# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0
1. c1t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f8c663,0
Specify disk (enter its number): 0
selecting c1t0d0
format> analyze
analyze> read
Ready to analyze (won't harm SunOS). This takes a long time,
but is interruptable with CTRL-C. Continue? y


pass 0
Medium error during read: block 4057969 (0x3deb71) (1404/16/101)
ASC: 0x11 ASCQ: 0x0
Sep 15 09:26:59 bobbob scsi: WARNING: /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0 (ssd1):
Sep 15 09:26:59 bobbob Error for Command: read(10) Error Level: Retryable
Sep 15 09:26:59 bobbob scsi: Requested Block: 4057969 Error Block: 4057969
Sep 15 09:26:59 bobbob scsi: Vendor: SEAGATE Serial Number: 0107D1MVCF
Sep 15 09:26:59 bobbob scsi: Sense Key: Media Error
Sep 15 09:26:59 bobbob scsi: ASC: 0x11 (unrecovered read error), ASCQ: 0x0, FRU: 0xe4
Repairing hard error on 4057969 (1404/16/101)...ok.


24619/26/53


pass 1
24619/26/53


Total of 1 defective blocks repaired.


Now running metasync completes.

# metasync d10
# metastat d10
d10: Mirror
Submirror 0: d0
State: Needs maintenance
Submirror 1: d1
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 69078879 blocks


d0: Submirror of d10
State: Needs maintenance
Invoke: after replacing "Maintenance" components:
metareplace d10 c1t0d0s0 <new device>
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t0d0s0 0 No Last Erred


d1: Submirror of d10
State: Okay
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t1d0s0 0 No Okay


And then, metareplace can be executed.

# metareplace -e d10 c1t0d0s0
# metastat d10
d10: Mirror
Submirror 0: d0
State: Okay
Submirror 1: d1
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 69078879 blocks


d0: Submirror of d10
State: Okay
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t0d0s0 0 No Okay


d1: Submirror of d10
State: Okay
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t1d0s0 0 No Okay

regards pressy

Maybe I misunderstood your post, but here is what I did. It looks like nothing is happening and I dont see anything in the logs.

Code:

# metasync d50
# metastat d50
d50: Mirror
    Submirror 0: d51
      State: Needs maintenance
    Submirror 1: d52
      State: Needs maintenance
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 65431680 blocks (31 GB)

d51: Submirror of d50
    State: Needs maintenance
    Invoke: metareplace d50 c1t0d0s5 <new device>
    Size: 65431680 blocks (31 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c1t0d0s5          0     No     Maintenance   Yes


d52: Submirror of d50
    State: Needs maintenance
    Invoke: after replacing "Maintenance" components:
                metareplace d50 c1t1d0s5 <new device>
    Size: 65431680 blocks (31 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c1t1d0s5          0     No      Last Erred   Yes


Device Relocation Information:
Device   Reloc  Device ID
c1t0d0   Yes    id1,ssd@w2000000c50568c1d
c1t1d0   Yes    id1,ssd@w2000000c50566da1

It looks to me like you lost a disk: c1t1d0s5. I'll bet that "iostat -En" will confirm that. That format command that pressy shows does look interesting, but I don't like trying to repair a disk. I would replace it.

I dont see anything strange in that output:
Also, my root mirror is complaining. It's posted in the original post. Anyhow, how can I be sure that 1) its a disk failure 2) which disk I need to replace.

With nothing showing up in iostat -En, now I think it probably isn't a bad disk. So I don't know what to tell you.

I think you need to give more info - I noticed the ssd on one of your outputs.

What type of server? Are these internal drives to the server or in arrays?
What type of arrays (if they are)?

Where are your metadb state databases (found with metadb command with no options)?

What are the failing partitions? What's on the failing partitions (OS only, OS and Applications - and of course, what applications)?

I'm assuming that SVM was the standard with Solaris 9 - if not, please post the version of it.

Also, what if anything, was changed before you noticed all of this - reboots, upgrades,...etc.?

And you state no errors in messages file - is syslogd running? Do you normally get error messages on this system? Double check that you are looking at the correct file for errors by looking at syslog.conf.