Sponsored Content
Operating Systems Solaris zfs raidz2 - insufficient replicas Post 302584000 by skk on Thursday 22nd of December 2011 07:36:27 AM
Old 12-22-2011
I thought I was OK, but zpool scrub hangs forever at 20% across multiple cold boots, and importing from oracle solaris 11 live hangs forever... I what do I do now, older versions of open solaris will not import the pool anymore...

---------- Post updated at 07:36 AM ---------- Previous update was at 12:48 AM ----------

Well it's 4AM and now I am getting mad. I think this whole bloody mess is caused by Oracle's 'great new solaris 11' package. My advice, don't touch this piece.

To recap, I wanted to try the best newest Solaris release for my super critical file server, so I downloaded from Oracle what I thought was the right iso. I booted, and just as the boot menu came up I got a phone call. When I came back the thing had automatically installed a new version of solaris over one of the drives in the zpool. (BAD #1 ).

Since I have a raidz2 I wasn't too worried ( at first ) and I booted into my original 2009 opensolaris. However, I got the errors shown in the above posts. I exported and could not re-import my pool. Oracale had done something to that drive breaking my entire pool even though it is a radz2 and should be tolerant of 2 drive failures at the same time. ( BAD #2 ).

Since I could do nothing with my pool with my old OS, I tried on the new Oracle solaris, and could indeed import my pool in degraded state because of the one overwritten drive. Fine, I wanted to scrub everything first ( I don't know if this was wise or not ) so I did zpool scrub, which eventually hung forever at 11%. All access to the drive was similarly hung. Rebooting the machine did not change this situation which seemed increasingly dire ( BAD #3 )

I finally got out of this problem by unplugging drives to fault the pool and rebooting in single user mode. Eventually I was able to stop the scrub with the "zpool scrub -s" command, in single user mode. And when I rebooted, I could access my pool again. My first priority at this point was to back up all data immediately. I began to copy off my most important stuff, but unfortunately before I could copy even a fraction off the file system hung again ( BAD #4 ).

Googling around I found most causes for hanging zpool commands are related to hardware failure, so going on a hunch, I figured Oracle phased out or screwed up the drivers for my disks. I still could not import my pool in the old opensolaris OS, because of whatever the Oracle install wrote on that drive. So I booted in Oracle solaris, in single user mode, and did "zpool offline <pool> <drive>" and it worked! Then I rebooted into good old opensolaris, and imported my pool. It worked!!!!

So at this point it is now more like 5AM and I have backed up most of my critical data, way more than I could before anyway. It appears at this point that I was correct and the drivers for either my motherboard or my hard drive controller card were broken by the Oracle release in a way that let it silently trash my zpool. I have a SIIG 2 drive sata and a MSI n1996 motherboard, not sure which the problem is with, but whichever, it works fine in opensolaris 2009 and previous versions.

I just want to warn people that are not real Solaris experts from even trying this Oracle package. Personally I am migrating to fbsd as soon as I can...
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

SDS and replicas

Hello, We are using Solstice Disk Suite on Solaris 2.7. We want to add two striped volume with six disks. On each disk, we take a slice and we create the stripe. That I want to know : Is it necessary to add two replicas on the same slice on the new disks, as made before on the others... (1 Reply)
Discussion started by: christophe
1 Replies

2. HP-UX

Insufficient permissions on ftp'ed files from WIN2K to HP-UX

We have an application running on Win2K and this application ftp files to HP-UX using ftpdc as user id. The files are created in HP-UX with following permissions: -rw-r----- 1 ftpdc users 968321 Apr 12 22:57 aaaa.txt There is a job that runs on HP-UX trying to modify this file using the... (7 Replies)
Discussion started by: Anamika
7 Replies

3. Solaris

insufficient metadevice database replicas ERROR

First I would like to thank this forum for assisting me in setting up my 1st sunbox. Could not have done it if it had not been for you guys and google :D I have mirrored my box and have SUCCESSFULLY tested booting from the rootdisk and rootmirror successfully. I am now looking at configuring... (2 Replies)
Discussion started by: mr_manny
2 Replies

4. Solaris

Problem with DiskSuite replicas

Good morning, I have Solstice disk suite installed on my server. One disk broke so I sostitute it. A replica was present on this disk. I had delete and then recreate it with commands metadb -d and metadb -a. Now when I inquire for the status of replicas I see this: stp11# metadb -i ... (2 Replies)
Discussion started by: bonovox
2 Replies

5. UNIX for Advanced & Expert Users

insufficient available memory

Hi, when navigating on application webpages (peoplesoft applications) the users receive : INSUFFICIENT AVAILABLE MEMORY. I issued vmstat on UNIX server ( where is hosted Web server and application server and DB). Here is the result : :vmstat 2 10 System configuration: lcpu=4 mem=30720MB... (8 Replies)
Discussion started by: big123456
8 Replies

6. UNIX for Advanced & Expert Users

Issue with insufficient swap or memory space

Hi, When I execute one of my shellscript I am getting the below mentioned error message .This application takes 2input files which have the records counts 26463 and 1178046 exec(2): insufficient swap or memory available. exec(2): insufficient swap or memory available. exec(2): insufficient swap... (3 Replies)
Discussion started by: kavithakuttyk
3 Replies

7. Solaris

metadevices: how to test metadb (how to corrupt replicas/understanding replicas)

Hi all, I recently started exploring Solaris 10. I am testing metadevices now. I have been reading about the state databases here: 6.State Database (Overview) (Solaris Volume Manager Administration Guide) - Sun Microsystems So I created 3 metadbs on 2 slices (6 in total; c1t1d0s3... (3 Replies)
Discussion started by: deadeyes
3 Replies

8. Solaris

Solaris Volume Manger - Database Replicas Question - Benefits of Increasing Default Size?

Hey all! I was hoping someone knew anything about this one... I know with Solaris Volume Manager the default Database Replica size is 8192 blocks (4MB approximately) Now I know you can increase this amount but is there any point? The reason I am asking this is that I've setup mirroring on... (2 Replies)
Discussion started by: Keepcase
2 Replies

9. Solaris

13 disk raidz2 pool lost

Hi guys, I appreciate any help in this regard, we have lost sensitive data in the company. One box with 2 disk mirrored and a 3ware controller handling 13 disks in a raidz2 pool. Suddenly the box restart and keeps "Reading ZFS config" for hours. Unplugging disk by disk we isolate the disk... (3 Replies)
Discussion started by: tatxo
3 Replies
MPTUTIL(8)						    BSD System Manager's Manual 						MPTUTIL(8)

NAME
mptutil -- Utility for managing LSI Fusion-MPT controllers SYNOPSIS
mptutil version mptutil [-u unit] show adapter mptutil [-u unit] show config mptutil [-u unit] show drives mptutil [-u unit] show events mptutil [-u unit] show volumes mptutil [-u unit] fail drive mptutil [-u unit] online drive mptutil [-u unit] offline drive mptutil [-u unit] name volume name mptutil [-u unit] volume status volume mptutil [-u unit] volume cache volume enable|disable mptutil [-u unit] clear mptutil [-u unit] create type [-q] [-v] [-s stripe_size] drive[,drive[,...]] mptutil [-u unit] delete volume mptutil [-u unit] add drive [volume] mptutil [-u unit] remove drive DESCRIPTION
The mptutil utility can be used to display or modify various parameters on LSI Fusion-MPT controllers. Each invocation of mptutil consists of zero or more global options followed by a command. Commands may support additional optional or required arguments after the command. Currently one global option is supported: -u unit unit specifies the unit of the controller to work with. If no unit is specified, then unit 0 is used. Volumes may be specified in two forms. First, a volume may be identified by its location as [xx:]yy where xx is the bus ID and yy is the target ID. If the bus ID is omitted, the volume is assumed to be on bus 0. Second, on the volume may be specified by the corresponding daX device, such as da0. The mpt(4) controller divides drives up into two categories. Configured drives belong to a RAID volume either as a member drive or as a hot spare. Each configured drive is assigned a unique device ID such as 0 or 1 that is show in show config, and in the first column of show drives. Any drive not associated with a RAID volume as either a member or a hot spare is a standalone drive. Standalone drives are visible to the operating system as SCSI disk devices. As a result, drives may be specified in three forms. First, a configured drive may be identi- fied by its device ID. Second, any drive may be identified by its location as xx:yy where xx is the bus ID and yy is the target ID for each drive as displayed in show drives. Note that unlike volumes, a drive location always requires the bus ID to avoid confusion with device IDs. Third, a standalone drive that is not part of a volume may be identified by its corresponding daX device as displayed in show drives. The mptutil utility supports several different groups of commands. The first group of commands provide information about the controller, the volumes it manages, and the drives it controls. The second group of commands are used to manage the physical drives attached to the con- troller. The third group of commands are used to manage the logical volumes managed by the controller. The fourth group of commands are used to manage the drive configuration for the controller. The informational commands include: version Displays the version of mptutil. show adapter Displays information about the RAID controller such as the model number. show config Displays the volume and drive configuration for the controller. Each volume is listed along with the physical drives that the volume spans. If any hot spare drives are configured, then they are listed as well. show drives Lists all of the physical drives attached to the controller. show events Display all the entries from the controller's event log. Due to lack of documentation this command is not very useful currently and just dumps each log entry in hex. show volumes Lists all of the logical volumes managed by the controller. The physical drive management commands include: fail drive Mark drive as ``failed requested''. Note that this state is different from the ``failed'' state that is used when the firmware fails a drive. Drive must be a configured drive. online drive Mark drive as an online drive. Drive must be part a configured drive in either the ``offline'' or ``failed requested'' states. offline drive Mark drive as offline. Drive must be a configured, online drive. The logical volume management commands include: name volume name Sets the name of volume to name. volume cache volume enable|disable Enables or disables the drive write cache for the member drives of volume. volume status volume Display more detailed status about a single volume including the current progress of a rebuild operation if one is being performed. The configuration commands include: clear Delete the entire configuration including all volumes and spares. All drives will become standalone drives. create type [-q] [-v] [-s stripe_size] drive[,drive[,...]] Create a new volume. The type specifies the type of volume to create. Currently supported types include: raid0 Creates one RAID0 volume spanning the drives listed in the single drive list. raid1 Creates one RAID1 volume spanning the drives listed in the single drive list. raid1e Creates one RAID1E volume spanning the drives listed in the single drive list. Note: Not all volume types are supported by all controllers. If the -q flag is specified after type, then a ``quick'' initialization of the volume will be done. This is useful when the drives do not contain any existing data that need to be preserved. If the -v flag is specified after type, then more verbose output will be enabled. Currently this just provides notification as drives are added to volumes when building the configuration. The -s stripe_size parameter allows the stripe size of the array to be set. By default a stripe size of 64K is used. The list of valid values for a given type are listed in the output of show adapter. delete volume Delete the volume volume. Member drives will become standalone drives. add drive [volume] Mark drive as a hot spare. Drive must not be a member of a volume. If volume is specified, then the hot spare will be dedicated to that volume. Otherwise, drive will be used as a global hot spare backing all volumes for this controller. Note that drive must be as large as the smallest drive in all of the volumes it is going to back. remove drive Remove the hot spare drive from service. It will become a standalone drive. EXAMPLES
Mark the drive at bus 0 target 4 as offline: mptutil offline 0:4 Create a RAID1 array from the two standalone drives da1 and da2: mptutil create raid1 da1,da2 Mark standalone drive da3 as a global hot spare: mptutil add da3 SEE ALSO
mpt(4) HISTORY
The mptutil utility first appeared in FreeBSD 8.0. BUGS
The handling of spare drives appears to be unreliable. The mpt(4) firmware manages spares via spare drive ``pools''. There are eight pools numbered 0 through 7. Each spare drive can only be assigned to a single pool. Each volume can be backed by any combination of zero or more spare pools. The mptutil utility attempts to use the following algorithm for managing spares. Global spares are always assigned to pool 0, and all volumes are always backed by pool 0. For dedicated spares, mptutil assigns one of the remaining 7 pools to each volume and assigns dedicated drives to that pool. In practice however, it seems that assigning a drive as a spare does not take effect until the box has been rebooted. Also, the firmware renumbers the spare pool assignments after a reboot which undoes the effects of the algorithm above. Simple cases such as assigning global spares seem to work ok (albeit requiring a reboot to take effect) but more ``exotic'' configurations may not work reliably. Drive configuration commands result in an excessive flood of messages on the console. The mpt version 1 API that is used by mptutil and mpt(4) does not support volumes above two terabytes. This is a limitation of the API. If you are using this adapter with volumes larger than two terabytes, use the adapter in JBOD mode. Utilize geom(8), zfs(8), or another soft- ware volume manager to work around this limitation. BSD
August 16, 2009 BSD
All times are GMT -4. The time now is 06:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy