Sponsored Content
Special Forums Hardware Filesystems, Disks and Memory iostat output vs TPC output (array layer) Post 302517362 by DGPickett on Tuesday 26th of April 2011 02:59:55 PM
Old 04-26-2011
Well, it is a complex world, with security and speed in opposition. One oddity of expanding sidk sizes is that one new big disk may be overwhelmed with the level of I/O that used to be handled by 8 disks, so size attracting query and churn is a negative! Striping allows the bandwidth of many drives to be applied to the combined storage, with supports more buffering with faster buffer fills, if things are sequential often enough. If everything was sequential and failure was no worry, you could stipe all together for max bandwidth, but you might do better with 2 or more virtual volumes so copying, database joining and such can be sequential on each virtual device. So, there are sometimes ways to force smart parallelism, the ability to join huge sets without seeks. However, RAM and 64 bit VM have made buffering so ample that it may dilute that sort of approach. RAID has not entirely freed us from failure worry, since with all the layers of software and hardware and vendors, it seems RAID errors often never get heard until they are 2 devices down. Rebuild time is not inconsequential, either. So, your approach should go beyond hot spots to maximizing the bandwidth of a managable number of virtual volumes. Along the way, look at the pathways and how they figure into the redundancy and striping. If a controller handles both sides of a mirror, and goes wonky . . . . If striping runs across all controllers, scsi cables, then any controller or cable bottleneck is diluted. Intellegent use of simple mirror for high churn and RAID-N for low churn is nice, too! Sometimes, this discussion can be extended down into the app, as DB2 append tables with insert never update or delete are churn free except at the end. Disk is cheap and 100% history is wise. Churn-free data might even migrate to some hierarchical read-only store like DVD arrays. Assuming control of chaos is someone else's job can be a luxury.

But, yeah, it seems like it is still good, but might not be sufficient, and an approach with sufficiency might not make it necessary.
This User Gave Thanks to DGPickett For This Post:
 

10 More Discussions You Might Find Interesting

1. Solaris

iostat -e / -E output explanation

Hi all, hope you are having a nice day, its nice and warm today in Canberra Australia. iostat -e / -E reports soft and hard errors. Any idea what these are exactly? All I hear are I/O's failing and needing to retry, but no cause as to why they fail. My SUN guru tells me its our EMC SAN... (1 Reply)
Discussion started by: scottman
1 Replies

2. UNIX for Advanced & Expert Users

iostat output what is that mean

Hi all, i have run iostat -em, and get below result. Can i know what is this output meaning, and how to fix that problem. iostat -em ---- errors --- device s/w h/w trn tot sd7 0 1 0 1 sd8 1 1 0 2 sd9 0 1 0 1 sd10 0 ... (2 Replies)
Discussion started by: foongkt5220
2 Replies

3. Shell Programming and Scripting

output of an array

Hi gurus, I need to set up an array like this set - A arr 'A', 'B' The output of this array should be like this 'A','B' Right now, I get the output like this 'A B' Can anyone suggest me on how to achieve this. thanks (3 Replies)
Discussion started by: ragha81
3 Replies

4. Shell Programming and Scripting

Formatting output from iostat

So I use Cacti for monitoring IO statistics on my servers, now originally I couldnt monitor Multipath deviced servers as they have alot of /dev/sdxx and /dev/emcpowerxx, I have devised a method of trimming them down to just the actual devices but the issue is the output looks like so. # iostat... (0 Replies)
Discussion started by: RiSk
0 Replies

5. Shell Programming and Scripting

finding greatest value in a column using awk from iostat output in linux

Friends, . On linux i have to run iostat command and in each iteration have to print the greatest value in each column. e.g iostat -dt -kx 2 2 | awk ' !/sd/ &&!/%util/ && !/Time/ && !/Linux/ {print $12}' 4.38 0.00 0.00 0.00 What i would like to print is only the... (3 Replies)
Discussion started by: achak01
3 Replies

6. Shell Programming and Scripting

finding greatest value in a column using awk from iostat output in linux

Friends, Need some help. On linux i have to run iostat command and in each iteration have to print the greatest value in each column. e.g iostat -dt -kx 2 2 | awk ' !/sd/ &&!/%util/ && !/Time/ && !/Linux/ {print $12}' 4.38 0.00 0.00 0.00 WHhat i would like to... (15 Replies)
Discussion started by: achak01
15 Replies

7. Solaris

Unmatched ssd create huge unuseful iostat output

My scheduled collection of statistics is giving very large output because of an high number of ssd device not associated to any disk The iostat -x command is collecting statistics from them and the output is very large. I.g. if a run iostat -x|tail +3|awk '{print $1}'>f0.txt.$$ iostat... (5 Replies)
Discussion started by: sun-mik
5 Replies

8. Solaris

Asvc_t values in iostat output

Noticed that asvc_t values in iostat command outputs are mostly more than 100 in our previous iostat analysis. Also found the following detail from an alternate site IO Bottleneck - Disk performance issue - UnixArena ---- 1. asvc_t average service time of active transactions, in... (2 Replies)
Discussion started by: saraperu
2 Replies

9. Shell Programming and Scripting

Need help to get the parsed output of "iostat" command

Hi, I have a requirement where parsed output from various linux commands like top, netstat, iostat, etc. will be the input for one javascript with the parsed output from these commands converted to JSON format For "iostat" command, since there are two outputs - one w.r.t CPU utilization and... (2 Replies)
Discussion started by: gopivallabha
2 Replies

10. Shell Programming and Scripting

Need help to parse iostat command output

Hi, I got the code below is one of the threads from this forum. lineCount=$(iostat | wc -l) numDevices=$(expr $lineCount - 7); iostat $interval -x -t | awk -v awkCpuFile=$cpuFile -v awkDeviceFile=$deviceFile -v awkNumDevices=$numDevices ' BEGIN { print... (2 Replies)
Discussion started by: gopivallabha
2 Replies
vxsparecheck(1M)														  vxsparecheck(1M)

NAME
vxsparecheck - monitor Veritas Volume Manager for failure events and replace failed disks SYNOPSIS
/etc/vx/bin/vxsparecheck [mail-address...] DESCRIPTION
The vxsparecheck command monitors Veritas Volume Manager (VxVM) by analyzing the output of the vxnotify command, waiting for failures to occur. It then sends mail via mailx to the logins specified on the command line, or (by default) to root. It then replaces any failed disks. After an attempt at replacement is complete, mail will be sent indicating the status of each disk replacement. The mail notification that is sent when a failure is detected follows this format: Failures have been detected by the Veritas Volume Manager: failed disks: medianame ... failed plexes: plexname ... failed subdisks: subdiskname ... failed volumes: volumename ... The Volume Manager will attempt to find hot-spare disks to replace any failed disks and attempt to reconstruct any data in volumes that have storage on the failed disk. The medianame list specifies disks that appear to have completely failed. The plexname list show plexes of mirrored volumes that have been detached due to I/O failures experienced while attempting to do I/O to subdisks they contain. The subdiskname list specifies subdisks in RAID-5 volumes that have been detached due to I/O errors. The volumename list shows non-RAID-5 volumes that have become unusable because disks in all of their plexes have failed (and are listed in the ``failed disks'' list) and shows those RAID-5 volumes that have become unusable because of multiple failures. If any volumes appear to have failed, the following paragraph will be included in the mail: The data in the failed volumes listed above is no longer available. It will need to be restored from backup. Replacement Procedure After mail has been sent, vxsparecheck finds a hot spare replacement for any disks that appear to have failed (that is, those listed in the medianame list). This involves finding an appropriate replacement for those eligible hot spares in the same disk group as the failed disk. A disk is eligible as a replacement if it is a valid Veritas Volume Manager disk (VM disk), has been marked as a hot-spare disk and con- tains enough space to hold the data contained in all the subdisks on the failed disk. To determine which disk from among the eligible hot spares to use, vxsparecheck first checks the file /etc/vx/sparelist (see Sparelist File below). If this file does not exist or lists no eligible hot spares for the failed disk, the disk that is ``closest'' to the failed disk is chosen. The value of ``closeness'' depends on the controller, target and disk number of the failed disk. A disk on the same controller as the failed disk is closer than a disk on a different controller; and a disk under the same target as the failed disk is closer than one under a different target. If no hot spare disk can be found, the following mail is sent: No hot spare could be found for disk medianame in diskgroup. No replacement has been made and the disk is still unusable. The mail then explains the disposition of volumes that had storage on the failed disk. The following message lists disks that had storage on the failed disk, but are still usable: The following volumes have storage on medianame: volumename These volumes are still usable, but the redundancy of those volumes is reduced. Any RAID-5 volumes with storage on the failed disk may become unusable in the face of further failures. If any non-RAID-5 volumes were made unusable due to the failure of the disk, the following message is included: The following volumes: volumename have data on medianame but have no other usable mirrors on other disks. These volumes are now unusable and the data on them is unavailable. If any RAID-5 volumes were made unavailable due to the disk failure, the following message is included The following RAID-5 volumes: volumename had storage on medianame and have experienced other failures. These RAID-5 volumes are now unusable and data on them is unavailable. If a hot-spare disk was found, a hot-spare replacement is attempted. This involves associating the device marked as a hot spare with the media record that was associated with the failed disk. If this is successful, the vxrecover(1M) command is used in the background to recover the contents of any data in volumes that had storage on the disk. If the hot-spare replacement fails, the following message is sent: Replacement of disk medianame in group diskgroup failed. The error is: error message If any volumes (RAID-5 or otherwise) are rendered unusable due to the failure, the following message is included: The following volumes: volumename occupy space on the failed disk and have no other available mirrors or have experienced other failures. These volumes are unusable, and the data they contain is unavailable. If the hot-spare replacement procedure completed successfully and recovery is under way, a final mail message is sent: Replacement of disk medianame in group diskgroup with disk device sparedevice has successfully completed and recovery is under way. If any non-RAID-5 volumes were rendered unusable by the failure despite the successful hot-spare procedure, the following message is included in the mail: The following volumes: volumename occupy spare on the replaced disk, but have no other enabled mirrors on other disks from which to perform recovery. These volumes must have their data restored. If any RAID-5 volumes were rendered unusable by the failure despite the successful hot-spare procedure, the following message is included in the mail: The following RAID-5 volumes: volumename have subdisks on the replaced disk and have experienced other failures that prevent recovery. These RAID-5 volumes must have their data restored. If any volumes (RAID-5 or otherwise) were rendered unusable, the following message is also included: To restore the contents of any volumes listed above, the volume should be started with the command: vxvol -f start volumename and the data restored from backup. Sparelist File The sparelist file is a text file that specifies an ordered list of disks to be used as hot spares when a specific disk fails. The system- wide sparelist file is located in /etc/vx/sparelist. Each line in the sparelist file specifies a list of spares for one disk. Lines beginning with the pound (#) character and empty lines are ignored. The format for a line in the sparelist file is: [ diskgroup:] diskname : spare1 [ spare2 ... ] The diskgroup field, if present, specifies the disk group within which the disk and designated spares reside. If this field is not speci- fied, the default disk group is determined using the rules given in the vxdg(1M) manual page. The diskname specifies the disk for which spares are being designated. The spare list after the colon lists the disks to be used as hot spares. The list is order dependent; in case of failure of diskname, the spares are tried in order. A spare will be used only if it is a valid hot spare (see above). If the list is exhausted without finding any spares, the default policy of using the closest disk is used. FILES
/etc/vx/sparelist Specifies a list of disks to serve as hot spares for a disk. NOTES
The sparelist file is not checked in any way for correctness until a disk failure occurs. It is possible to inadvertently specify a non- existent disk or inappropriate disk or disk group. Malformed lines are also ignored. SEE ALSO
mailx(1), vxintro(1M), vxnotify(1M), vxrecover(1M), vxrelocd(1M), vxunreloc(1M) VxVM 5.0.31.1 24 Mar 2008 vxsparecheck(1M)
All times are GMT -4. The time now is 04:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy