Sponsored Content
Special Forums Hardware Filesystems, Disks and Memory iostat output vs TPC output (array layer) Post 302517648 by DGPickett on Wednesday 27th of April 2011 11:17:56 AM
Old 04-27-2011
The SAN adds latency, which can be a problem for things like RDBMS commit where files are sync'd to media. Sometimes they just pretend it is on oxide and rely on battery backup, which might be OK, but be aware.

You can flow a lot of data per second in and out, but random might be a problem unless RAM cached. Some apps get more RAM caching by using mmap()/mmap64() to map files to VM -- no swap hit, just RAM or page fault to disk, which is how most OS do dynamic libraries and some OS have started doing this in kernel caching and inside standard library calls for all I/O. Huge random sets traversed once still run slow and waste RAM in the bargain, never mind SAN cache. If you have such latency problems, you may want to put those files on some sort of more local volume, even SSM. One way to have SAN backup is to have a hierarchical bit like the Sun ClientFS that backs all modified pages on the local disk to the SAN, using the local disk as a intermediate level cache. This handles larger volumes than most RAM budgets with less worry about right sizing, since you cannot run out of space on the local disk. The SAN load of backing it up can be tuned, so you do not have to write the same page twice if it gets only two mods close together.


However, while this solves random woes, for RDBMS commit sync, you may want either:
  • a solid (not cache) local allocation (mirrored on two controllers, not RAIDN) striped fast disk or SSM for speed or
  • to take the hit on sync to SAN, for low churn very random sets.
Luckily, many OS and RDBMS utilities, turned on or frequently applied, can optimize things so they are more often serial in a fast way, like range scans in an index or table scans in smaller tables. Be careful to apply them in the right order, as one might undo the other. Usually, RDBMS are on raw partitions, so there is no conflict; you just need to do both OS utilities for the non-raw and RDBMS utilities for their files/partitions. As I said, tricks like the APPEND table, or similar physical keying, ensure no holes and no churn of old pages, immediately or once defragged.
 

10 More Discussions You Might Find Interesting

1. Solaris

iostat -e / -E output explanation

Hi all, hope you are having a nice day, its nice and warm today in Canberra Australia. iostat -e / -E reports soft and hard errors. Any idea what these are exactly? All I hear are I/O's failing and needing to retry, but no cause as to why they fail. My SUN guru tells me its our EMC SAN... (1 Reply)
Discussion started by: scottman
1 Replies

2. UNIX for Advanced & Expert Users

iostat output what is that mean

Hi all, i have run iostat -em, and get below result. Can i know what is this output meaning, and how to fix that problem. iostat -em ---- errors --- device s/w h/w trn tot sd7 0 1 0 1 sd8 1 1 0 2 sd9 0 1 0 1 sd10 0 ... (2 Replies)
Discussion started by: foongkt5220
2 Replies

3. Shell Programming and Scripting

output of an array

Hi gurus, I need to set up an array like this set - A arr 'A', 'B' The output of this array should be like this 'A','B' Right now, I get the output like this 'A B' Can anyone suggest me on how to achieve this. thanks (3 Replies)
Discussion started by: ragha81
3 Replies

4. Shell Programming and Scripting

Formatting output from iostat

So I use Cacti for monitoring IO statistics on my servers, now originally I couldnt monitor Multipath deviced servers as they have alot of /dev/sdxx and /dev/emcpowerxx, I have devised a method of trimming them down to just the actual devices but the issue is the output looks like so. # iostat... (0 Replies)
Discussion started by: RiSk
0 Replies

5. Shell Programming and Scripting

finding greatest value in a column using awk from iostat output in linux

Friends, . On linux i have to run iostat command and in each iteration have to print the greatest value in each column. e.g iostat -dt -kx 2 2 | awk ' !/sd/ &&!/%util/ && !/Time/ && !/Linux/ {print $12}' 4.38 0.00 0.00 0.00 What i would like to print is only the... (3 Replies)
Discussion started by: achak01
3 Replies

6. Shell Programming and Scripting

finding greatest value in a column using awk from iostat output in linux

Friends, Need some help. On linux i have to run iostat command and in each iteration have to print the greatest value in each column. e.g iostat -dt -kx 2 2 | awk ' !/sd/ &&!/%util/ && !/Time/ && !/Linux/ {print $12}' 4.38 0.00 0.00 0.00 WHhat i would like to... (15 Replies)
Discussion started by: achak01
15 Replies

7. Solaris

Unmatched ssd create huge unuseful iostat output

My scheduled collection of statistics is giving very large output because of an high number of ssd device not associated to any disk The iostat -x command is collecting statistics from them and the output is very large. I.g. if a run iostat -x|tail +3|awk '{print $1}'>f0.txt.$$ iostat... (5 Replies)
Discussion started by: sun-mik
5 Replies

8. Solaris

Asvc_t values in iostat output

Noticed that asvc_t values in iostat command outputs are mostly more than 100 in our previous iostat analysis. Also found the following detail from an alternate site IO Bottleneck - Disk performance issue - UnixArena ---- 1. asvc_t average service time of active transactions, in... (2 Replies)
Discussion started by: saraperu
2 Replies

9. Shell Programming and Scripting

Need help to get the parsed output of "iostat" command

Hi, I have a requirement where parsed output from various linux commands like top, netstat, iostat, etc. will be the input for one javascript with the parsed output from these commands converted to JSON format For "iostat" command, since there are two outputs - one w.r.t CPU utilization and... (2 Replies)
Discussion started by: gopivallabha
2 Replies

10. Shell Programming and Scripting

Need help to parse iostat command output

Hi, I got the code below is one of the threads from this forum. lineCount=$(iostat | wc -l) numDevices=$(expr $lineCount - 7); iostat $interval -x -t | awk -v awkCpuFile=$cpuFile -v awkDeviceFile=$deviceFile -v awkNumDevices=$numDevices ' BEGIN { print... (2 Replies)
Discussion started by: gopivallabha
2 Replies
bcopy(9F)						   Kernel Functions for Drivers 						 bcopy(9F)

NAME
bcopy - copy data between address locations in the kernel SYNOPSIS
#include <sys/types.h> #include <sys/sunddi.h> void bcopy(const void *from, void *to, size_t bcount); INTERFACE LEVEL
Architecture independent level 1 (DDI/DKI). PARAMETERS
from Source address from which the copy is made. to Destination address to which copy is made. bcount The number of bytes moved. DESCRIPTION
bcopy() copies bcount bytes from one kernel address to another. If the input and output addresses overlap, the command executes, but the results may not be as expected. Note that bcopy() should never be used to move data in or out of a user buffer, because it has no provision for handling page faults. The user address space can be swapped out at any time, and bcopy() always assumes that there will be no paging faults. If bcopy() attempts to access the user buffer when it is swapped out, the system will panic. It is safe to use bcopy() to move data within kernel space, since kernel space is never swapped out. CONTEXT
bcopy() can be called from user or interrupt context. EXAMPLES
Example 1: Copying data between address locations in the kernel: An I/O request is made for data stored in a RAM disk. If the I/O operation is a read request, the data is copied from the RAM disk to a buffer (line 8). If it is a write request, the data is copied from a buffer to the RAM disk (line 15). bcopy() is used since both the RAM disk and the buffer are part of the kernel address space. 1 #define RAMDNBLK 1000 /* blocks in the RAM disk */ 2 #define RAMDBSIZ 512 /* bytes per block */ 3 char ramdblks[RAMDNBLK][RAMDBSIZ]; /* blocks forming RAM /* disk ... 4 5 if (bp->b_flags & B_READ) /* if read request, copy data */ 6 /* from RAM disk data block */ 7 /* to system buffer */ 8 bcopy(&ramdblks[bp->b_blkno][0], bp->b_un.b_addr, 9 bp->b_bcount); 10 11 else /* else write request, */ 12 /* copy data from a */ 13 /* system buffer to RAM disk */ 14 /* data block */ 15 bcopy(bp->b_un.b_addr, &ramdblks[bp->b_blkno][0], 16 bp->b_bcount); SEE ALSO
copyin(9F), copyout(9F) Writing Device Drivers WARNINGS
The from and to addresses must be within the kernel space. No range checking is done. If an address outside of the kernel space is selected, the driver may corrupt the system in an unpredictable way. SunOS 5.10 4 August 2003 bcopy(9F)
All times are GMT -4. The time now is 06:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy