All I can do is to guess with this kind of performance data. The asvc_t column has some really out of bounds values. Hopefully those disks with high numbers are not NFS.
You have several disks that, given the reads and writes number, are nowhere near busy enough to incur asvc_t times like that. It looks like you have databases locking resources. Or locking from something. Or NFS causing problems.
1. is this a db server?
2. are you running large reports during the times when interactive processes are also busy updating?
3. Can you try the
Code:
lockfs
command and get what it shows on those several disks? - I am assuming ufs. This will report locks. Please post the result.
I think that implementing this: DTrace Topics Locks - Siwiki might be hard to do, based on the way you asked your question, so I am going with lockfs. If the link seems like doing it will be a piece of cake, go for it by all means.
It looks like you're running Solaris Volume Manager with two RAID-1 mirrors that you're heavily oversubscribing by hitting them with an extreme number of very small random write operations. What are the actual physical disks you're writing too? They look like fiber-channel give the name format.
Look under /usr/demo/dtrace. Depending on the Solaris version you're running, you may have an whoio.d dtrace script there. I've duplicated it here:
Code:
/*
* Copyright 2005 Sun Microsystems, Inc. All rights reserved.
* Use is subject to license terms.
*
* This D script is used as an example in the Solaris Dynamic Tracing Guide
* wiki in the "io Provider" Chapter.
*
* The full text of the this chapter may be found here:
*
* https://wikis.oracle.com/display/DTrace/io+Provider
*
* On machines that have DTrace installed, this script is available as
* whoio.d in /usr/demo/dtrace, a directory that contains all D scripts
* used in the Solaris Dynamic Tracing Guide. A table of the scripts and their
* corresponding chapters may be found here:
*
* file:///usr/demo/dtrace/index.html
*/
#pragma D option quiet
io:::start
{
@[args[1]->dev_statname, execname, pid] = sum(args[0]->b_bcount);
}
END
{
printf("%10s %20s %10s %15s\n", "DEVICE", "APP", "PID", "BYTES");
printa("%10s %20s %10d %15@d\n", @);
}
Copy that to a file, such as whoio.d, and run it as root:
Code:
dtrace -s whoio.d
Let it run a while, then hit CTRL-C to see what processes are hitting which device with IO requests. You'll have to translate names such as "sd4" to actual devices.
Hi:
I have a program written in FORTRAN running on AIX platform.
It is because missing of documentation and without root password, therefore we want to modify the program so that we can find out which script/program that call this FORTRAN program.
I have google for few days, all of them are... (3 Replies)
Hello Everyone,
There is a column kthr:b in vmstat. How a process become blocked?
If the process calls system call pread and sleeps inside it, is the process blocked in that moment?
How a process sleeping because it has no work at all (as Notepad when we are not using it) differs from a... (3 Replies)
Hi All,
Heres a little background. We have essbase installed on a solaris server. We are running a report script. The script starts and runs ok..
1. after some time if i do the ps -ef i can see the process, and it generally completes successfully.
2. Most of the times, when i do the ps with... (2 Replies)
Hi reader,
I'm making a tool out of korn shell script that is running on a HP-UX server. But everytime i invoke the tool, it gets killed after a while (mid-process). I have tried re-running it a couple of times but each invocation ending up the same way ..
following is a snippet of the o/p... (8 Replies)
hi
i'm running a shell script that checks the amount of cpu idle either using /usr/bin/vmstat 1 2 or sar 1 2 (on unixware) before i run some tests(if cpu idle greater than 89 I run them).
These tests are run on many platforms, linux(suse, redhat) hp-ux, unixware, aix, solaris, tru64.
... (5 Replies)
Hi,
Can anyone help me out in writing the shell scrip which monitors a process which is running and gives me the output of the memory being used by the process, I have the requirement of monitorig the memory usage of the process when it is running.
Please help me out (3 Replies)
Due to the massive Upload speeds killing .... or overstressing our schools network...... my school has blocked port 6969 (the most common BitTorrent port). So I cant connect to the tracker anymore, in other words no more downloading from school :(
Does anyone know how I can get around the ports... (1 Reply)
