Sponsored Content
Operating Systems AIX Performance issue / tuning advice Post 302480507 by zaxxon on Wednesday 15th of December 2010 06:52:06 AM
Old 12-15-2010
I do not know SAS - I could see it is some Business Intelligence Software, which I guess can consist of different kinds of applications. More information would be interessting, if there is any type of database is involved, webservers etc.

Has anybody tuned this system already? There is a guide I just found which might be interessting for you:
http://www.sas.com/partners/director...stPractice.pdf

What does really not look good is:
  1. kthr/r Run Queue -- Did you try exporting AIXTHREAD_SCOPE=S? It looks to me, as if there are just far too few CPU resources to handle all ready-to-run threads. It would be really interessting to see, how the system behaves if you could add 3 more physical CPU units. Also it would be interessting, if your application benefits from partitioning your 3 physical CPUs to 6 virtual CPUs which would result with SMT turned on (which is the case in your system) to 12 logical CPUs instead of just 6.
  2. For scanning memory pages, the ratio is ok, example fr: 83624 sr: 126816 which is about 1:1,5. Though it is a lot of free pages the system demands and so with has to scan. Beside that it nearly always has about 80k pages on the free list. Could you please post the output of vmo -x minfree -x maxfree, thanks. Best add the output of vmo -x lru_file_repage too, just in case. Also the output of svmon -G might be interessting.
  3. What is definetly tunable, is the bad high counter for
    596244370 external pager filesystem I/Os blocked with no fsbuf. Setting the double of your current (check with ioo -x j2_dynamicBufferPreallocation) or if it is still 16 which is default, try 128 instead and check over time, if counter is still increasing by large number. You can also tune the ioo -x j2_nBufferPerPagerDevice to let's say 2048 and check if the counter increases rapidly still. It will take some time to monitor it. Check the man page for ioo, which of the 2 parameters can be changed dynamic and which needs a remounts of the filesystems.
  4. Currently your box is not paging in or out to Paging Space, but you have this: 731746 paging space I/Os blocked with no psbuf
    which shows, there has been such activity. I bet a lsps -a will show, that some of your Paging Space is used. If the output of ioo -x lru_file_repage turns out to be 0 already, then I would add some more memory to this box.

So far my 1st ideas for your box. Would be nice to hear any feedback if anything of it helped.

Last edited by zaxxon; 12-15-2010 at 07:57 AM.. Reason: typos, formatting
This User Gave Thanks to zaxxon For This Post:
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Performance tuning.

can someone tell me a good site to go to in order to learn this. please do not recommen nay books because i dont have interest in that. if you know of any good sites with good straight forward explanation on how to split loads on machines that has excessive loading, please let me know Also,... (1 Reply)
Discussion started by: TRUEST
1 Replies

2. Filesystems, Disks and Memory

EXT3 Performance tuning

Hi all, long time ago I posted something, but now, it is needed again :( Currently, I am handling with a big NFS Server for more than 200 clients, this sever has to work with 256 NFSDs. Because of this huge amount of NFSDs, there are thousands of small write accesses down to the disk and... (3 Replies)
Discussion started by: malcom
3 Replies

3. UNIX for Dummies Questions & Answers

Performance Tuning

Hi to all, I'm interested in finding an introduction about Performance Tuning under Unix (or Linux); can somebody please point me in the right direction? Best regards (1 Reply)
Discussion started by: domyalex
1 Replies

4. Shell Programming and Scripting

Oracle-performance tuning

Sorry, This is out of scope of this group.But I require the clarification pretty urgently. My Oracle database is parallely enabled. Still,in a particular table queries do not work "parallely" always. How is this? (9 Replies)
Discussion started by: kthri
9 Replies

5. Shell Programming and Scripting

Performance Tuning

Hi All, In last one week, i have posted many questions in this portal. At last i am succeeded to make my 1st unix script. following are 2 points where my script is taking tooooo long. 1. Print the total number of records excluding header & footer. I have found that awk 'END{print NR -... (2 Replies)
Discussion started by: Amit.Sagpariya
2 Replies

6. UNIX for Advanced & Expert Users

Performance diagnosis & tuning

Hi, I am facing a strange issue. Application is deployed in a cluster with 2 Unix nodes (with same configuration). On one node the application is working fine but on another node we see this behavior I found using vmstat- when the server is not yet started everything is OK; when you start the... (3 Replies)
Discussion started by: ash.abrol
3 Replies

7. Solaris

Solaris Performance tuning

Dear all, I have a Local zone , where users feel that performance is not good. Is it wise to collect the inputs from the local zone rather than taking from the global zone. And also Can I tune from Global zone , so that it will reflect in local zone. Rgds rj (2 Replies)
Discussion started by: jegaraman
2 Replies

8. Tips and Tutorials

The Most Incomplete Guide to Performance Tuning

Overview: Introduction What Does Success Mean? What Does Performance Mean? Every Picture is Worth a Thousand Words Work Like a Physicist Work Like You Walk - One Step at a Time Learn to Know Your System Choose Your Weapons! Tools of the Trade 1 - vmstat A Little Theory Along the Way -... (1 Reply)
Discussion started by: bakunin
1 Replies

9. AIX

IBM AIX I/O Performance Tuning

I have a IBM Power9 server coupled with a NVMe StorWize V7000 GEN3 storage, doing some benchmarks and noticing that single thread I/O (80% Read / 20% Write, common OLTP I/O profile) seems slow. ./xdisk -R0 -r80 -b 8k -M 1 -f /usr1/testing -t60 -OD -V BS Proc AIO read% IO Flag IO/s ... (8 Replies)
Discussion started by: c3rb3rus
8 Replies
vmstat(1M)						  System Administration Commands						vmstat(1M)

NAME
vmstat - report virtual memory statistics SYNOPSIS
vmstat [-cipqsS] [disks] [interval [count]] DESCRIPTION
vmstat reports virtual memory statistics regarding kernel thread, virtual memory, disk, trap, and CPU activity. On MP (multi-processor) systems, vmstat averages the number of CPUs into the output. For per-processor statistics, see mpstat(1M). vmstat only supports statistics for certain devices. For more general system statistics, use sar(1), iostat(1M), or sar(1M). Without options, vmstat displays a one-line summary of the virtual memory activity since the system was booted. During execution of the kernel status command, the state of the system can change. If relevant, a state change message is included in the vmstat output, in one of the following forms: <<device added: sd0>> <<device removed: sd0>> <<processors added: 1, 3>> <<processors removed: 1, 3>> See for device naming conventions for disks. OPTIONS
The following options are supported: -c Report cache flushing statistics. This option is obsolete, and no longer meaningful. This option might be removed in a future version of Solaris. -i Report the number of interrupts per device. count and interval does not apply to the -i option. -p Report paging activity in details. This option will display the following, respectively: epi Executable page-ins. epo Executable page-outs. epf Executable page-frees. api Anonymous page-ins. apo Anonymous page-outs. apf Anonymous page-frees. fpi File system page-ins. fpo File system page-outs. fpf File system page-frees. When executed in a zone and if the pools facility is active, all of the above only report actitivity on the processors in the proces- sor set of the zone's pool. -q Suppress messages related to state changes. -s Display the total number of various system events since boot. count and interval does not apply to the -s option. -S Report on swapping rather than paging activity. This option will change two fields in vmstat's ``paging'' display: rather than the ``re'' and ``mf'' fields, vmstat will report ``si'' (swap-ins) and ``so'' (swap-outs). OPERANDS
The following operands are supported: count Specifies the number of times that the statistics are repeated. count does not apply to the -i and -s options. disks Specifies which disks are to be given priority in the output (only four disks fit on a line). Common disk names are id, sd, xd, or xy, followed by a number (for example, sd2, xd0, and so forth). interval Specifies the last number of seconds over which vmstat summarizes activity. This number of seconds repeats forever. interval does not apply to the -i and -s options. EXAMPLES
Example 1 Using vmstat The following command displays a summary of what the system is doing every five seconds. example% vmstat 5 kthr memory page disk faults cpu r b w swap free re mf pi p fr de sr s0 s1 s2 s3 in sy cs us sy id 0 0 0 11456 4120 1 41 19 1 3 0 2 0 4 0 0 48 112 130 4 14 82 0 0 1 10132 4280 0 4 44 0 0 0 0 0 23 0 0 211 230 144 3 35 62 0 0 1 10132 4616 0 0 20 0 0 0 0 0 19 0 0 150 172 146 3 33 64 0 0 1 10132 5292 0 0 9 0 0 0 0 0 21 0 0 165 105 130 1 21 78 1 1 1 10132 5496 0 0 5 0 0 0 0 0 23 0 0 183 92 134 1 20 79 1 0 1 10132 5564 0 0 25 0 0 0 0 0 18 0 0 131 231 116 4 34 62 1 0 1 10124 5412 0 0 37 0 0 0 0 0 22 0 0 166 179 118 1 33 67 1 0 1 10124 5236 0 0 24 0 0 0 0 0 14 0 0 109 243 113 4 56 39 ^C example% The fields of vmstat's display are kthr Report the number of kernel threads in each of the three following states: r the number of kernel threads in run queue b the number of blocked kernel threads that are waiting for resources I/O, paging, and so forth w the number of swapped out lightweight processes (LWPs) that are waiting for processing resources to finish. memory Report on usage of virtual and real memory. swap available swap space (Kbytes) free size of the free list (Kbytes) page Report information about page faults and paging activity. The information on each of the following activities is given in units per second. re page reclaims -- but see the -S option for how this field is modified. mf minor faults -- but see the -S option for how this field is modified. pi kilobytes paged in po kilobytes paged out fr kilobytes freed de anticipated short-term memory shortfall (Kbytes) sr pages scanned by clock algorithm When executed in a zone and if the pools facility is active, all of the above (except for "de") only report activity on the pro- cessors in the processor set of the zone's pool. disk Report the number of disk operations per second. There are slots for up to four disks, labeled with a single letter and number. The letter indicates the type of disk (s = SCSI, i = IPI, and so forth); the number is the logical unit number. faults Report the trap/interrupt rates (per second). in interrupts sy system calls cs CPU context switches When executed in a zone and if the pools facility is active, all of the above only report actitivity on the processors in the processor set of the zone's pool. cpu Give a breakdown of percentage usage of CPU time. On MP systems, this is an average across all processors. us user time sy system time id idle time When executed in a zone and if the pools facility is active, all of the above only report actitivity on the processors in the processor set of the zone's pool. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |See below. | +-----------------------------+-----------------------------+ Invocation is evolving. Human readable output is unstable. SEE ALSO
sar(1), iostat(1M), mpstat(1M), sar(1M), attributes(5) NOTES
The sum of CPU utilization might vary slightly from 100 because of rounding errors in the production of a percentage figure. The -c option (Report cache flushing statistics) is not supported in this release. SunOS 5.11 20 Dec 2004 vmstat(1M)
All times are GMT -4. The time now is 04:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy