Sponsored Content
Operating Systems AIX Lots of page faults and free memory Post 302380443 by flpgdt on Tuesday 15th of December 2009 07:50:03 AM
Old 12-15-2009
Lots of page faults and free memory

Hello,

I've been reading your forums for quite a while and the great amount of information I find here always come in hand.This time however, I need some specific help...

I have a doubt with an AIX server which I'm failing to understand as I'm new to its concept of memory management...

Straight to the point, I have a sever which yields high number of page faults even though it has plenty of available memory.

This server runs a file-reading-intensive program and an oracle database. I have no serious performance problems so far, but these page faults started to worry me as we plan to stuff some more tasks into this 40GB men server.

I first went on and did my homework reading about AIX VMM (this is my first time with AIX servers Smilie ) and got a glance of it's peculiar way of paging everything files and programs and the way it uses a deamon to steal and clean whenever it runs short. Well... I come from Solaris, where short memory causes pages faults. Page faults causes scan-rate and scan-rate most-likely means paging. When I issue my vmstat and see the 'sr' column with 4 digit numbers in a production server, it feels... wrong...

Now, I'll past below my stats.. could someone give me more perspective of what I'm seeing?

System (from nmon startup):
Code:
│                               6 - CPUs currently                   │
│                               6 - CPUs configured                  │
│                            1900 - MHz CPU clock rate               │
│                  PowerPC_POWER5 - Processor                        │
│                          64 bit - Hardware                         │
│                          64 bit - Kernel                           │
│                         Dynamic - Logical Partition                │
│                    5.3.7.1 ML07 - AIX Kernel Version               │

Code:
$ vmstat 10 5
 r b avm fre re pi po fr sr cy in sy cs us sy id wa pc ec
 3 1 6892117 4855669 0 1 0 543 3695 0 3333 26670 12425 82 5 13 0 2.30 91.9
 5 0 6892111 4855614 0 0 0 439 3895 0 3245 74660 12366 77 13 10 0 2.37 94.9
 3 0 6891110 4856689 0 0 0 635 4806 0 3129 44884 12170 80 6 14 0 2.29 91.4
 4 0 6891517 4856185 0 0 0 504 4241 0 3208 41366 13178 80 6 14 0 2.29 91.5
 3 0 6891105 4856693 0 0 0 388 2059 0 3162 27696 13502 82 5 12 0 2.33 93.1

Code:
$ vmstat -v
             12582896 memory pages
             11905361 lruable pages
              4858699 free pages
                    1 memory pools
              3055581 pinned pages
                 80.0 maxpin percentage
                 10.0 minperm percentage
                 20.0 maxperm percentage
                 19.9 numperm percentage
              2379986 file pages
                  0.0 compressed percentage
                    0 compressed pages
                 19.9 numclient percentage
                 20.0 maxclient percentage
              2379986 client pages
                    0 remote pageouts scheduled
                 5114 pending disk I/Os blocked with no pbuf
               151149 paging space I/Os blocked with no psbuf
                 2484 filesystem I/Os blocked with no fsbuf
                41094 client filesystem I/Os blocked with no fsbuf
                 8101 external pager filesystem I/Os blocked with no fsbuf
                    0 Virtualized Partition Memory Page Faults
                 0.00 Time resolving virtualized partition memory page faults

nmon shot (memory and paging):
Code:
│ Memory ────────────────────────────────────────────────────────────────────────│
│          Physical  PageSpace |        pages/sec  In     Out | FileSystemCache  │
│% Used       62.6%     40.5%  | to Paging Space   0.0    0.0 | (numperm) 18.7%  │
│% Free       37.4%     59.5%  | to File System    0.0  207.8 | Process   21.6%  │
│MB Used   30779.8MB  7405.8MB | Page Scans        0.0        | System    22.3%  │
│MB Free   18372.1MB 10898.2MB | Page Cycles       0.0        | Free      37.4%  │
│Total(MB) 49151.9MB 18304.0MB | Page Steals       0.0        |           ------ │
│                              | Page Faults    5474.0        | Total    100.0%  │
│------------------------------------------------------------ | numclient 18.7%  │
│Min/Maxperm     4651MB(  9%)  9301MB( 19%) <--% of RAM       | maxclient 18.9%  │
│Min/Maxfree     960   1088       Total Virtual   65.9GB      | User      35.7%  │
│Min/Maxpgahead    2      8    Accessed Virtual   27.0GB 41.0%  Pinned    24.3%  │
│                                                                                │
│ Paging-Space ──────────────────────────────────────────────────────────────────│
│    Volume-Group PagingSpace-Name Type LPs  MB    Used IOpending                │
│          rootvg              hd6  LV  128  4096  60%    0    Active    Auto    │
│          rootvg         paging00  LV  126  4032  60%    0    Active    Auto    │
│          rootvg         paging01  LV  318 10176  25%    0    Active    Auto    │
│────────────────────────────────────────────────────────────────────────────────│

topas shot:
Code:
 Tue Dec 15 13:45:24 2009   Interval:  2         Cswitch   11920  Readch  3124.7K
                                                 Syscall   29015  Writech 2256.6K
 Kernel    9.4   |###                         |  Reads       703  Rawin         1
 User     82.4   |########################    |  Writes      326  Ttyout      238
 Wait      0.0   |                            |  Forks         3  Igets         0
 Idle      8.2   |###                         |  Execs         3  Namei      2381
 Physc =  2.40                     %Entc=  96.1  Runqueue    4.5  Dirblk        0
                                                 Waitqueue   0.0
 Network  KBPS   I-Pack  O-Pack   KB-In  KB-Out
 en4    6243.7   5640.5  1053.0  6179.9    63.7  PAGING           MEMORY
 lo0       0.0      0.0     0.0     0.0     0.0  Faults     3226  Real,MB   49151
                                                 Steals        0  % Comp     42.2
 Disk    Busy%     KBPS     TPS KB-Read KB-Writ  PgspIn        0  % Noncomp  18.8
 hdisk3    2.0     1.7K   28.5     0.0     1.7K  PgspOut       0  % Client   18.8
 hdisk23   1.5   512.9     4.0     0.0   512.9   PageIn        3
 hdisk14   0.5   172.3    35.1     0.0   172.3   PageOut     560  PAGING SPACE
 hdisk2    0.0     0.0     0.0     0.0     0.0   Sios        563  Size,MB   18304
                                                                  % Used     40.0
 Name            PID  CPU%  PgSp Owner           NFS (calls/sec)  % Free     60.0
 java        1208454  68.5 144.4 util            ServerV2       0
 syncd        348408   4.8   0.5 root            ClientV2       0   Press:
 java        1364098   1.1  60.0 root            ServerV3       0   "h" for help
 topas        577692   0.0   2.0 util            ClientV3       0   "q" to quit

I appreciate any feedback!

cheers!

f.
 

10 More Discussions You Might Find Interesting

1. HP-UX

Copy on Write page faults

Hello Please can you tell me how to access COPY ON WRITE page faults in HP -UNIX. I found the structure in /usr/include/sys/vmmeter with the structure name vmmeter. Please tell me the function to fill the values to this structure and also the arguments for function.:: (5 Replies)
Discussion started by: manjunath
5 Replies

2. HP-UX

Intransient blocking page faults

Hi, Will anybody tell me what is this 'intransient blocking page faults' in HP-UX, it is in the structure _pst_vminfo in the header file /ust/include/sys/pstat/vm_pstat_body.h? (4 Replies)
Discussion started by: sushaga
4 Replies

3. UNIX for Advanced & Expert Users

Shared memory shortage but lots of unused memory

I am running HP-UX B.11.11. I'm increasing a parameter for a database engine so that it uses more memory to buffer the disk drive (to speed up performance). I have over 5GB of memory not being used. But when I try to start the DB with the increased buffer parameter I get told. "Not... (1 Reply)
Discussion started by: cjcamaro
1 Replies

4. AIX

High Page Faults

Sorry my poor english In 570 pseries nmon shows excessive page faults, ascents of something more than 30000 Page faults. System: AIX 5.2 ML5 Processor Type: PowerPC_POWER5 Number Of Processors: 2 Processor Clock Speed: 1656 MHz CPU Type: 64-bit Kernel Type: 64-bit Memory Size: 2816 MB ... (1 Reply)
Discussion started by: daviguez
1 Replies

5. Solaris

how to get the more memory free space (see memory free column)

Hi all, Could please let me know how to get the more memory free space (not added the RAM) in local zone. -bash-3.00# vmstat 2 5 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr s0 s1 s1 s1 in sy cs us sy... (3 Replies)
Discussion started by: murthy76
3 Replies

6. Programming

How to deal with lots of data in memory in order not to run out of memory

Hi, I'm trying to learn how to manage memory when I have to deal with lots of data. Basically I'm indexing a huge file (5GB, but it can be bigger), by creating tables that holds offset <-> startOfSomeData information. Currently I'm mapping the whole file at once (yep!) but of course the... (1 Reply)
Discussion started by: emitrax
1 Replies

7. AIX

Lots of page faults on AIX mySQL lpar

Hi, OS = AIX 5.3 Large number of page faults recently start to occure on AIX 5.3 lpar with mysql database installed. I need help in setting AIX OS parameter to solve the paging problem and some guidance on interpreting my stats t Code: # vmstat... (5 Replies)
Discussion started by: crosys
5 Replies

8. AIX

AIX 7.1 high page faults

hi guys i hope you can help me with this situation. i have 2 lpar with aix 7.1 and oracle 11gr2 in grid mode. when i start nmon to check the current system health i notice that page fault are over 3000/s. than i have opened a case with ibm and they say that the problem is not paging nor... (10 Replies)
Discussion started by: gullio23
10 Replies

9. Solaris

Page faults on OS

Hi guys, I have a zone on a M5000 server running solaris 10. The zone has an SAP application running on it and facing some performance issues. As part of the troubleshooting, I've been recommended to look for any paging on the OS. Please advise how to look for the paging. I've been looking at... (4 Replies)
Discussion started by: frum
4 Replies

10. AIX

High Paging when lots of free memory AIX 5.3

I am new to AIX, I have few AIX 5.3 servers and I could see there are significant difference in paging space utilization on servers even though they are running same applications below server is working fine which shows 2-5 % paging usage throuh out the day cpu_scale_memp = 8... (12 Replies)
Discussion started by: bibish
12 Replies
All times are GMT -4. The time now is 06:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy