Login or Register to Ask a Question and Join Our Community


Solaris

High Page In and Executable page In

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris High Page In and Executable page In
# 1  
Old 11-13-2008
High Page In and Executable page In
Hi,

Currently I'm experience very high page ins on my system running on solaris 10.

From vmstat, the page ins figure is very high, further drill down shows the page ins are from file system and occassional spike in executable page ins.

Details as follow:

Code:
oracle@perch:/files>> vmstat 5
 kthr      memory            page            disk          faults      cpu
 r b w   swap     free   re   mf     pi      po fr de sr m1 m1 m2 m2   in   sy   cs us sy id
 3 0 0 7268376 5110432 1041 2719 729590752414 753 753 0 0 6 0 6 0 7766 100603 12925 56 19 25
 5 0 0 11818608 9292016 451 918 13573290886586 3 3 0 0 2 0 1 0 10952 161811 14018 80 20 0
 2 0 0 11818432 9292960 385 792 13459633370179 2 2 0 0 1 0 2 0 10053 165300 13082 81 19 1
 5 0 0 11819720 9294472 322 623 12844506325972 0 0 0 0 1 0 1 0 11025 162384 14302 81 19 0
 3 0 0 11819400 9294664 468 1132 12864149119550 0 0 0 0 5 0 5 0 10555 163452 13193 78 21 1
 5 0 0 11818912 9295784 430 926 11488048016240 3 3 0 0 1 0 1 0 10935 158379 14086 80 20 0
 2 0 0 11819560 9296824 381 906 12037450017951 0 0 0 0 1 0 1 0 10671 165815 13422 81 19 0
 5 0 0 11819968 9298064 440 863 13209752927129 0 0 0 0 1 0 1 0 11219 158202 14557 80 20 0
 6 0 0 11819776 9297168 500 953 12490715053962 2 2 0 0 2 0 1 0 11777 157736 15521 80 20 0

Code:
oracle@perch:/files>> vmstat -p 5
     memory           page          executable      anonymous      filesystem
   swap  free  re  mf  fr  de  sr  epi  epo  epf  api  apo  apf  fpi  fpo  fpf
 7268296 5110360 1041 2719 753 0 0 11811407681 0 0  0    0    0 717270832025 753 753
 11820496 9292224 2055 2415 0 0 0    0    0    0    0    0    0 52087519922399 0 0
 11822032 9293648 2120 2163 5 0 0    0    0    0    0    0    0 55982485259240 5 5
 11824408 9296288 2414 2562 2 0 0    0    0    0    0    0    0 54414466418190 2 2
 11824328 9296576 2698 2719 0 0 0    0    0    0    0    0    0 52983811694559 0 0
 11822288 9293912 2332 1951 3 0 0    0    0    0    0    0    0 49271214261636 3 3
 11820448 9291312 2509 1115 0 0 0   18    0    0    0    0    0 48591187881736 0 0
 11817528 9294472 2480 871 2 0  0    0    0    0    0    0    0 54421309283685 2 2
 11820304 9304032 2429 860 8837 0 0 164187898623 0 0 0   0    0 47628172925439 8837 8837
 11814208 9413840 2067 860 29007 0 0 0    0    0    0    0    0 43272384005459 29009 29007
 11815744 9486424 2333 860 835 0 0   0    0    0    0    0    0 38585601084343 837 835

What area should we be looking into? Our swap space is being consumed steadily and reducing by the day. Not sure if it's because of runaway process.

Any opinion is appreciated.

Thanks in advance
ET.
Last edited by jim mcnamara; 11-15-2008 at 01:49 PM.. Reason: Moved to SUN forum
# 2  
Old 11-16-2008
So, for example, you have 13,209,752,927,129 page-in's in a 5 second period? That's a little bit more powerful than the systems I work with. Please tell us about your system. It's gotta be ccNUMA, but how many cpu's?

Well, since you have enough disks to supply 2.6 quadrillion pages in a second, you can afford some more swap space. So just add a few TB more swap.
# 3  
Old 11-16-2008
OS: Solaris 10

Quote:
oracle@perch:/u01/usr/lss/mdw>> prtdiag |more
System Configuration: Sun Microsystems sun4u Sun Fire 6800
System clock frequency: 150 MHz
Memory size: 16384 Megabytes

========================= CPUs ===============================================

CPU Run E$ CPU CPU
FRU Name ID MHz MB Impl. Mask
---------- ------- ---- ---- ------- ----
/N0/SB2/P0 8 1200 8.0 US-III+ 11.0
/N0/SB2/P1 9 1200 8.0 US-III+ 11.0
/N0/SB2/P2 10 1200 8.0 US-III+ 11.0
/N0/SB2/P3 11 1200 8.0 US-III+ 11.0
/N0/SB5/P0 20 1200 8.0 US-III+ 11.0
/N0/SB5/P1 21 1200 8.0 US-III+ 11.0
/N0/SB5/P2 22 1200 8.0 US-III+ 11.0
/N0/SB5/P3 23 1200 8.0 US-III+ 11.0

========================= Memory Configuration ===============================

Logical Logical Logical
Port Bank Bank Bank DIMM Interleave Interlea
ve
FRU Name ID Num Size Status Size Factor Segment
------------- ---- ---- ------ ----------- ------ ---------- --------
--
/N0/SB2/P0/B0 8 0 1024MB pass 512MB 8-way 0
/N0/SB2/P0/B0 8 2 1024MB pass 512MB 8-way 0
/N0/SB2/P1/B0 9 0 1024MB pass 512MB 8-way 0
/N0/SB2/P1/B0 9 2 1024MB pass 512MB 8-way 0
/N0/SB2/P2/B0 10 0 1024MB pass 512MB 8-way 0
/N0/SB2/P2/B0 10 2 1024MB pass 512MB 8-way 0
/N0/SB2/P3/B0 11 0 1024MB pass 512MB 8-way 0
/N0/SB2/P3/B0 11 2 1024MB pass 512MB 8-way 0
/N0/SB5/P0/B0 20 0 1024MB pass 512MB 8-way 1
/N0/SB5/P0/B0 20 2 1024MB pass 512MB 8-way 1
/N0/SB5/P1/B0 21 0 1024MB pass 512MB 8-way 1
/N0/SB5/P1/B0 21 2 1024MB pass 512MB 8-way 1
/N0/SB5/P2/B0 22 0 1024MB pass 512MB 8-way 1
/N0/SB5/P2/B0 22 2 1024MB pass 512MB 8-way 1
/N0/SB5/P3/B0 23 0 1024MB pass 512MB 8-way 1
/N0/SB5/P3/B0 23 2 1024MB pass 512MB 8-way 1
I'm trying to look at why the swap space is being consumed consistently, and it isn't released back to the system. At this rate, it's just a matter of time the swap goes out again. Our bomb just exploded just week, and bring our oracle instance down due to out of /tmp forcing a restart of the instance which releases the memory.
# 4  
Old 11-16-2008
Are you using ZFS?
# 5  
Old 11-16-2008
Actually I was trying to be facetious. Numbers like that make me think that vmstat must be broken. If I ignore the pi and assume that the rest of the numbers are valid, I don't really see any problem. You have swap and free physical memory. Page-outs are low and so is the scan rate. So pi is impossible and everything else looks good. I guess I would look for a vmstat patch though.

But if swap is disappearing, do a "df -k". /tmp uses swap for sure and I think maybe /var/run or something like that does as well. Could /tmp be eating your swap area? If it's a program, the size as reported by ps would be growing over time. Also I often see people over-allocate shared memory to Oracle. So when Oracle is running I do a "ipcs -mb" to look at that.
# 6  
Old 11-16-2008
Yes, these pi and fpi numbers can't be but bogus. 20000 TeraBytes per second is simply unrealistic.

Is your system up to date with patches ?

Especially one that fix a Veritas bug:

Huge PI in vmstat
Last edited by jlliagre; 11-16-2008 at 10:30 PM..
# 7  
Old 11-17-2008
Quote:
Originally Posted by jlliagre
Yes, these pi and fpi numbers can't be but bogus. 20000 TeraBytes per second is simply unrealistic.

Is your system up to date with patches ?

Especially one that fix a Veritas bug:

Huge PI in vmstat
yea, that's what I'm looking at currently too. Vxfs 4.1 bug.

@reborg, it's running on vxfs. which may trigger the the high pi bogus numbers.

I have another concern is the disappearing swap space. I can see clearly that the free swap space is dropping by the day, but unable to nail down the culprit. any suggestion how should I approach it?

thanks for the views all.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk: Page number with total number of pages, EG Page 1 of 5

So I've worked how to add page numbers based on regex. It's using the footer text. How do we get the total amount added so we have page number with the total number of pages? Desired output: Page No:1 of 5 Thanks in advance. (15 Replies)
Discussion started by: tugar
15 Replies

2. AIX

AIX 7.1 high page faults

hi guys i hope you can help me with this situation. i have 2 lpar with aix 7.1 and oracle 11gr2 in grid mode. when i start nmon to check the current system health i notice that page fault are over 3000/s. than i have opened a case with ibm and they say that the problem is not paging nor... (10 Replies)
Discussion started by: gullio23
10 Replies

3. Shell Programming and Scripting

script for adding page number before page breaks

Hi, If there is an expert that can help: I have many txt files that are produced from pdftotext that include page breaks the page breaks seem to be unix style hex 0C. I want to add page numbers before each page break as in : Page XXXX Regards antman (9 Replies)
Discussion started by: antman
9 Replies

4. Shell Programming and Scripting

Print multiple copies page by page using lp command

Hi I have a pdf file that is being generated using the rwrun command in the shell script. I then have the lp command in the shell script to print the same pdf file. Suppose there are 4 pages in the pdf file , I need to print 2 copies of the first page, 2 copies of the second page , then 2... (7 Replies)
Discussion started by: megha2525
7 Replies

5. Web Development

Page load time- local page

Hi Is there a way to calculate the page load time, I am trying to calculate the load time of a page locally. I found tools to do this over http or https but none that work locally. Any ideas? Thanks. (4 Replies)
Discussion started by: jamie_123
4 Replies

6. UNIX for Dummies Questions & Answers

display command output page per page

Good afternoon, I wonder how i could use unix commands to ease the reading of long command result output ? like the "php -i" or any other command that returns a long answer. I could not find the right terms to Google it or search the forum. Therefore I bother you with this question. ... (3 Replies)
Discussion started by: Mat_k
3 Replies

7. Shell Programming and Scripting

lpr- how to print from page to page

hi all i have file_1 which contains 66 pages and i want to print only page 1 to 3 i try to write like this lp -d name_of_printer file_1 -P 1-7 this command does not work any ideas ? (4 Replies)
Discussion started by: naamas03
4 Replies

8. Solaris

Page Fault very high

Hi, We are running SUN sparc 5.8, notice frequent "Page Faults" message from our monitor s/w "ServerVantage" coming but no message display in SUN syslog or messages log. Any ideas why or what to check? thanks, Ahmad (2 Replies)
Discussion started by: Ahmad
2 Replies

9. AIX

High Page Faults

Sorry my poor english In 570 pseries nmon shows excessive page faults, ascents of something more than 30000 Page faults. System: AIX 5.2 ML5 Processor Type: PowerPC_POWER5 Number Of Processors: 2 Processor Clock Speed: 1656 MHz CPU Type: 64-bit Kernel Type: 64-bit Memory Size: 2816 MB ... (1 Reply)
Discussion started by: daviguez
1 Replies

10. AIX

map Page Down and Page up to ] and [

how do I map Page Down and Page up to ] and [ in my terminal using the .exrc file. Currently its not working and the use of ^F and ^B doesn't attract me at all. (0 Replies)
Discussion started by: buddy_amazing
0 Replies
Login or Register to Ask a Question