Monitoring Paging and Swapping


 
Thread Tools Search this Thread
Operating Systems Solaris Monitoring Paging and Swapping
# 8  
Old 10-16-2018
Quote:
Originally Posted by javanoob
1) I am seeing recommendations of doing vmstat with a large interval - why do we need do use a large interval ? How is that different from doing multiple counts with short intervals ?
.e.g. vmstat 10 2 vs vmstat 1 10 ?
vmstat 10 2 will monitor an interval of 20 seconds, whereas vmstat 1 10 will only cover 10 seconds. Both are not sensible invocations for long-term monitoring. Try vmstat 600 (every 10 minutes) or something like that. The reason why you want a long interval is that you are not interested in any "spike" but the average usage of a system. Further, every polling of the stats also places a (slight but non-zero) load onto the system. You don't want your monitoring system to create the bottleneck it is intended to prevent.

Quote:
Originally Posted by javanoob
2) When there is a physical memory shortage, and memory need to be page out, will this piece of memory that is paged out will resides on the physical swap volume ?
Yes, correct

Quote:
Originally Posted by javanoob
If so, why isn't swap -l an accurate way of saying if there is a memory shortage ?
You don't want to avoid swap usage you want to avoid swapping activity. Something being in the swap doesn't hurt - putting it there and getting it back into memory again is what hurts. Therefore looking at a number which only tells you how much swap is used doesn't tell you what you really want to know.

Quote:
Originally Posted by javanoob
is it because sometimes paging unused memory out is healthy and necessary
Sometimes this is the case - it might be better overall if the damage you take by swapping something out is outweighed by the gain you may get from the memory freed that way which can be used otherwise. In general (but notice: "in general" doesn't mean "always") it makes sense to size a system in a way that swapping doesn't take place in normal operation. Swap is generally a "plan B", not the "plan A".


Quote:
Originally Posted by javanoob
i3) However, can we also directly say, if my swap -l always display 'free' = 'blocks', it definitely means i have sufficient memory, because nothing is paged nor swaped to physical swap - right ?
Yes. But in a professional environment it makes sense to size a system sufficiently. Memory does not come free of charge and it pays (or rather - saves) to give a system as much as it needs - but not more. It is the job of a systems administrator to walk that fine line between oversized (=wasting money) and undersized (=hurting the purpose).

You might want to read this little introduction to performance tuning and monitoring for more details about swapping.

I hope that helps.

bakunin
These 2 Users Gave Thanks to bakunin For This Post:
# 9  
Old 10-16-2018
Hi both (Jilliagre and Bakunin) and all,

Thank you so much for your advice and guidance. I can't see where i could have possibly understand more if not here.
Thanks for the link on the introduction to performance tuning and monitoring as well.

It seems like understanding the OS kernel and its working is slowly like a lost trade; people no longer care how it works, people just want things to work..

Thanks for staying around to keep the forum alive, we need you guys around..

=====================================================================

On a side note on ZFS ARC, i do realize it is sucking up RAM and its not reflected as freemem in neither vmstat or sar.

As mentioned my MadeInGermany, i do set a reserve hint -> this kinda set a upper limit on how much ZFS ARC can grow up to.

But how do i know if the amount of memory available and allocated to ZFS ARC is sufficient or not ?

Regards,
Noob
# 10  
Old 10-16-2018
Quote:
Originally Posted by javanoob
But how do I know if the amount of memory available and allocated to ZFS ARC is sufficient or not ?
Check the cache hit ratio to see how efficient it is. The closer to 100%, the better.

If you can fit all the files you use in RAM (hit rate 100%) and still have room enough for the applications and other consumers RAM usage, go for it and set the ZFS ARC size to be able to host everything.

On the opposite and more likely situation, that's a trade off and you should measure usage, do tests that last a few days or more, see how your RAM is used, see how performance is impacted by giving more memory to the cache or more memory to the applications, and act accordingly.

Last edited by jlliagre; 10-17-2018 at 04:18 AM..
This User Gave Thanks to jlliagre For This Post:
# 11  
Old 10-17-2018
Quote:
Originally Posted by jlliagre
Check the cache hit ratio to see how efficient it is. The closer to 100%, the better.

If you can fit all the files you use in RAM (hit rate 100%) and still have room enough for the applications and other consumers RAM usage, go for it and set the ZFS ARC size to be able to host everything.

On the opposite and more likely situation, that's a trade off and you should measure usage, do tests that last a few days or more, see how your RAM is used, see how performance is impacted by giving more memory to the cache or more memory to the applications, and act accordingly.
Hi Jilliagre,


Sorry for asking.. where/how do i see the cache hit for ZFS ARC...


Regards,
Noob
# 12  
Old 10-17-2018
That's a good question. You can compute it from kstat values, e.g.:

Code:
kstat -p "::arcstats:demand_*data*" 10 10

The hit ratio is equal to 100.*xxx_hits/(xxx_hits+xxx_misses) with xxx being demand_data or demand_metadata
These 2 Users Gave Thanks to jlliagre For This Post:
# 13  
Old 12-13-2018
Hi all,

I am sorry to bring up this thread again.
Recently, i have same heavy swap usage against the physical swap space despite having physical free ram.

I read up this old thread > Out of swap but RAM available >> and my understanding is that memory reservation will eat up swap-space if there isn't enough virtual swap for reservation ?

But again, read these directly from Solaris (Doc ID 1010585.1) in Oracle metalink somehow indicate that memory reservation against physical memory or swap doesn't actually take up the physical space.

Quote:
When a process calls malloc()/sbrk() only virtual swap is reserved.
Reservation is done against the physical disk swap first.
If that is exhausted or not configured then reservation is done against physical memory. If both are exhausted then malloc() fails

As discussed, when measuring virtual swap available for reservation, consider monitoring vmstat ("swap" column) or swap -s (the "available" value).
Free memory as reported by vmstat ("free" column) or swap usage as reported by swap -l ("free" column) is unrelated with virtual swap available for reservation.

When allocating swap reservation from memory, there is no memory deducted and vmstat continues to show same amount of "free" memory.
Similarly, when a swap reservation is made from physical disk-based swap, swap -l will continue to show the same amount of "free" swap.


Free memory is only deducted due to a page fault and free swap is deducted during a memory shortage, when data needs to be migrated from the physical memory to physical disk swap to maintain a sufficient supply of free memory.

So does memory reservation actually take up swap space or not ?

Regards,
Noob

Last edited by javanoob; 12-13-2018 at 02:50 AM..
# 14  
Old 12-13-2018
Both the linked thread and MOS Document are correct.

vmstat is reporting that there is unused RAM and it is true. There is no data stored on that RAM.

malloc fails because there is no more swap or RAM available. All of it is either used or unused but reserved.

Let me try a metaphor: if you enter a restaurant and see many empty tables, that doesn't mean you can sit on any of them. They might all be reserved by customers that aren't there yet (and maybe some will never show up).
This User Gave Thanks to jlliagre For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Swapping lines

Hi there, I have a text that I'm trying to format into something more readable. However, I'm stuck in the last step. I've searched and tried things over the internet with no avail. OS: Mac After parsing the original text that I won't put here, I managed to get something like this, but this... (8 Replies)
Discussion started by: Kibou
8 Replies

2. Solaris

Swapping

Hi Guys I am using SPARC-T4 (chipid 0, clock 2998 MHz), SunOS 5.10 Generic_150400-38 sun4v. How do I see if the server was doing some swapping like yesterday? I had a java application error with java.lang.OutOfMemoryError, now I want to check if the server was not doing some swapping at... (4 Replies)
Discussion started by: Phuti
4 Replies

3. Shell Programming and Scripting

Swapping fields

Hallo Team, This is the command that i am running : grep ",Call Forward Not Reachable" *2013* this is the output that i am getting (i did a head -10 but the files can be more than 1000) ... (8 Replies)
Discussion started by: kekanap
8 Replies

4. Shell Programming and Scripting

Swapping three lines

I have some text: <date>some_date</date> <text>some_text</text> <name>some_name<name> and I want to transform it to smthng like that: some_name on some_date: some_text I've tried sed: sed 's/<text>\(.*\)<\/text> <name>\(.*\)<\/name>/\2 - \1/' but it says unterminated... (13 Replies)
Discussion started by: dsjkvf
13 Replies

5. UNIX for Dummies Questions & Answers

Swapping in VI editor

Hi, I am attempting to replace several similar words with another word in vi. Here is what I have written for the script: 3dTcat -prefix SuperBrik_4WAY_HRF ../JULY10_2007A/results2TENT/stats.JULY10_2007A+tlrc ../JULY10_2007G/results2TENT/stats.JULY10_2007G+tlrc... (1 Reply)
Discussion started by: Jahn
1 Replies

6. UNIX for Advanced & Expert Users

virtual memory management, swapping paging

can anybody explain me the concepts virtual memory mangement, swapping and paging? although i roughly know what they are , i need more solid distinction between them, and also i want to figure out the relations between them? do you have any well-defined definitons for this concepts? (2 Replies)
Discussion started by: gfhgfnhhn
2 Replies

7. SuSE

Swapping

Hello! Why does my SuSE GNU/Linux machine swap? I have a Gig of ram, currently 14MBs of free RAM, 724MB - buffers and caches... That is 685MB of cached RAM, then kernel really should'nt have to swap, It should release cached memory in my thinkin... It has only swaped 3MB's but still,... (3 Replies)
Discussion started by: Esaia
3 Replies

8. UNIX for Dummies Questions & Answers

how to get swapping info

Hi How can I determine if swapping is occuring on a server. Thanks, Leo (2 Replies)
Discussion started by: leo
2 Replies

9. Filesystems, Disks and Memory

Paging and Swapping

Hi Guys: Would like to know how to check system swapping and paging and some theory on how they function. I am an oracle dba and my environment is 8171 on AIX 433. We have a 1GB of RAM on the box and I am educating myself to see how much more SGA can be accommodated on the box and what are the... (2 Replies)
Discussion started by: ST2000
2 Replies

10. UNIX for Advanced & Expert Users

Excessive Paging&Swapping!

Hi all! Working on Oracle v8i/9i on Unix Sun Solaris v8.0. I am experiencing excessive paging & Swapping.Would like to know the cause. I guess:could be due to inappropriate setting of Unix Kernel Parameters... Please correct me if I am wrong! Thanks&Regards, Amit. (5 Replies)
Discussion started by: Amitstora
5 Replies
Login or Register to Ask a Question