Monitoring Paging and Swapping


 
Thread Tools Search this Thread
Operating Systems Solaris Monitoring Paging and Swapping
# 22  
Old 12-14-2018
Yes, a process needs not to steal memory from another process given the fact that for it to be able to use memory, it first needs to perform a successful allocation.

A process only deals with virtual memory. Memory allocated with malloc or mmap is by definition always contiguous in the process virtual space (it has an address and a size). Malloc can use brk or mmap system call to get space. Malloc'd areas might be anywhere in the process virtual space. When this virtual space is fragmented and limited (32 bit processes), a large allocation might fail even if smaller than the sum of total free space.

Contiguous virtual memory pages are mapped to physical pages. The latter don't have to be physically contiguous. That wouldn't make sense as physical pages can be on RAM and later on disk and later again, somewhere else on RAM.

A process virtual space is unrelated to another process virtual space, the same addresses can be used on either side but map to different physical pages.
# 23  
Old 12-15-2018
Hi Both,

Thank you for the detailed explanations.

To sumup,

- Virtual memory reservation does not take up physical space. (vmstat free still reflect as unused space, swap -l still reflect as free blocks)
- Virtual memory reservation is just a reserved amount and has nothing with the physical allocation of the ram / swap disk. In the sense that an allocated virtual memory area (non-shared) cannot be shared with another process, but the underlying physical ram could be randomly use by different processes at different time.

Please kindly let me know if my understanding thusfar is correct

===================

Back to the reason on the post

- I have 3GB of swap disk used (in swap -l)
- I have 10GB+ of free physical RAM
- I have 0 scanrate and vmstat available swap = 37GB

Since i am 10G of physical ram and 0 SR - i am not short on ram
Since i have 37GB of virtual swap available - i am not short on virtual swap

What could have contributed to the 3GB swap disk ?

Could it be at some point of time, i am running low on physical ram and swap/paging need to be done ?
When does physical space used in swapdisk be release ?

It is gradually increasing (slow.. but like 5-10MB more of swapdisk used per week) - that is the worrying part.

Regards,
Noob
# 24  
Old 12-15-2018
Quote:
Virtual swap = ram + disk. (shown in swap -s)
Quote:
Originally Posted by javanoob
- Virtual memory reservation does not take up physical space. (vmstat free still reflect as unused space, swap -l still reflect as free blocks)
- Virtual memory reservation is just a reserved amount and has nothing with the physical allocation of the ram / swap disk. In the sense that an allocated virtual memory area (non-shared) cannot be shared with another process, but the underlying physical ram could be randomly use by different processes at different time.

Please kindly let me know if my understanding thusfar is correct
More or less: yes. Still, you shouldn't use incorrect terminology because at some point - language is the "coagulated thought process" - it will be in the way of your understanding. There is no "virtual swap". There is "virtual memory", which is RAM + swap space(s). It is called "virtual" because to processes it looks like one contiguous address space. To running processes there is no difference between RAM and swap. There is just "memory". Only the operating system (to be precise: the part of it which manages memory) "knows" that RAM is faster than swap and therefore tries its best to keep the most possible of the most used process memory in RAM so that paging operations (swap in/swap out) happen as rarely as possible. A process can ask for the memory it allocates to be (and remain) in RAM, but it has no way to ascertain that this is the case.

Memory reservation is in fact like reserving a table at a restaurant like jiliagre has so astutely explained: you call and say you want to come tomorrow at 20:00, table for four. They will either take or refuse your reservation but if they take it then the table cannot be used by anybody but your party, regardless of you showing up (using the table) or not. A process can reserve (=allocate) memory. Directly it will have no control over where this memory will be: RAM or swap. It can ask the OS for it being in RAM (database software does this regularly, i.e. Oracle allocates its SGA as "not swappable"), but it will have to take the OS's word for it that this in fact has happened.

Quote:
Originally Posted by javanoob
What could have contributed to the 3GB swap disk ?

Could it be at some point of time, i am running low on physical ram and swap/paging need to be done ?
When does physical space used in swapdisk be release ?

It is gradually increasing (slow.. but like 5-10MB more of swapdisk used per week) - that is the worrying part.
There is "early" and "late" swap allocation as i have explained up in this thread. I do not know your application but maybe it uses shared memory and allocates it via the shmget() system call. This function can reserve swap space (see the SHM_NORESERVE flag) along with the allocated memory so that the process is guaranteed to have swap space should the necessity ever occur to swap it out. Alternatively it may use memory mapped I/O, the mmap() system call has an analogous functionality. If this is the case it might be sloppily programmed and thus create a memory sink: that is: memory is allocated but instead of giving it back the process "forgets" about it and allocates new memory the next time it performs the same function. (The sad truth is that this is a very common phenomenon.) This could account for the slow increase of allocated swap over time.

But even then, "swap allocated" is not the same as "swap used" which in fact comes down to: "pages swapped out" and "pages swapped in". This is why it is useless to look at swap -l's output. This only tells you if you need to increase the amount of available swap space. It will not tell you how much is actually written to/read from the swap. This is only told by the pi/po counter of vmstat (and all the other counter discussed above in detail).

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 25  
Old 12-15-2018
Quote:
Originally Posted by javanoob
Back to the reason on the post

- I have 3GB of swap disk used (in swap -l)
- I have 10GB+ of free physical RAM
- I have 0 scanrate and vmstat available swap = 37GB

Since i am 10G of physical ram and 0 SR - i am not short on ram
Since i have 37GB of virtual swap available - i am not short on virtual swap

What could have contributed to the 3GB swap disk ?
Three gigabytes of memory were used (i.e. read/written) sometime in the past by some process(es). They have not been accessed for a while so the kernel decided to put the data on disk, to keep the free RAM high.
Quote:
Could it be at some point of time, i am running low on physical ram and swap/paging need to be done ?
You need not to starve on RAM for paging to occur.
Quote:
When does physical space used in swapdisk be release ?
When the processes owning it will die.
Quote:
It is gradually increasing (slow.. but like 5-10MB more of swapdisk used per week) - that is the worrying part.
That might be just some optimization done by the kernel.
There might be a memory leak in a process, 10 MB per week is not among the fiercest ones.
There might be a growing file in /tmp or any tmpfs based file system. The storage area of tmpfs is virtual memory (not any process virtual memory but the OS virtual memory, i.e. RAM + SWAP as Bakunin wrote).

Note also the free memory might be actually used by the kernel, which isn't constrained by process virtual memory rules.

Solaris uses free memory as UFS and NFS cache, so this free memory contains actual data, but it is nevertheless reported as free by vmstat and similar commands, because it is immediately available for processes allocations.

Last edited by jlliagre; 12-15-2018 at 06:36 PM..
This User Gave Thanks to jlliagre For This Post:
# 26  
Old 12-16-2018
Quote:
Originally Posted by bakunin
But even then, "swap allocated" is not the same as "swap used" which in fact comes down to: "pages swapped out" and "pages swapped in". This is why it is useless to look at swap -l's output. This only tells you if you need to increase the amount of available swap space. It will not tell you how much is actually written to/read from the swap. This is only told by the pi/po counter of vmstat (and all the other counter discussed above in detail).

I hope this helps.
bakunin
Hi Bakunin,

Thank you for your reply and clarification. I got confused with terminology used over virtual memory and virtual swap as in the metalink doc (it is using the term "virtual swap" to represent memory + disk)
Quote:
When a process calls the malloc()/sbrk() commands, only virtual swap is allocated. The operating system allocates the memory from physical disk-based swap first. If disk-based swap is exhausted or unconfigured, the reservation is allocated from physical memory.....
Follow the guidelines below to calculate amount of virtual swap usage:Virtual swap = Physical Memory + Fixed Disk swap
But I believe they meant the same thing though.. (virtual memory) = physical mem + disk swap.

Lastly, can i confirm the following one last time
Quote:
But even then, "swap allocated" is not the same as "swap used" which in fact comes down to: "pages swapped out" and "pages swapped in".
Swap allocated will not be shown swap -l. swap used will be shown in swap -l.
"pages swapped out" and "pages swapped in" eventually ends as -> Current swap_used

Is there anyway to look into the physical swap area and see what's there and hold by which process ?

Regards,
Noob

--- Post updated at 02:02 AM ---

Quote:
Originally Posted by jlliagre
Three gigabytes of memory were used (i.e. read/written) sometime in the past by some process(es). They have not been accessed for a while so the kernel decided to put the data on disk, to keep the free RAM high.

You need not to starve on RAM for paging to occur.
When the processes owning it will die.

That might be just some optimization done by the kernel.
There might be a memory leak in a process, 10 MB per week is not among the fiercest ones.
There might be a growing file in /tmp or any tmpfs based file system. The storage area of tmpfs is virtual memory (not any process virtual memory but the OS virtual memory, i.e. RAM + SWAP as Bakunin wrote).

Note also the free memory might be actually used by the kernel, which isn't constrained by process virtual memory rules.

Solaris uses free memory as UFS and NFS cache, so this free memory contains actual data, but it is nevertheless reported as free by vmstat and similar commands, because it is immediately available for processes allocations.
Hi jlliagre,

Thanks for your reply.
Is there anyway we can see what processes are actually holding onto the disk-swap area ?

On a side note, after going through the entire thread, can i say it is absolutely possible for a system to have plenty of free physical ram and even disk swap, but very limited virtual memory due to virtual memory reservation and due to the fact that Solaris doesn't over commit virtual-memory ?

Regards,
Noob
# 27  
Old 12-16-2018
Quote:
Originally Posted by javanoob
Lastly, can i confirm the following one last time
Swap allocated will not be shown swap -l. swap used will be shown in swap -l.
"pages swapped out" and "pages swapped in" eventually ends as -> Current swap_used
No. What i meant was: a process gets (though whatever means, mmap(), shmget(), the OS or something else) space in the swap. This space will show up in swap -l as "used". But from a performance POV you do not want to avoid "used swap", you want to avoid transferring memory pages to and from the swap because the actual act of transferring is what slows the system down. This "act of transferring" pages is either "page in" (a page transferred from swap to memory) or "page out" (a page transferred from emory to swap) and this shows in vmstat. To extend jiliagres metaphor: if the restaurant in question serves unhealthy food, reserving the table there is not what gets you into trouble, only actually eating there is. So it doesn't help to look at the reservations to discern if someone is in danger or not.

Quote:
Originally Posted by javanoob
Is there anyway to look into the physical swap area and see what's there and hold by which process ?
Now, this is a good question! I would refer you to the ps command and - if you are really fearless - the kdb (kernel debugger) command but the difference between me and jiliagre is that he is a Solaris expert and i am not. (Whatever i told you above is "general UNIX knowledge", not specialised Solaris knowledge). There is maybe some special Solaris way he is a aware of that i am not, so his word will be the last in this matter.

I hope this helps.

bakunin
# 28  
Old 12-16-2018
Quote:
Originally Posted by javanoob
Swap allocated will not be shown swap -l. swap used will be shown in swap -l.
Yes. The issue is the word "swap" has several meanings. When you say "swap allocated", you talk about virtual memory but when you say "swap used", that might still be virtual memory (swap -s) or on disk swap (swap -l).
Quote:
"pages swapped out" and "pages swapped in" eventually ends as -> Current swap_used
Not that much. Generally, most of the paging activity won't be related to the on-disk swap area but just regular files read/write operations. Therefore, hi page-in/page-out counters do not imply there is a shortage in RAM, just some active disk I/Os. As I wrote, the relevant statistic about RAM shortage is the scan rate (sr).
Quote:
Is there anyway to look into the physical swap area and see what's there and hold by which process ?
Not an easy task, and depending on the method used, reading what is there would have the side effect to page it in, i.e. to put it back into RAM.
You can have detailed information about a process memory usage with the pmap -xs command.
The blocks related to swap are tagged "[anon]" for anonymous. That means the pages have no file back-end. The RSS column tells the number of pages that are currently in RAM while the KBytes column gives the size of the block.
Knowing these addresses, you can use the kernel modular debugger (mdb -k)to retrieve the mapping between virtual addresses returned by pmap and the corresponding physical addresses.
Quote:
On a side note, after going through the entire thread, can i say it is absolutely possible for a system to have plenty of free physical ram and even disk swap, but very limited virtual memory due to virtual memory reservation and due to the fact that Solaris doesn't over commit virtual-memory ?
Yes, and I have already observed such puzzling at first sight situations in real life systems.
This User Gave Thanks to jlliagre For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Swapping lines

Hi there, I have a text that I'm trying to format into something more readable. However, I'm stuck in the last step. I've searched and tried things over the internet with no avail. OS: Mac After parsing the original text that I won't put here, I managed to get something like this, but this... (8 Replies)
Discussion started by: Kibou
8 Replies

2. Solaris

Swapping

Hi Guys I am using SPARC-T4 (chipid 0, clock 2998 MHz), SunOS 5.10 Generic_150400-38 sun4v. How do I see if the server was doing some swapping like yesterday? I had a java application error with java.lang.OutOfMemoryError, now I want to check if the server was not doing some swapping at... (4 Replies)
Discussion started by: Phuti
4 Replies

3. Shell Programming and Scripting

Swapping fields

Hallo Team, This is the command that i am running : grep ",Call Forward Not Reachable" *2013* this is the output that i am getting (i did a head -10 but the files can be more than 1000) ... (8 Replies)
Discussion started by: kekanap
8 Replies

4. Shell Programming and Scripting

Swapping three lines

I have some text: <date>some_date</date> <text>some_text</text> <name>some_name<name> and I want to transform it to smthng like that: some_name on some_date: some_text I've tried sed: sed 's/<text>\(.*\)<\/text> <name>\(.*\)<\/name>/\2 - \1/' but it says unterminated... (13 Replies)
Discussion started by: dsjkvf
13 Replies

5. UNIX for Dummies Questions & Answers

Swapping in VI editor

Hi, I am attempting to replace several similar words with another word in vi. Here is what I have written for the script: 3dTcat -prefix SuperBrik_4WAY_HRF ../JULY10_2007A/results2TENT/stats.JULY10_2007A+tlrc ../JULY10_2007G/results2TENT/stats.JULY10_2007G+tlrc... (1 Reply)
Discussion started by: Jahn
1 Replies

6. UNIX for Advanced & Expert Users

virtual memory management, swapping paging

can anybody explain me the concepts virtual memory mangement, swapping and paging? although i roughly know what they are , i need more solid distinction between them, and also i want to figure out the relations between them? do you have any well-defined definitons for this concepts? (2 Replies)
Discussion started by: gfhgfnhhn
2 Replies

7. SuSE

Swapping

Hello! Why does my SuSE GNU/Linux machine swap? I have a Gig of ram, currently 14MBs of free RAM, 724MB - buffers and caches... That is 685MB of cached RAM, then kernel really should'nt have to swap, It should release cached memory in my thinkin... It has only swaped 3MB's but still,... (3 Replies)
Discussion started by: Esaia
3 Replies

8. UNIX for Dummies Questions & Answers

how to get swapping info

Hi How can I determine if swapping is occuring on a server. Thanks, Leo (2 Replies)
Discussion started by: leo
2 Replies

9. Filesystems, Disks and Memory

Paging and Swapping

Hi Guys: Would like to know how to check system swapping and paging and some theory on how they function. I am an oracle dba and my environment is 8171 on AIX 433. We have a 1GB of RAM on the box and I am educating myself to see how much more SGA can be accommodated on the box and what are the... (2 Replies)
Discussion started by: ST2000
2 Replies

10. UNIX for Advanced & Expert Users

Excessive Paging&Swapping!

Hi all! Working on Oracle v8i/9i on Unix Sun Solaris v8.0. I am experiencing excessive paging & Swapping.Would like to know the cause. I guess:could be due to inappropriate setting of Unix Kernel Parameters... Please correct me if I am wrong! Thanks&Regards, Amit. (5 Replies)
Discussion started by: Amitstora
5 Replies
Login or Register to Ask a Question