Is virtual memory so precious? about brk()


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Is virtual memory so precious? about brk()
# 1  
Old 11-24-2012
Is virtual memory so precious? about brk()

We know for every process, it has 4G virtual memory on a 32bit machine.
since virtual memory is not physical memory, why don't operating system allocate all it's virtual memory to it, but set a "program break" to limit it's heap space?
Even if the operating system allocate all the 4G virtual memory to a process, only when the process access an address that not mapped into physical memory, the OS allocate physical memory to it. So, what is the reason to limit a process's virtual memory space by "program break"?

Thanks.
# 2  
Old 11-25-2012
Here are the first reasons that come to mind:
  • easily detect bogus code that reference unallocated memory
  • avoid exhausting the system virtual memory for those operating systems that do not overcommit memory
  • allow non heap memory to exist, eg. allow a new thread to get its own stack
  • allow libraries that manage memory allocation (like libc malloc/free) to know what part of memory they own so can use the way they like.
# 3  
Old 11-26-2012
1, reasonable
2,don't understand
3,AFAIK, on linux threads' stack is allocated on heap of the main thread and managed by thread lib.
4,if the operating system allocate a very large heap to a process, and malloc knows it has all the heap space, then it can also use the way it like.
# 4  
Old 11-26-2012
Quote:
Originally Posted by vistastar
2,don't understand
That means as I have 200 processes running like my laptop on Solaris 11, I would need more than 800 GB of RAM+swap area. This would also affect to a lesser extent most Linux installations as "always overcommit" is not the default configuration.
See this page for some background about linux and memory overcommitment.
Quote:
3,AFAIK, on linux threads' stack is allocated on heap of the main thread and managed by thread lib.
Possibly but that was just an example. Other memory parts are neither heap nor stack, like mmaped areas (eg: shared libraries).
Quote:
4,if the operating system allocate a very large heap to a process, and malloc knows it has all the heap space, then it can also use the way it like.
Malloc can already use the whole heap space the way it likes as brk() is a cheap operation. Having all of it allocated at startup like you suggest wouldn't make that much of a difference on that side but there is no much point wasting resources, even virtual ones.
# 5  
Old 11-26-2012
Quote:
That means as I have 200 processes running like my laptop on Solaris 11, I would need more than 800 GB of RAM+swap area.
I thought OS don't allocate any swap area nor physical memory to a new got virtual space, but allocate physical pages directly when this space is written/read, and swap out when done some thing.Do you mean, OS allocate swap area to a process when it use malloc to ask for memory, but swap it into physical memory when this space is written/read?
I've written a test program that ask for nearly 3GB by sbrk() on my Archlinux that only have 128M RAM and 40M swap partition. But when I use the 'free' command , I see it does'nt take any swap space.

Last edited by vistastar; 11-26-2012 at 07:27 AM..
# 6  
Old 11-26-2012
Quote:
Originally Posted by vistastar
I thought OS don't allocate any swap area nor physical memory to a new got virtual space
That depends on how you define "allocate". Most OSes do either a partial or a full memory reservation. This reservation is backed by available disk and ram combined
Quote:
but allocate physical pages directly when this space is written/read, and swap out when done some thing.
You are describing pagination more than allocation.
Quote:
Do you mean, OS allocate swap area to a process when it use malloc to ask for memory, but swap it into physical memory when this space is written/read?
No. I mean most Unix OSes reserve (some or all of the allocated) space and this space must exist somewhere, whether in RAM or on disk or on tape or whatever doesn't matter.

Last edited by jlliagre; 11-26-2012 at 09:17 AM..
This User Gave Thanks to jlliagre For This Post:
# 7  
Old 11-26-2012
Also, overallocating memory just because it probably won't all be used is a horrible way to run a critical server with stringent uptime requirements.

When the server does run out of memory, the usual response is to kill the process using the most memory - quite likely the very process that's the entire reason you're running that server in the first place.

Example: you deploy a database server on an OS that overcommits memory. When it runs out of memory, the main database process is killed.

Brilliant. Smilie
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

Low Virtual memory available

Hi I am running AIX 5.2. My server is running low on memory. It it using about 1307775 file pages on a total of 1511424 (from vmstat -v). I looked at the memory yesterday morning, and we had plenty of free memory. I did a backup from Windows (ftp mget command) of a large file selection. From... (5 Replies)
Discussion started by: fredrivard
5 Replies

2. UNIX for Advanced & Expert Users

Virtual file or memory?

Hi Experts I encountered a situation recently. I wanted to discuss here and understand the reason behind this. My scenario is something like this: yes > temp & The above command keeps writing the output to the file temp. And this file keeps growing every second. And in the every... (4 Replies)
Discussion started by: guruprasadpr
4 Replies

3. Linux

Virtual Memory issue

Hi all, I was compiling my glibc 2.6.1 source files on a new kernel 2.66.22.6 and it seems that i am running into issues with the Virtual Memory. It displays the error message: virtual memory exhausted: Cannot allocate memory‏ I saw an article on how to adjust the parameters but i can't... (5 Replies)
Discussion started by: scriptingmani
5 Replies

4. UNIX for Dummies Questions & Answers

Virtual Memory

Hi, Can anyone please help me workout how much virtual memory I have running on a T2000 running Solaris 10. Thanks # df -h swap 3.5G 1.0M 3.5G 1% /etc/svc/volatile swap 3.5G 208K 3.5G 1% /tmp swap 3.5G 56K ... (2 Replies)
Discussion started by: jamba1
2 Replies

5. UNIX for Dummies Questions & Answers

cpu, memory and virtual memory usage

Hi All, Does anyone know what the best commands in the UNIX command line are for obtaining this info: current CPU usage memory usage virtual memory usage preferably with date and time parameters too? thanks ocelot (4 Replies)
Discussion started by: ocelot
4 Replies

6. AIX

ulimits max locked memory virtual memory

Hi, Would any one be so kind to explain me : are ulimits defined for each user seperately ? When ? Specialy what is the impact of : max locked memory and virtual memory on performance of applications for a user. Many thanks. PS : this is what I can see in MAN : ulimit ] ... (5 Replies)
Discussion started by: big123456
5 Replies

7. HP-UX

Virtual Memory

Hi! I work with HP-UX and I have to monitorize the use of virtual memory for different processes. (java processes for Tibco Adapter) And if these processes exceed a limit send a message to the syslog. I donīt know how to monitorize this... Should I do a script? or use an aplication, for example... (3 Replies)
Discussion started by: Kurohana
3 Replies

8. Programming

how allocate virtual memory

Hi Folks can any body suggest how to allocate virtual memory any function for that (2 Replies)
Discussion started by: munnu
2 Replies

9. Programming

about virtual memory and memory leak

Hi, First of all I appreciate this group very much for its informative discussions and posts. Here is my question. I have one process whose virtual memory size increases linearly from 6MB to 12MB in 20 minutes. Does that mean my process has memory leaks? In what cases does the... (4 Replies)
Discussion started by: shriashishpatil
4 Replies

10. UNIX for Dummies Questions & Answers

virtual memory

Hi, can anyone explain me what virtual memory is ( for which we use vmstat commande line ) comparing with RAM ? many thanks before. (2 Replies)
Discussion started by: big123456
2 Replies
Login or Register to Ask a Question