Programming

Hi ,

You can use sizeof() operator to find the size of any data structure by providing the base address of the structure. the implementation of memory allocation are different on different architectures and they may allocate slightly more space than you requested for in order to have alignment of the memory allocated which can increase performance benefits.

Thanks and Regards,
Gaurav.

heap is just a moniker given to the dynamic part of the data segment...the part from where memory is allocated at runtime.
Whether mmap or sbrk is used for mem allocation...the point is that it is always allocated from the dynamic portion of the data segment aka the heap.
Not exactly because of the the above.

Incorrect. The "dynamic portion of the data segment" is delimited by the break. The break is manipulated with the brk() and sbrk() system calls. As its name implies, the segment is contiguous. mmap(), however, is allowed to map/allocate memory at random locations in non-contiguous segments beyond the break. Such behavior has nothing to do with the heap.

Perhaps your mmap implementation does not do so, but there's nothing to forbid it.

Alternatively, you have a very loose definition of "data segment".

Regards,
Alister

Yes you are right the heap is capped by the break pointer...and mmap allocates memory in the hole between break and the upper bound of the stack segment.
All segments are contiguous virtually speaking but none are contiguous physically.
mmap is only used in openbsd for dynamic storage allocation and since my system isnt openbsd it still uses malloc.
That might be true as anything between the text and stack segment i'm calling collectively the data segment

I believe that the malloc statements in my original post in this thread are 100% correct. At the risk of appearing to be trying to win an internet argument, I am dissecting your responses only in the hope that the thread contains useful and accurate information for those who may read it in the future.

Well, for the purposes of this discussion, physical memory is irrelevant; it's all about process virtual memory maps. Still, the statement isn't quite accurate. It implies that each segment is guaranteed to be fragmented in physical memory. It very well could be, but it's not a certainty.

OpenBSD uses malloc(), as do all UNIX/POSIX systems (to my knowledge).

You seem to be confusing interfaces that operate at different levels. mmap() is a lower level system call whose implementation is part of the kernel. malloc() is a higher level function whose implementation is part of the userland standard C library. malloc() does not directly allocate any memory; it must invoke a system call to do the job. The question is, which system call is used. Some malloc() implementations use brk()/sbrk(), some use mmap(), some use both.

If you're using Linux, you're almost certainly using glibc, and glibc's malloc implementation uses sbrk() for small allocations and mmap() for large ones.

FreeBSD tries to use mmap() for all allocations but if mmap() fails it will retry with sbrk().

OpenBSD's malloc() uses mmap() exclusively.

I believe OSX's malloc() uses mmap(), but I am not sure.

I could be mistaken, but I suspect that if there are any malloc() implementations still in widespread use which depend exclusively on brk()/sbrk(), they belong to the more conservative, proprietary Unices out there.

You are of course free to choose your nomenclature as you wish, but in this instance your usage is not in agreement with standard practice.

Being a rebel only makes life harder. Submit. Be one with the collective. Resistance is futile.

Regards,
Alister

man mapmalloc

There are a lot of ways to make a heap.

Sun once even released a library that when preloaded would put your heap into shared memory. But it may have only been for SPARC, and it was definitely only for 32-bit processes. It also failed miserably for multithreaded processes, IIRC.

So I wrote a better one for a client who wanted fast restarts for a process that used about a few-hundred GB heap. We'd create and zero-fill a single huge ISM shared memory segment upon system boot, when it would be the fastest since memory wasn't fragmented yet. Then the process would start up in about 10 seconds, instead of the 10+ minutes or longer it would take to create and zero-fill the giant heap.

And yes, you have to zero-fill such a heap upon creation, or whenever your application code that by requirement has to be really, really fast may very well hang up waiting for the OS to actually create the VM page the process merely reserved with its call to brk()/sbrk(). Like trying to do a few tens of GB of IO from a SAN at a few GB/sec into a newly-created chunk of memory that hasn't actually been allocated yet... That can generate some nasty problems since the data is coming in faster than the kernel's VM code can create application pages to put them into.

Or if your system supports it, MAP_POPULATE.

hugepages may also be helpful here.