Sponsored Content
Full Discussion: Memory release latency issue
Operating Systems Linux Red Hat Memory release latency issue Post 302915767 by Corona688 on Thursday 4th of September 2014 02:46:29 PM
Old 09-04-2014
In short, malloc() is the wrong tool for throwing around entire gigabytes of memory at once. You should cut out the middleman and use mmap().

The first time you request an entire gigabyte of memory, malloc() probably has to call brk to extend the heap segment. (This is a system memory call related to mmap.) This adds a vast new region of unused memory to the heap -- memory that's all guaranteed to hold nothing but ASCII NULLs. It just gives it straight to you and doesn't bother to clean it.

Then you free() and malloc() it again. Because it's been used, malloc() will memset() that entire gig of memory to NULL for you to make it "clean" again.

By using mmap() instead, you can let the OS do that as-needed instead of in one 1-gig write. mmap also has other useful features like file backing -- if all you're doing is dumping 5 gigs of file into memory, mmap can save you a ton of trouble and speed and RAM.

Or, if you went the other direction, you could just keep reusing the same block of memory around all the time without free()ing it.

Last edited by Corona688; 09-04-2014 at 03:56 PM..
 

10 More Discussions You Might Find Interesting

1. Windows & DOS: Issues & Discussions

Memory Issue

Hi There, I have upgraded the DELL poweredge 2600 server memory from 2GB to 4GB. However, the memory only showed at 2GB of utilization. How to make sure that the server is full utilize of 4GB of memory. Is there the Virtual memory need to be reconfigure as this server is run on windows 2000 and... (2 Replies)
Discussion started by: vestro
2 Replies

2. Linux

Memory issue while diff !!!

Hi All Any idea? why am I having this memory issue? perforce@ixca-pforce-bak:/home/p4-demo-root$ diff checkpoint_offline ../p4-demo-root2/checkpoint.1150 diff: memory exhausted Thanks a lot C Saha (0 Replies)
Discussion started by: csaha
0 Replies

3. AIX

Shared memory issue

Hi friends.. Help to solve this issue... Is there any parameter setting to control or limit the size of the shared memory a process can attach for the below specified environment? The man pages says it can attach upto segments of size 2GB. But when our process (which also connects to... (0 Replies)
Discussion started by: sdspawankumar
0 Replies

4. Linux

Virtual Memory issue

Hi all, I was compiling my glibc 2.6.1 source files on a new kernel 2.66.22.6 and it seems that i am running into issues with the Virtual Memory. It displays the error message: virtual memory exhausted: Cannot allocate memory‏ I saw an article on how to adjust the parameters but i can't... (5 Replies)
Discussion started by: scriptingmani
5 Replies

5. Solaris

Locked memory issue

One of our project has exceeded its assigned max-memory-locked by 3 times .. The said project is using around 9 gigs as described by rss parameter in prstat -J .. and the max-project-memory parameter has been defined as 3gigs .. is it normal or we are monitoring the project memory usage in wrong... (2 Replies)
Discussion started by: fugitive
2 Replies

6. AIX

AIX memory issue

Currently server have load while there is no heavy things running, just oracle database/ application server oracle. I don't understand why server have heavy load, 22GB is under buffer, how to clean buffer/memory in AIX load averages: 9.42, 9.43, 9.68; 05:25:08 141 processes: 125 idle, 16... (12 Replies)
Discussion started by: learnbash
12 Replies

7. Solaris

Zone memory issue

We have a zone configured in our X4600 machine with memory capped to 16GB. Most of the time zone is running with high physical memory utilization. It seems from "top" command shows that the command "kernel" is locks 15GB phy. memory and not using swap memory. Whenever we restart the application... (2 Replies)
Discussion started by: rock123
2 Replies

8. AIX

Memory issue

I have a server with 300Gb allocated to it. Some times I observed in topas Comp% 73 and Non comp 35% and client is also 35% and my paging is showing 92%. If my physical memory utilized only 70% then why paging is so high. And what is relation between Comp, noncomp and client? If the memory... (1 Reply)
Discussion started by: powerAIX
1 Replies

9. Red Hat

Memory Issue

I could not find what is consuming the memory, generated DSET reports and NO hardware wise memory issue. 64 GB RAM on a server yet all I could see is a very limited memory available. I am not sure if I am reading this correct or not. I have used free -t -m and cat /proc/meminfo (results below)... (3 Replies)
Discussion started by: rsheikh01
3 Replies

10. Linux

Swap memory issue

Hi, In our production box i can see the Swap space using the below command free total used free shared buffers cached Mem: 65963232 41041084 24922148 0 877160 35936292 -/+ buffers/cache: 4227632 61735600 Swap: 4192880 ... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies
MALLOC(3)						     Linux Programmer's Manual							 MALLOC(3)

NAME
malloc, free, calloc, realloc - allocate and free dynamic memory SYNOPSIS
#include <stdlib.h> void *malloc(size_t size); void free(void *ptr); void *calloc(size_t nmemb, size_t size); void *realloc(void *ptr, size_t size); DESCRIPTION
The malloc() function allocates size bytes and returns a pointer to the allocated memory. The memory is not initialized. If size is 0, then malloc() returns either NULL, or a unique pointer value that can later be successfully passed to free(). The free() function frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Otherwise, or if free(ptr) has already been called before, undefined behavior occurs. If ptr is NULL, no operation is per- formed. The calloc() function allocates memory for an array of nmemb elements of size bytes each and returns a pointer to the allocated memory. The memory is set to zero. If nmemb or size is 0, then calloc() returns either NULL, or a unique pointer value that can later be success- fully passed to free(). The realloc() function changes the size of the memory block pointed to by ptr to size bytes. The contents will be unchanged in the range from the start of the region up to the minimum of the old and new sizes. If the new size is larger than the old size, the added memory will not be initialized. If ptr is NULL, then the call is equivalent to malloc(size), for all values of size; if size is equal to zero, and ptr is not NULL, then the call is equivalent to free(ptr). Unless ptr is NULL, it must have been returned by an earlier call to mal- loc(), calloc() or realloc(). If the area pointed to was moved, a free(ptr) is done. RETURN VALUE
The malloc() and calloc() functions return a pointer to the allocated memory that is suitably aligned for any kind of variable. On error, these functions return NULL. NULL may also be returned by a successful call to malloc() with a size of zero, or by a successful call to calloc() with nmemb or size equal to zero. The free() function returns no value. The realloc() function returns a pointer to the newly allocated memory, which is suitably aligned for any kind of variable and may be dif- ferent from ptr, or NULL if the request fails. If size was equal to 0, either NULL or a pointer suitable to be passed to free() is returned. If realloc() fails the original block is left untouched; it is not freed or moved. CONFORMING TO
C89, C99. NOTES
By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer. For more information, see the description of /proc/sys/vm/overcommit_memory and /proc/sys/vm/oom_adj in proc(5), and the Linux kernel source file Documentation/vm/overcommit-accounting. Normally, malloc() allocates memory from the heap, and adjusts the size of the heap as required, using sbrk(2). When allocating blocks of memory larger than MMAP_THRESHOLD bytes, the glibc malloc() implementation allocates the memory as a private anonymous mapping using mmap(2). MMAP_THRESHOLD is 128 kB by default, but is adjustable using mallopt(3). Allocations performed using mmap(2) are unaffected by the RLIMIT_DATA resource limit (see getrlimit(2)). To avoid corruption in multithreaded applications, mutexes are used internally to protect the memory-management data structures employed by these functions. In a multithreaded application in which threads simultaneously allocate and free memory, there could be contention for these mutexes. To scalably handle memory allocation in multithreaded applications, glibc creates additional memory allocation arenas if mutex contention is detected. Each arena is a large region of memory that is internally allocated by the system (using brk(2) or mmap(2)), and managed with its own mutexes. The UNIX 98 standard requires malloc(), calloc(), and realloc() to set errno to ENOMEM upon failure. Glibc assumes that this is done (and the glibc versions of these routines do this); if you use a private malloc implementation that does not set errno, then certain library routines may fail without having a reason in errno. Crashes in malloc(), calloc(), realloc(), or free() are almost always related to heap corruption, such as overflowing an allocated chunk or freeing the same pointer twice. Recent versions of Linux libc (later than 5.4.23) and glibc (2.x) include a malloc() implementation which is tunable via environment vari- ables. For details, see mallopt(3). SEE ALSO
brk(2), mmap(2), alloca(3), malloc_get_state(3), malloc_info(3), malloc_trim(3), malloc_usable_size(3), mallopt(3), mcheck(3), mtrace(3), posix_memalign(3) COLOPHON
This page is part of release 3.53 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. GNU
2012-05-10 MALLOC(3)
All times are GMT -4. The time now is 06:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy