09-04-2014
Plus, if you repeatedly call malloc/free for varying very large sized chunks of memory, malloc will gladly fragment heap to the point where it becomes less efficient. This is due in part to the fact that some OS flavors may reclaim memory after a free call. Especially if there are other processes calling for memory chunks. Numa also plays into big chunk operations.
Several years ago we ran a test on a non-prod Solaris 10 box with 64GB of memory. We malloced one single giant chunk, never called malloc again. We reused the chunk over and over with varying sized buffers. By adding back in the malloc/free calls between every operation on new "new" chunk, the same test code ran about 15% slower and spent most of that extra time in kernel mode.
NUMA really slows down accessing large memory allocations because of locality issues. The system cannot relocate gigantic memory chunks to more convenient locations. Since you have a commodity cpu (multicoore x86 ) then NUMA is a concern.
You need to look into cpu affinity for threads.
If you are reading from and then writing to vastly distant memory chunks you need to be aware of the order of accessing neighboring memory rather than doing something like copying the contents of arr[0] to arr[2000000], then reading in arr[1000000]. Each one of those example actions can mean reloading an L2 cache - as an example. As it is nowadays, memory is about an order of magnitude or more slower than your cpus.
Edit: You really should consider this article:
http://www.akkadia.org/drepper/cpumemory.pdf
It is somewhat old, but still completely applicable.
Last edited by jim mcnamara; 09-04-2014 at 05:44 PM..
This User Gave Thanks to jim mcnamara For This Post:
10 More Discussions You Might Find Interesting
1. Windows & DOS: Issues & Discussions
Hi There,
I have upgraded the DELL poweredge 2600 server memory from 2GB to 4GB. However, the memory only showed at 2GB of utilization. How to make sure that the server is full utilize of 4GB of memory. Is there the Virtual memory need to be reconfigure as this server is run on windows 2000 and... (2 Replies)
Discussion started by: vestro
2 Replies
2. Linux
Hi All
Any idea? why am I having this memory issue?
perforce@ixca-pforce-bak:/home/p4-demo-root$ diff checkpoint_offline ../p4-demo-root2/checkpoint.1150
diff: memory exhausted
Thanks a lot
C Saha (0 Replies)
Discussion started by: csaha
0 Replies
3. AIX
Hi friends..
Help to solve this issue...
Is there any parameter setting to control or limit the size of the shared memory a process can attach for the below specified environment?
The man pages says it can attach upto segments of size 2GB. But when our process (which also connects to... (0 Replies)
Discussion started by: sdspawankumar
0 Replies
4. Linux
Hi all,
I was compiling my glibc 2.6.1 source files on a new kernel 2.66.22.6 and it seems that i am running into issues with the Virtual Memory. It displays the error message:
virtual memory exhausted: Cannot allocate memory
I saw an article on how to adjust the parameters but i can't... (5 Replies)
Discussion started by: scriptingmani
5 Replies
5. Solaris
One of our project has exceeded its assigned max-memory-locked by 3 times .. The said project is using around 9 gigs as described by rss parameter in prstat -J .. and the max-project-memory parameter has been defined as 3gigs .. is it normal or we are monitoring the project memory usage in wrong... (2 Replies)
Discussion started by: fugitive
2 Replies
6. AIX
Currently server have load while there is no heavy things running, just oracle database/ application server oracle. I don't understand why server have heavy load, 22GB is under buffer, how to clean buffer/memory in AIX
load averages: 9.42, 9.43, 9.68; 05:25:08
141 processes: 125 idle, 16... (12 Replies)
Discussion started by: learnbash
12 Replies
7. Solaris
We have a zone configured in our X4600 machine with memory capped to 16GB. Most of the time zone is running with high physical memory utilization. It seems from "top" command shows that the command "kernel" is locks 15GB phy. memory and not using swap memory. Whenever we restart the application... (2 Replies)
Discussion started by: rock123
2 Replies
8. AIX
I have a server with 300Gb allocated to it. Some times I observed in topas
Comp% 73 and Non comp 35% and client is also 35% and my paging is showing 92%. If my physical memory utilized only 70% then why paging is so high.
And what is relation between Comp, noncomp and client? If the memory... (1 Reply)
Discussion started by: powerAIX
1 Replies
9. Red Hat
I could not find what is consuming the memory, generated DSET reports and NO hardware wise memory issue. 64 GB RAM on a server yet all I could see is a very limited memory available. I am not sure if I am reading this correct or not. I have used free -t -m and cat /proc/meminfo (results below)... (3 Replies)
Discussion started by: rsheikh01
3 Replies
10. Linux
Hi,
In our production box i can see the Swap space using the below command
free
total used free shared buffers cached
Mem: 65963232 41041084 24922148 0 877160 35936292
-/+ buffers/cache: 4227632 61735600
Swap: 4192880 ... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies
MALLOC(3) Linux Programmer's Manual MALLOC(3)
NAME
calloc, malloc, free, realloc - Allocate and free dynamic memory
SYNOPSIS
#include <stdlib.h>
void *calloc(size_t nmemb, size_t size);
void *malloc(size_t size);
void free(void *ptr);
void *realloc(void *ptr, size_t size);
DESCRIPTION
calloc() allocates memory for an array of nmemb elements of size bytes each and returns a pointer to the allocated memory. The memory is
set to zero.
malloc() allocates size bytes and returns a pointer to the allocated memory. The memory is not cleared.
free() frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Oth-
erwise, or if free(ptr) has already been called before, undefined behaviour occurs. If ptr is NULL, no operation is performed.
realloc() changes the size of the memory block pointed to by ptr to size bytes. The contents will be unchanged to the minimum of the old
and new sizes; newly allocated memory will be uninitialized. If ptr is NULL, the call is equivalent to malloc(size); if size is equal to
zero, the call is equivalent to free(ptr). Unless ptr is NULL, it must have been returned by an earlier call to malloc(), calloc() or
realloc().
RETURN VALUE
For calloc() and malloc(), the value returned is a pointer to the allocated memory, which is suitably aligned for any kind of variable, or
NULL if the request fails.
free() returns no value.
realloc() returns a pointer to the newly allocated memory, which is suitably aligned for any kind of variable and may be different from
ptr, or NULL if the request fails. If size was equal to 0, either NULL or a pointer suitable to be passed to free() is returned. If real-
loc() fails the original block is left untouched - it is not freed or moved.
CONFORMING TO
ANSI-C
SEE ALSO
brk(2), posix_memalign(3)
NOTES
The Unix98 standard requires malloc(), calloc(), and realloc() to set errno to ENOMEM upon failure. Glibc assumes that this is done (and
the glibc versions of these routines do this); if you use a private malloc implementation that does not set errno, then certain library
routines may fail without having a reason in errno.
Crashes in malloc(), free() or realloc() are almost always related to heap corruption, such as overflowing an allocated chunk or freeing
the same pointer twice.
Recent versions of Linux libc (later than 5.4.23) and GNU libc (2.x) include a malloc implementation which is tunable via environment vari-
ables. When MALLOC_CHECK_ is set, a special (less efficient) implementation is used which is designed to be tolerant against simple
errors, such as double calls of free() with the same argument, or overruns of a single byte (off-by-one bugs). Not all such errors can be
protected against, however, and memory leaks can result. If MALLOC_CHECK_ is set to 0, any detected heap corruption is silently ignored;
if set to 1, a diagnostic is printed on stderr; if set to 2, abort() is called immediately. This can be useful because otherwise a crash
may happen much later, and the true cause for the problem is then very hard to track down.
Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the mem-
ory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the infamous OOM
killer.
GNU
1993-04-04 MALLOC(3)