calloc fails: 'Cannot allocate memory'


 
Thread Tools Search this Thread
Top Forums Programming calloc fails: 'Cannot allocate memory'
# 1  
Old 02-20-2012
Data calloc fails: 'Cannot allocate memory'

Hi , experts.

I work on Linux station (RedHat 5.7), regular user, but have root password.

%> uname -a
Linux ran1log06 2.6.18-238.1.1.el5 #1 SMP Tue Jan 4 13:32:19 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
%> cat /etc/issue
Red Hat Enterprise Linux Client release 5.7 (Tikanga)
Kernel \r on an \m


My C code application allocates memory by a lot of calloc(...)

The application fails - calloc returns NULL pointer and error "Cannot allocate memory" Smilie

But I see a lot of free memory here:

%> free -t
total used free shared buffers cached
Mem: 8017116 7872188 144928 0 113356 2409372
-/+ buffers/cache: 5349460 2667656
Swap: 2097144 184 2096960
Total: 10114260 7872372 2241888


there is no problem with limit also:
%> limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize unlimited
coredumpsize unlimited
memoryuse unlimited
vmemoryuse unlimited
descriptors 1024
memorylocked 32 kbytes
maxproc 65535


I also ran it under
It runs fine in other linux, also Red Hat 5.7

What may be the problem here?
May be some limit on the process on ran1log06 machine?
How can I check it?













----------------------------------------
Note: This post may be relates to "Programming" forum.
# 2  
Old 02-20-2012
You can fragment memory by making a lots of calls to malloc or calloc, freeing some blocks and keeping others. After a while substantially larger allocation requests fail because there is not enough free contiguous memory.

Based on your free -t output I would say you have a lot of memory. So, my best guess is fragmentation.

As a possible fix, consider adding more (way more) swap, say 50GB.

But - whatever your code does it would appear not to play well with others. Running it on a busy production system could bring that system to its knees. Your code needs help.

Start by debugging your malloc calls (calloc() calls malloc() )
There are good malloc debugging libraries out there for linux for example:
Dmalloc - Debug Malloc Home Page This will show where your problems occur.
# 3  
Old 02-21-2012
Jim, thanks for reply.

I can not add swap on this station.
I succeeded to stop the program just before the exit out and run /bin/top - to see how memory it consumes: VIRT=79248, RES=28m Smilie
when there is a lot of free memory: ~400MB

It looks so:


%> top
Mem: 8017116k total, 7601900k used, 415216k free, 21300k buffers
Swap: 2097144k total, 184k used, 2096960k free, 2211488k cached

14315 baruchgu 25 0 79248 28m 472 R 99.8 0.4 0:43.47 vgen



Why malloc may not allocate more?
Again - it runs fine on five other linux stations.

note: I go to see into Dmalloc
# 4  
Old 02-23-2012
Network calloc fails: 'Cannot allocate memory'

Hi , experts.

I work on Linux station (RedHat 5.7), regular user, but have a root password.

Code:
%> uname -a
Linux ran1log06 2.6.18-238.1.1.el5 #1 SMP Tue Jan 4 13:32:19 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
%> cat /etc/issue
Red Hat Enterprise Linux Client release 5.7 (Tikanga)
Kernel \r on an \m

My C code application allocates memory by a lot of calloc(...)

The application fails when the station has many processes - calloc returns NULL pointer and error "Cannot allocate memory"
I see in /bin/top, that my program holds ~30MB RES memory and ~80MB vitual. Smilie

But there is a lot of free memory here.

Code:
%> /bin/top
.....
Mem:   8017116k total,  7977268k used,    39848k free,   223948k buffers
Swap:  2097144k total,      184k used,  2096960k free,  3070888k cached

there is no problem with limit also:
Code:
%> limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize unlimited
coredumpsize unlimited
memoryuse unlimited
vmemoryuse unlimited
descriptors 1024
memorylocked 32 kbytes
maxproc 65535

It runs fine in other not loaded Linux systems, also Red Hat 5.7

What may be the problem here?
May be some limit on the process on this system?
Or gcc compiler limits it?
How can I check it?


Note: I have already ported this problem in "Linux" forum. But got no help till now.

Last edited by baruchgu; 02-23-2012 at 07:04 AM..
# 5  
Old 02-23-2012
Think of process heap memory as a chessboard - one area made up of 64 small blocks.
suppose each one of those is 1 MB.

You make 64 calls to calloc, get 64 pointers to 1MB each. Next you free 10 of them at random.
How many will contiguous? Probably none. Will you have four adjacent squares. Very probably: No.

Now what happens when you want to calloc 4MB? You have 10MB free but it is not contiguous. Some versions of malloc will barf on this, some will call brk() to get an additional n MB from the OS. This is heap fragmentation. You now know that there are several different versions of malloc code. When you have malloc problems like you are having:
1. Try a malloc debugger
2. See what malloc you are using, change it if need be.


sysadmins can control during system installation and later system setup which version(s) of malloc are on a system, and which version of malloc is used by default during linking. So can other programmers.

Different versions of gcc also have different malloc implementations.

This is hypothesis you get to test with dmalloc. And other tools.

Code:
# show if there is library interposition

echo $LD_PRELOAD  # during runtime

# see what .so files you are really linked against
ldd compiled_program

# gcc version - note: this tells you how gcc was compiled which affects your problem.
gcc --version

Each of these results suggests a different workaround.
Your problem box may show something different than what you see elsewhere.

Did you compile locally or port the executable directly?

Last edited by jim mcnamara; 02-23-2012 at 08:36 AM..
This User Gave Thanks to jim mcnamara For This Post:
# 6  
Old 02-23-2012
That is incorrect. The last post in your last thread isn't even yours. You already got answers, suggestions -- you just didn't like them.

In any case, we can hardly help you without seeing your code.

Why not post it in your original thread?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

unable to allocate enough memory

On SunOS 10 get an error when starting a large Java process with over 2Gb memory. Error occurred during initialization of VM Could not reserve enough space for object heap i have 32G memory !! , swap = 31G Please any advice !!! (3 Replies)
Discussion started by: moata_u
3 Replies

2. Programming

How to allocate memory to a string in C?

hi I want to take string as a input from user and the string is very very length. From the lengthy string i have to substring take first 16 letters, then next 8 letters,................... Please guide me how to write program to take lengthy string from user and sub string it. Thanks (4 Replies)
Discussion started by: atharalikhan
4 Replies

3. Linux

shmget failed - cannot allocate memory

Hi, In my proj, one process was running for 2 days. after 2 days its throwing an error message "shmget failed cannot allocate memory". the same problem happened every time.i.e. i can reproduce the same issue if my process is running for every 2 days for a same operation.Within this 2 days there... (1 Reply)
Discussion started by: ManoharanMani
1 Replies

4. Linux

shmget failed - cannot allocate memory

Hi, In my proj, one process was running for 2 days. after 2 days its throwing an error message "shmget failed cannot allocate memory". the same problem happened every time.i.e. i can reproduce the same issue if my process is running for every 2 days for a same operation.Within this 2 days there... (1 Reply)
Discussion started by: ManoharanMani
1 Replies

5. SuSE

shmget failed - cannot allocate memory

Hi, In my proj, one process was running for 2 days. after 2 days its throwing an error message "shmget failed cannot allocate memory". the same problem happened every time.i.e. i can reproduce the same issue if my process is running for every 2 days for a same operation.Within this 2 days there... (1 Reply)
Discussion started by: ManoharanMani
1 Replies

6. UNIX for Advanced & Expert Users

Oracle how many memory allocate really

hi... i want to find oracle allocate how many memory really.. i execute this code to list memory on unix system : ps -eo pmem,args | sort -k 1 -r -n outputs ; %mem command 12.9|ora_smon_RTX 12.9|ora_s000_RTX 12.9|ora_reco_RTX 12.9|ora_qmnc_RTX... (2 Replies)
Discussion started by: utoptas
2 Replies

7. UNIX for Advanced & Expert Users

How to allocate memory

Hi, I have 2 systems with same hardware and software. One system is giving me error "Error occurred during initialization of VM Could not reserve enough space for object " when I tried to increase JBoss App's heap size to 2GB while the other system is running fine without any issue. Is there... (5 Replies)
Discussion started by: ravi3553
5 Replies

8. UNIX for Advanced & Expert Users

unable to allocate enough memory

On SunOS 5.8 I get an error when starting a large Java process with over 2Gb memory. Error occurred during initialization of VM Could not reserve enough space for object heap When stopping several other Java processes we can start this process. This seems to indicate that we don't have... (11 Replies)
Discussion started by: rein
11 Replies

9. Programming

how allocate virtual memory

Hi Folks can any body suggest how to allocate virtual memory any function for that (2 Replies)
Discussion started by: munnu
2 Replies

10. UNIX for Dummies Questions & Answers

Unix Help - allocate more memory to /tmp

Hi Guys I using Solaris 8 and I need to change the size of my /tmp file. Was wondering does anyone know how to do this. Thanks Carson (1 Reply)
Discussion started by: cmackin
1 Replies
Login or Register to Ask a Question