Login or Register to Ask a Question and Join Our Community


Emergency UNIX and Linux Support

Memory leaks on compilations

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Homework and Emergencies Emergency UNIX and Linux Support Memory leaks on compilations
# 1  
Old 10-04-2010
Data Memory leaks on compilations
Hello!

I've been struggling for not few hours with memory leaks on this
machine. I'm running linux 2.6.32-5-686, and the problem is as follows:

Some months ago, I have compiled kernel 2.6.33-2-686 without any issues
in this same machine. This week I have tried compiling GNUzilla Icecat
and the 2.6.35 kernel. But when compiling the kernel, when it is just
starting compiling, all real memory (2GB) fills, and when beginning to
swap, the machine becomes absolutely unresponsive. With Icecat, it
takes 15 minutes before this happens, and today I tried leaving it
working for almost 5 hours, unresponsive, before killing the machine.

I killed the machine almost 10 times this week with high I/O activity.
I've already tried changing vm.swapinness to 60, and
vm.overcommit_memory to 2, but nothing helps. Also disabled the memlock
available for audio applications, but nothing yet. Increased my swap to
10GB, but nothing.

So here goes the output of some commands:

Code:
# sysctl -a | grep vm. | sort
error: permission denied on key 'net.ipv4.route.flush'
error: permission denied on key 'net.ipv6.route.flush'
vm.block_dump = 0
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 500
vm.drop_caches = 0
vm.highmem_is_dirtyable = 0
vm.hugepages_treat_as_movable = 0
vm.hugetlb_shm_group = 0
vm.laptop_mode = 0
vm.legacy_va_layout = 0
vm.lowmem_reserve_ratio = 256    32    32
vm.max_map_count = 65530
vm.memory_failure_early_kill = 0
vm.memory_failure_recovery = 1
vm.min_free_kbytes = 3789
vm.mmap_min_addr = 65536
vm.nr_hugepages = 0
vm.nr_overcommit_hugepages = 0
vm.nr_pdflush_threads = 0
vm.oom_dump_tasks = 0
vm.oom_kill_allocating_task = 0
vm.overcommit_memory = 0
vm.overcommit_ratio = 50
vm.page-cluster = 3
vm.panic_on_oom = 0
vm.percpu_pagelist_fraction = 0
vm.scan_unevictable_pages = 0
vm.stat_interval = 1
vm.swappiness = 60
vm.vdso_enabled = 1
vm.vfs_cache_pressure = 100

Code:
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Code:
$ free -m
             total       used       free     shared    buffers cached
Mem:          1987        437       1549          0    51         192
-/+ buffers/cache:        193       1793
Swap:        10240          0      10240

Code:
# swapon -s
Filename				Type		Size	Used	Priority
/dev/sda1                               partition	10486776	0	-1

Your help will be very appreciated!
Teresa e Junior
# 2  
Old 10-04-2010
I suggest you retry after booting another kernel .
Then try another compiler if you can, and in the order that you like .

Memory leaks often happend by the running prog. in here the compiler(buggy), but might be the kernel, very rare and usually because of a faulty driver .

If it still happends , then the sources are so tight that the compiler is hitting a real bug .

Regards
# 3  
Old 10-04-2010
Hello, remi75!

I tried it with 3 different kernels: the Debian Kernel, the Zen Kernel, and the Grml Kernel. The output of "sysctl -a | grep vm. | sort" is from the Debian Kernel. The Zen Kernel sets swappiness to 0, but the Debian sets it to 60, so... and the Grml is similar to the Debian one.

I'd like to try, but don't know how to use other compiler. I'm compiling from Debian sources, so it is just "debuild -k$GPGKEY", and it calls the compiler probably from debian/rules.

If it is a faulty driver, how could I debug it (without freezing again the system and killing it)?

I don't think the problem is in the sources. I have compiled the Debian kernel a few months ago and hadn't noticed anything wrong.

I suspect of some problem in my partitions? Though when I succeeded in compiling the kernel, I had no swap at all (already tried "swapoff -a", and it just started killing many procs...)

Thanks for your help!
Teresa e Junior
# 4  
Old 10-04-2010
Try debugging by using IBM Rational purify
# 5  
Old 10-04-2010
Oh, I think I will have to compile it! Smilie

Thanks anyway!
# 6  
Old 10-05-2010
Quote:
Originally Posted by teresaejunior
Oh, I think I will have to compile it! Smilie

Thanks anyway!
Compile what ( the kernel ) ? most probably the compiler is alredy buggy !

do an apt-get update style thingy , if one of the packages to update is a compiler (gcc***) go ahead .

otherwise please give me more infos on your distro and gcc -V etc

Regards

---------- Post updated at 09:45 AM ---------- Previous update was at 09:44 AM ----------
Quote:
Originally Posted by vkca
Try debugging by using IBM Rational purify
debug what, the compiler ? it has to be a debug version of gcc besides
i doubt it the user (no effence) will able to ...

Regards

---------- Post updated at 10:27 AM ---------- Previous update was at 09:45 AM ----------
Quote:
Originally Posted by teresaejunior
Hello, remi75!

I tried it with 3 different kernels: the Debian Kernel, the Zen Kernel, and the Grml Kernel. The output of "sysctl -a | grep vm. | sort" is from the Debian Kernel. The Zen Kernel sets swappiness to 0, but the Debian sets it to 60, so... and the Grml is similar to the Debian one.

I'd like to try, but don't know how to use other compiler. I'm compiling from Debian sources, so it is just "debuild -k$GPGKEY", and it calls the compiler probably from debian/rules.

If it is a faulty driver, how could I debug it (without freezing again the system and killing it)?

I don't think the problem is in the sources. I have compiled the Debian kernel a few months ago and hadn't noticed anything wrong.

I suspect of some problem in my partitions? Though when I succeeded in compiling the kernel, I had no swap at all (already tried "swapoff -a", and it just started killing many procs...)

Thanks for your help!
Teresa e Junior
yes but its a definition of a bug - otherwise it will be called a nonworking prog Smilie

you compiler worked with sourceX.c but might not with sourY.c
thats why it would be called a bug ...

i am still waiting on your details .

regards
# 7  
Old 10-05-2010
Hello! Here are some details.

Code:
$ gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.4.5-2'
--with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++
--prefix=/usr --program-suffix=-4.4 --enable-shared --enable-multiarch
--enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib --enable-nls
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
--enable-targets=all --with-arch-32=i586 --with-tune=generic
--enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu
--target=i486-linux-gnu
Thread model: posix
gcc version 4.4.5 (Debian 4.4.5-2)

Code:
uname -a
Linux localhost 2.6.35-grml #1 SMP PREEMPT Sat Sep 4 10:58:14 UTC 2010 i686 GNU/Linux

But I tried also with kernels 2.6.35-6.dmz.2-liquorix-686 and 2.6.32-5-686 from Debian

After talking to you I tried a couple of different things, including "ulimit -v NUMBER" and "ulimit -m NUMBER" before compiling, but these don't seem to be respected, but I got to kill it before it killed me, so the build log is attached. You will notice that right in the beginning of the build it goes mad.

This one also looks problematic, the BSD daemon doesn't look so evil:
Code:
         (__) 
         (oo) 
   /------\/ 
  / |    ||   
 *  /\---/\ 
    ~~   ~~   
...."Have you mooed today?"...

Any more information you might need, I'll be glad to give.

Best regards,
Teresa e Junior

I forgot to mention, I have done a dist-upgrade this morning, but still no results.
linux-family-2.6_2.6.35-17family_i386.build.txt (15.9 KB) 
Last edited by teresaejunior; 10-05-2010 at 07:25 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Memory Leaks

Suppose I have a main() function with only one malloc statement allocating say some 1 gb memory. Also say my system has 1 gb of ram. main() { malloc(1gb) return(0) } The program above exits without freeing the memory. In this case will the 1 gb of heap memory be returned... (9 Replies)
Discussion started by: rupeshkp728
9 Replies

2. Solaris

[DOUBT] Memory high in idle process on Solaris 10 (Memory Utilization > 90%)

Hi Experts, Our servers running Solaris 10 with SAP Application. The memory utilization always >90%, but the process on SAP is too less even nothing. Why memory utilization on solaris always looks high? I have statement about memory on solaris, is this true: Memory in solaris is used for... (4 Replies)
Discussion started by: edydsuranta
4 Replies

3. AIX

valgrind - pthread memory leaks on AIX

Hi all, I have written a small code just to invoke main and return immediately. When built with libpthread on AIX box, valgrind throws lots of memory leak errors. But when built without libpthread, no issues at all. Here is the sample run for your look. Any idea where I might be going wrong?... (3 Replies)
Discussion started by: visionofarun
3 Replies

4. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know... (4 Replies)
Discussion started by: 0ktalmagik
4 Replies

5. UNIX and Linux Applications

Looking for memory leaks freeware tools

Hello all Is there good free ware tools to check software memory leaks ? Some thing like purify on unix platforms sun/hp/linux Thanks (3 Replies)
Discussion started by: umen
3 Replies

6. Shell Programming and Scripting

Find memory leaks shell script ???

Hi all, Has anyone out there a shell script to detect memory leaks on unix machines? And if so what way did they go about it .? (5 Replies)
Discussion started by: nano2
5 Replies

7. UNIX for Advanced & Expert Users

strange problem with memory leaks

Hi Unix lovers, I am facing a strange problem about memory leak. One component of our product show memory leak at customer's end but not in development environment. The memory used by the exe goes on increasing at customer end but not in dev. customer has same m/c(HP unix 11i) , the same... (1 Reply)
Discussion started by: shriashishpatil
1 Replies

8. Programming

Tool for finding memory leaks

hi, i am a c++ programmer working on linux(redhat linux8.0) environment, i need to find out the memory leaks, so far i didn't used any tools, so what are the tools are available, and whic one is good to use. plz provide with a small example. (1 Reply)
Discussion started by: sarwan
1 Replies

9. HP-UX

Memory leaks on HP-UX 11i

Hi folks, We are using following listed configurations for a particular application. HP-UX 11i Sun Java 2 SDK Standard Edition 1.4.1 (version shipped with WebLogic 8) Oracle 9i Release 2 (Oracle 9.2.0) BEA WebLogic Server 8.1 SP3 It seems a memory leak when we use above configurations.... (1 Reply)
Discussion started by: gimhan90
1 Replies

10. UNIX for Advanced & Expert Users

memory leaks

Hi!! Experts, Any ideas how to check for the memory leaks in a process during performance testing?? I dont use purify.. Any way of finding it out using default S/W in HP UX-11 Can U gimme pointers to site having good scripts/tutorials on performance testing?? Thanx in Advance.. :) (3 Replies)
Discussion started by: jyotipg
3 Replies
Login or Register to Ask a Question