Login or Register to Ask a Question and Join Our Community


Emergency UNIX and Linux Support

Memory leaks on compilations

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Homework and Emergencies Emergency UNIX and Linux Support Memory leaks on compilations
# 15  
Old 10-05-2010
MySQL
Quote:
Originally Posted by teresaejunior
4.3.5-4 in testing, and 4.5.1-8 in experimental. I'll try 4.3.5-4 now to see if it helps, and then 4.5.1-8 from experimental, but without enabling the experimental repositories. There is also 4.1.2-29, which already hit stable (maybe should be the wisest choice).

Actually, I find it more stable than testing!
Ok good luck Smilie keep us informed if you 'd like .


Regards
# 16  
Old 10-05-2010
I tried the 4.1.3 one, which is the more stable. Still no results... I suspect of some loaded kernel module (maybe a third-party) or some daemon...

It's difficult to debug: I left htop open while compiling, and it shows memory usage growing, but doesn't show which process is using this memory... also because the memory fills to fast, I have not time to see what's going on. The memory owned by gcc doesn't seem that big.

I have written a live system installer, and within some days I need to retest it. It means I'm actually forced to do a clean system install soon. So, maybe I'll get rid of this problem after that... < I'm just afraid this won't happen!

Thanks for all your help!
Teresa e Junior
# 17  
Old 10-05-2010
hmm, ok

Can you please get us your lsmod ?

But if the memory gets maxed out only when you run a compiler, it can be only it .

I ll see what your command line (that generates the compiling command) does exacly when i got a minute (i am at work ...no linuxes under my hand), maybe it s calling another set of progs that might be buggy .

Thanx
These 2 Users Gave Thanks to remi75 For This Post:
Neo teresaejunior
# 18  
Old 10-05-2010
Code:
$ lsmod
Module                  Size  Used by
ppp_deflate             3312  0 
zlib_deflate           19179  1 ppp_deflate
bsd_comp                4663  0 
ppp_async               6347  1 
crc_ccitt               1319  1 ppp_async
ppp_generic            21927  7 ppp_deflate,bsd_comp,ppp_async
slhc                    5247  1 ppp_generic
option                 12973  1 
usb_wwan                9505  1 option
usbserial              31470  4 option,usb_wwan
nls_utf8                1005  0 
isofs                  30044  0 
usb_storage            38770  0 
parport_pc             29263  0 
ppdev                   5282  0 
lp                      7629  0 
parport                29803  3 parport_pc,ppdev,lp
sco                     7323  2 
bnep                    9305  2 
rfcomm                 31365  0 
l2cap                  35610  6 bnep,rfcomm
bluetooth              49116  6 sco,bnep,rfcomm,l2cap
ipt_REJECT              1945  1 
ipt_LOG                 4426  5 
ipt_REDIRECT             929  1 
xt_owner                 886  2 
iptable_nat             3555  1 
xt_limit                1266  7 
xt_tcpudp               1895  10 
ipt_addrtype            1483  4 
xt_state                 918  7 
ip6table_filter         1042  1 
ip6_tables             11372  1 ip6table_filter
ipv6                  264324  22 
nf_nat_irc              1168  0 
nf_conntrack_irc        3444  1 nf_nat_irc
nf_nat_ftp              1398  0 
nf_nat                 15739  4 ipt_REDIRECT,iptable_nat,nf_nat_irc,nf_nat_ftp
nf_conntrack_ipv4      10006  10 iptable_nat,nf_nat
nf_defrag_ipv4          1053  1 nf_conntrack_ipv4
nf_conntrack_ftp        5521  1 nf_nat_ftp
nf_conntrack           61885  8 iptable_nat,xt_state,nf_nat_irc,nf_conntrack_irc,nf_nat_ftp,nf_nat,nf_conntrack_ipv4,nf_conntrack_ftp
iptable_filter          1102  1 
ip_tables              10133  2 iptable_nat,iptable_filter
x_tables               15028  13 ipt_REJECT,ipt_LOG,ipt_REDIRECT,xt_owner,iptable_nat,xt_limit,xt_tcpudp,ipt_addrtype,xt_state,ip6table_filter,ip6_tables,iptable_filter,ip_tables
fuse                   61261  3 
uvcvideo               54152  0 
videodev               42308  1 uvcvideo
rtc_cmos                8151  0 
v4l1_compat            12753  2 uvcvideo,videodev
rtc_core               13327  1 rtc_cmos
joydev                  8594  0 
arc4                    1101  2 
rtc_lib                 2197  1 rtc_core
jmb38x_ms               7222  0 
ecb                     1595  2 
snd_hda_codec_conexant    28498  1 
snd_hda_codec_intelhdmi     9286  1 
snd_hda_intel          20423  2 
snd_hda_codec          78538  3 snd_hda_codec_conexant,snd_hda_codec_intelhdmi,snd_hda_intel
iwlagn                136564  0 
snd_hwdep               5200  1 snd_hda_codec
snd_pcm_oss            33067  0 
snd_mixer_oss          12856  1 snd_pcm_oss
iwlcore                92738  1 iwlagn
snd_pcm                66760  3 snd_hda_intel,snd_hda_codec,snd_pcm_oss
tpm_tis                 7338  0 
mac80211              178519  2 iwlagn,iwlcore
snd_timer              17635  1 snd_pcm
snd                    51309  12 snd_hda_codec_conexant,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_pcm_oss,snd_mixer_oss,snd_pcm,snd_timer
soundcore               6331  1 snd
snd_page_alloc          6896  2 snd_hda_intel,snd_pcm
processor              27642  2 
cfg80211              138040  3 iwlagn,iwlcore,mac80211
rfkill                 15268  3 bluetooth,cfg80211
psmouse                50426  0 
evdev                   7092  21 
tpm                    13037  1 tpm_tis
tpm_bios                4705  1 tpm
memstick                7382  1 jmb38x_ms
serio_raw               3640  0 
ac                      2394  0 
battery                 8142  0 
ext4                  236068  2 
mbcache                 5918  1 ext4
jbd2                   53697  1 ext4
crc16                   1319  2 l2cap,ext4
sg                     19038  0 
i915                  262539  2 
sd_mod                 35317  4 
sr_mod                 13352  0 
cdrom                  33344  1 sr_mod
drm_kms_helper         29051  1 i915
drm                   160750  3 i915,drm_kms_helper
ahci                   18537  0 
libahci                19561  4 ahci
sdhci_pci               6046  0 
sdhci                  14882  1 sdhci_pci
libata                168958  2 ahci,libahci
tg3                   118266  0 
mmc_core               57329  1 sdhci
i2c_algo_bit            4736  1 i915
uhci_hcd               19803  0 
thermal                10464  0 
intel_agp              26929  2 i915
agpgart                29779  2 drm,intel_agp
led_class               2505  1 sdhci
ehci_hcd               34294  0 
i2c_core               20464  5 videodev,i915,drm_kms_helper,drm,i2c_algo_bit
video                  16881  1 i915
output                  1691  1 video
button                  4406  1 i915

Thank you!
Teresa e Junior
# 19  
Old 10-05-2010
Smilie a very busy server , nice Smilie (adsl/atm , firewall ...wireless... Smilie )
anyway, i love challeges, if you have enought patience we could try another thing :
compile under root please .
before lunching the compilation, can you throw in a loop in another shell / terminal that dumps a ps -edf or ps aux to a file , and at the end of the script (the loop) do a sync , etc ...

like

while true
do
ps -edf > /tmp/mylog.txt
sync
sleep 1
done

there is another like sleep command that takes shorter delays than a second, if u can find it , a half second would be even better Smilie
so when the server crashes, the file should be on disk,
I know this is hardcore but hell , that's how we discover things, and as you said you might reinstall it anyway .
after the crash (sorry !!! ) that file will contain the answer Smilie
These 2 Users Gave Thanks to remi75 For This Post:
Neo teresaejunior
# 20  
Old 10-05-2010
Computer [SOLVED]
Finally, unbelievable! This is what I love about UNIX systems. You're never hopeless!

So, I've done all the steps, but the problem was actually an endless call of a make option!

I have a script which calls debuild and gives it the `-j' option, then debuild calls dpkg-buildpackage and gives it the same option again, then dpkg-buildpackage calls make and gives it the same option...

I was using -j2, and had already tried changing it, but I ended unsetting it, so it was being called without arguments. From `man make':

Quote:
-j [jobs], --jobs[=jobs]
Specifies the number of jobs (commands) to run simultaneously. If
there is more than one -j option, the last one is effective. If
the -j option is given without an argument, make will not limit
the number of jobs that can run simultaneously.
"make will not limit the number of jobs that can run simultaneously", very enough!.

Also, strange that when called as -j2, it was enough for it to go nuts. Without the option -j, memory usage is relatively low!
See: https://forums.gentoo.org/viewtopic-...6-start-0.html
When compiling icecat, memory usage reached 7GB!!! I still believe this is a bug in make.

EDIT: my intention was using both CPU cores to compile faster, so does it mean it is not possible?

Thank you so much!
Teresa e Junior
Last edited by teresaejunior; 10-05-2010 at 02:20 PM..
This User Gave Thanks to teresaejunior For This Post:
Neo
# 21  
Old 10-06-2010
Quote:
Originally Posted by teresaejunior
Finally, unbelievable! This is what I love about UNIX systems. You're never hopeless!

So, I've done all the steps, but the problem was actually an endless call of a make option!

I have a script which calls debuild and gives it the `-j' option, then debuild calls dpkg-buildpackage and gives it the same option again, then dpkg-buildpackage calls make and gives it the same option...

I was using -j2, and had already tried changing it, but I ended unsetting it, so it was being called without arguments. From `man make':



"make will not limit the number of jobs that can run simultaneously", very enough!.

Also, strange that when called as -j2, it was enough for it to go nuts. Without the option -j, memory usage is relatively low!
See: https://forums.gentoo.org/viewtopic-...6-start-0.html
When compiling icecat, memory usage reached 7GB!!! I still believe this is a bug in make.

EDIT: my intention was using both CPU cores to compile faster, so does it mean it is not possible?

Thank you so much!
Teresa e Junior

Of course it is possible Smilie

But when you build thru scripts the -j option is very often badly handeled ,
i forgot where, but passing to scripts like make-kpkg i think, -jX will break the whole process of the compilation if passed by hand
but if you put it in /etc/make-kpkg.conf (i think (i dont have all the names in mide)) it will be just fine Smilie i just love seeing the 4 procs at 90% working and gcc runing all over the place Smilie

That's why I usually go or all manually , or all automatically by editing conf files ...


Voila Smilie have a great one
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Memory Leaks

Suppose I have a main() function with only one malloc statement allocating say some 1 gb memory. Also say my system has 1 gb of ram. main() { malloc(1gb) return(0) } The program above exits without freeing the memory. In this case will the 1 gb of heap memory be returned... (9 Replies)
Discussion started by: rupeshkp728
9 Replies

2. Solaris

[DOUBT] Memory high in idle process on Solaris 10 (Memory Utilization > 90%)

Hi Experts, Our servers running Solaris 10 with SAP Application. The memory utilization always >90%, but the process on SAP is too less even nothing. Why memory utilization on solaris always looks high? I have statement about memory on solaris, is this true: Memory in solaris is used for... (4 Replies)
Discussion started by: edydsuranta
4 Replies

3. AIX

valgrind - pthread memory leaks on AIX

Hi all, I have written a small code just to invoke main and return immediately. When built with libpthread on AIX box, valgrind throws lots of memory leak errors. But when built without libpthread, no issues at all. Here is the sample run for your look. Any idea where I might be going wrong?... (3 Replies)
Discussion started by: visionofarun
3 Replies

4. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know... (4 Replies)
Discussion started by: 0ktalmagik
4 Replies

5. UNIX and Linux Applications

Looking for memory leaks freeware tools

Hello all Is there good free ware tools to check software memory leaks ? Some thing like purify on unix platforms sun/hp/linux Thanks (3 Replies)
Discussion started by: umen
3 Replies

6. Shell Programming and Scripting

Find memory leaks shell script ???

Hi all, Has anyone out there a shell script to detect memory leaks on unix machines? And if so what way did they go about it .? (5 Replies)
Discussion started by: nano2
5 Replies

7. UNIX for Advanced & Expert Users

strange problem with memory leaks

Hi Unix lovers, I am facing a strange problem about memory leak. One component of our product show memory leak at customer's end but not in development environment. The memory used by the exe goes on increasing at customer end but not in dev. customer has same m/c(HP unix 11i) , the same... (1 Reply)
Discussion started by: shriashishpatil
1 Replies

8. Programming

Tool for finding memory leaks

hi, i am a c++ programmer working on linux(redhat linux8.0) environment, i need to find out the memory leaks, so far i didn't used any tools, so what are the tools are available, and whic one is good to use. plz provide with a small example. (1 Reply)
Discussion started by: sarwan
1 Replies

9. HP-UX

Memory leaks on HP-UX 11i

Hi folks, We are using following listed configurations for a particular application. HP-UX 11i Sun Java 2 SDK Standard Edition 1.4.1 (version shipped with WebLogic 8) Oracle 9i Release 2 (Oracle 9.2.0) BEA WebLogic Server 8.1 SP3 It seems a memory leak when we use above configurations.... (1 Reply)
Discussion started by: gimhan90
1 Replies

10. UNIX for Advanced & Expert Users

memory leaks

Hi!! Experts, Any ideas how to check for the memory leaks in a process during performance testing?? I dont use purify.. Any way of finding it out using default S/W in HP UX-11 Can U gimme pointers to site having good scripts/tutorials on performance testing?? Thanx in Advance.. :) (3 Replies)
Discussion started by: jyotipg
3 Replies
Login or Register to Ask a Question