Shelling Out to Give a System Command Drains Free Memory

Thread Tools Search this Thread
Top Forums Programming Shelling Out to Give a System Command Drains Free Memory
# 1  
Old 09-07-2011
Shelling Out to Give a System Command Drains Free Memory

I am working on a large program which is always up and must run in as many Linux flavors as possible, i.e. be portable. I have created a number of general utility functions for common tasks it needs to do, and one of these utility functions goes out to the shell to give a Linux command and return the output from the command. Depending on the exact Linux command being performed, occasionally it can drain free memory and not free it immediately.

In one case, I need to call this function to give a shell command and get data back a hundred or so times in a loop, which executes in just a moment. I find that during that loop my free memory decreases dramatically. I presume that it's freed at some point, but during execution of the loop, it can get quite low, and I think it may have caused a program abend in one environment. If you have read this far and are wondering, the command being given is to determine if a service is up, and it does this for every service which has script in /etc/init.d. The command, for bluetooth, for example, is:

service bluetooth status

My question is specifically this, is there anything I can do to my code in this function so that it either uses less memory of frees it better. Here is the code:

  int GeneralUtilities::ExecSystemCmd(string cmd, vector<string>& commandOutput)
  FILE *fp = NULL;
  char commandOutputLine[5000];
  int numLines = 0;
    cmd += " 2>&1";  // Get stderr.  This may not be sufficiently portable.
    /* Open the command for reading. */
    fp = popen(cmd.c_str(), "r");
    if (fp == NULL)
    /* Read the output a line at a time - output it. */
    for (numLines = 0; (fgets(commandOutputLine, sizeof(commandOutputLine)-1, fp) != NULL); numLines++)
    /* close */
    if (fp != NULL)
  return numLines;

I have checked and the character array commandOutputLine[] has no bearing on this problem. How might I re-write this function to make it less of a memory hog or to make it free its memory at once? Thanks in advance.


Last edited by pludi; 09-07-2011 at 08:20 PM..
# 2  
Old 09-07-2011
How, exactly, are you measuring "free" memory? It's a common beginner mistake to see "free" low and panic, but it doesn't usually mean you're out of memory -- "cached" is as good as free, so as long as "cached" is high, you've got plenty.

---------- Post updated at 04:55 PM ---------- Previous update was at 04:51 PM ----------

Another thing -- you're saving every single line to a structure in memory. When the output is enormous you'd be better off saving it to a temp file. If it's tiny, saving it to a temp file won't hurt performance because it'll all fit in cache.

Aside from that I don't see anything in this function which would be a memory hog. Depending on what command it's running though, that may end up converting lots of 'free' memory into 'cache' memory (again, harmless.)
# 3  
Old 09-08-2011
I was judging free memory from:

grep MemFree /proc/meminfo
# 4  
Old 09-08-2011
You are having the standard linux newbie memory freakout. We sometimes see a few of these a month. Take a deep breath and relax -- you're not running out of memory.

Memory that's otherwise sitting idle gets used for caches when there's anything to cache, because it's wasted just sitting around doing nothing. The kernel gives it up just as readily as any other unused memory -- you don't need to "free" anything or put your computer through weird convolutions to "release the cache". Everything is fine.

Just consider 'cache' and 'free' to get truer numbers on how much memory is available.
# 5  
Old 09-08-2011
I took a look with "free -m" and the situation looked entirely different. Also, I found the true cause of that one system crash and it had nothing to do with memory. My program was going through the scipts in /etc/init.d and doing a "status" on them to see what processes were up. It turns out that in openSUSE 11.4, that directory contains a script called boot which isn't a startup script at all, but the first script run at system startup, which does a lot of very basic things like mounting file systems. So, when my program gave the command:

service boot status

it was simply ignoring the command line argument "status" and executing, causing my program to stop and often causing a crash of my whole virtual environment. It had nothing to do with running out of resources.

Thanks again, Corona.
# 6  
Old 09-08-2011
Originally Posted by BrandonShw
I took a look with "free -m" and the situation looked entirely different.
Naturally it would, after the fact... But it reports the same information. You'll have to add free and cache.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

Is there a way to free up memory manually ?

Hi, I am wondering if there is a way to free up memory in Solaris manually ? the way we can do it in Linux for example : echo `/bin/date` "************* Memory Info Before *************" free -m sync echo 1 > /proc/sys/vm/drop_caches echo 2 > /proc/sys/vm/drop_caches echo 3 >... (13 Replies)
Discussion started by: terrykhatri531
13 Replies

2. Programming

Memory free() in C

Hi guys. I've a question, if we are using a syscall that receives a string allocated dynamicaly to a determined size, or NUL and it will allocate the apropriate size. We should free the memory or the OS will do it for us? If a function returns a pointer we should free that poiter when we are done... (7 Replies)
Discussion started by: pharaoh
7 Replies

3. UNIX for Advanced & Expert Users

Out of Memory error when free memory size is large

I was running a program and it stopped and showed "Out of Memory!". at that time, the RAM used by this process is around 4G and the free memory size of the machine is around 30G. Does anybody know what maybe the reason? this program is written with Perl. the OS of the machine is Solaris U8. And I... (1 Reply)
Discussion started by: lilili07
1 Replies

4. Solaris

Free memory in top and prstat command

Hi Export, i execute 'top' command to show the free memory in Solaris host, but the read is much lower than the RSS value shown in prstat command. Which one can reflect the real status and it is possible the difference caused by any patch of OS? Top command (only 883 memory is free)... (3 Replies)
Discussion started by: eiga
3 Replies

5. Solaris

how to get the more memory free space (see memory free column)

Hi all, Could please let me know how to get the more memory free space (not added the RAM) in local zone. -bash-3.00# vmstat 2 5 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr s0 s1 s1 s1 in sy cs us sy... (3 Replies)
Discussion started by: murthy76
3 Replies

6. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know... (4 Replies)
Discussion started by: 0ktalmagik
4 Replies

7. UNIX for Advanced & Expert Users

Can anyone give more details about the system calls?

Hello, Please can any one explain about the parameters to the write systemcalls?? How are they passed?? and how is the address of the user buffer is handled by the kenel?? for ex: write(fd,buf,count); How does the kernel handles this user buffer address?? After write does the kernel write... (1 Reply)
Discussion started by: prakash.kudreka
1 Replies

8. AIX

Free Memory

Hi, how to find free memory in aix? for installing oracle,I have used svmon but not getting proper output (1 Reply)
Discussion started by: manoj.solaris
1 Replies

9. Programming

How to free the memory?

For example if i have the piece of code as follows: CountryName = (char *)malloc((strlen(CountryName)+1)*sizeof(char)); memset(CountryName, 0, strlen(CountryName)+1); CountryName = SOME VALUE Now how do i free the memory after use of this code???? :confused: (3 Replies)
Discussion started by: jazz
3 Replies

10. News, Links, Events and Announcements

Sun to Give Out Operating System for Free

Reference: Sun to Give Out Operating System for Free Mon Nov 15, 7:31 AM ET By MATTHEW FORDAHL, AP Technology Writer SAN JOSE, Calif. - After investing roughly $500 million and... (1 Reply)
Discussion started by: Neo
1 Replies
Login or Register to Ask a Question