Login or Register to Ask a Question and Join Our Community


AIX

Error while script execution - 0403-029 there is not enough memory available now

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems AIX Error while script execution - 0403-029 there is not enough memory available now
# 1  
Old 08-11-2015
Error while script execution - 0403-029 there is not enough memory available now
Hi,

I am executing a shell script on AIX box where I need to list all the files in the file system and then do some other tasks with them. I am able to do this successfully on HP-UX and Linux boxes but I get the following error after 10-15 seconds when I try to execute the script on an AIX box.

Code:
0403-029 there is not enough memory available now

I have to search a string in the entire file system.

My code

Code:
for dir in $(ls -d /*)
     do           
            for file in $(find $dir -type f -print)
            do
                    echo $file                            
                    #search the string in the file once this runs successfully
             done
                
      done

Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments.
Last edited by Don Cragun; 08-11-2015 at 03:08 AM.. Reason: Change bold tags to CODE tags and add ICODE tags.
# 2  
Old 08-11-2015
Your shell is running out of space trying to get a list of all files in the first directory you are processing with:
Code:
for file in $(find $dir -type f -print)

Do you really need to print the name of every regular file on your system? Or, can you just print the names of the files containing the text for which you are searching along with the lines that contain the matching text in those files?

Are you searching for fixed strings, for strings matching a basic regular expression, or for strings that match an extended regular expression?

PS: Are you intentionally excluding dot files found in the root directory on your system from your search?
Last edited by Don Cragun; 08-11-2015 at 03:22 AM.. Reason: Add postscript.
# 3  
Old 08-11-2015
If it is just to search all files for a string wouldn't it make more sense to do something like:

Code:
find / -type f -exec /some/search/command {} \;

and then the command /some/search/command could report something sensible based on result.

And since the scripts works on HPUX and Linux - either they have more memory, fewer files and/or they store temporary results differently than AIX.
# 4  
Old 08-11-2015
I'll describe in more detail. I have to search a string in the entire file system. I start by getting the list of directories at the root level, then I go to each directory and search in all the files. While doing the search I have to exclude few directories and files as well. code is given below.

Code:
root_dir="/"
mysearchstring="sysuser"

for dir in $(ls -d $root_dir*)
do
                for file in $(find $dir -type d \( -name 'sys' -o -name 'proc' -o -name '*.svn*' -o -name '*.ssh*' -o -name '*.subversion*' -o -name '*.snapshot*' -o -name '*crash*' -o -name '*dbatools*' -o -name 'opt' -o -name 'lic98*' -o -name '*informix*' -o -name '*developer*' -o -name '*tmp*' -o -name '*temp*' -o -name '*log*' -o -name '*spool*' -o -name '*perl_build*' -o -name '*.Z' -o -name '*is*data*' -o -name '*shared1*' -o -name '*pi_inbound*' -o -name '*backup*' -o -name '*archive*' \) -prune -o -type f \( ! -name '*.gz' ! -name '*.Z' ! -name '*image*' ! -name '*.out' ! -name '*.log*' ! -name '*.LOG*' ! -name '*.csv' ! -name '*.dat' ! -name '*.rcv' ! -name '*temp*' ! -name '*.gif' ! -name '*.png' ! -name '*.unl' ! -name '*tp_cleanup.*' ! -name '*.send' \) -print)
                                do
                                                                if [[ $(grep -l -i -e ${mysearchstring} "${file}") != "" ]]
                                                                then
                                                                                                echo $file
                                                                fi                              
                                 done
done

Here I am excluding few directories and files while doing the search. Also if I do not use print, I still get the not enough memory error.

---------- Post updated at 01:05 PM ---------- Previous update was at 12:54 PM ----------

I'll describe in more detail. I have to search a string in the entire file system. I start by getting the list of directories at the root level, then I go to each directory and search in all the files. While doing the search I have to exclude few directories and files as well. code is given below.

Code:
root_dir="/"
mysearchstring="sysuser"

for dir in $(ls -d $root_dir*)
do
                for file in $(find $dir -type d \( -name 'sys' -o -name 'proc' -o -name '*.svn*' -o -name '*.ssh*' -o -name '*.subversion*' -o -name '*.snapshot*' -o -name '*crash*' -o -name '*dbatools*' -o -name 'opt' -o -name 'lic98*' -o -name '*informix*' -o -name '*developer*' -o -name '*tmp*' -o -name '*temp*' -o -name '*log*' -o -name '*spool*' -o -name '*perl_build*' -o -name '*.Z' -o -name '*is*data*' -o -name '*shared1*' -o -name '*pi_inbound*' -o -name '*backup*' -o -name '*archive*' \) -prune -o -type f \( ! -name '*.gz' ! -name '*.Z' ! -name '*image*' ! -name '*.out' ! -name '*.log*' ! -name '*.LOG*' ! -name '*.csv' ! -name '*.dat' ! -name '*.rcv' ! -name '*temp*' ! -name '*.gif' ! -name '*.png' ! -name '*.unl' ! -name '*tp_cleanup.*' ! -name '*.send' \) -print)
                                do
                                                                if [[ $(grep -l -i -e ${mysearchstring} "${file}") != "" ]]
                                                                then
                                                                                                echo $file
                                                                fi                              
                                 done
done

Here I am excluding few directories and files while doing the search. Also if I do not use print, I still get the not enough memory error.
# 5  
Old 08-11-2015
The following should do exactly the same thing without running into memory constraints as long as you don't start it in a directory where expanding $root_dir* overflows ARG_MAX limits:
Code:
root_dir="/"    # Note that if a directory other than / is used here, it must include a terminating /
mysearchstring="sysuser"
find "$root_dir"* -type d \( -name 'sys' -o -name 'proc' -o -name '*.svn*' -o -name '*.ssh*' \
    -o -name '*.subversion*' -o -name '*.snapshot*' -o -name '*crash*' \
    -o -name '*dbatools*' -o -name 'opt' -o -name 'lic98*' -o -name '*informix*' \
    -o -name '*developer*' -o -name '*tmp*' -o -name '*temp*' -o -name '*log*' \
    -o -name '*spool*' -o -name '*perl_build*' -o -name '*.Z' -o -name '*is*data*' \
    -o -name '*shared1*' -o -name '*pi_inbound*' -o -name '*backup*' \
    -o -name '*archive*' \) -prune -o \
    -type f \( ! -name '*.gz' ! -name '*.Z' ! -name '*image*' ! -name '*.out' \
    ! -name '*.log*' ! -name '*.LOG*' ! -name '*.csv' ! -name '*.dat' ! -name '*.rcv' \
    ! -name '*temp*' ! -name '*.gif' ! -name '*.png' ! -name '*.unl' ! -name '*tp_cleanup.*' \
    ! -name '*.send' \) -exec grep -lie -- "$mysearchstring" {} +

PS With what you're doing here, I don't see any reason why changing find "$root_dir"* -type d ... in the above to just find "$root_dir" -type d ... would produce different results (and it avoid the possibility of an ARG_MAX limit error).
Last edited by Don Cragun; 08-11-2015 at 06:11 AM.. Reason: Add postscript.
This User Gave Thanks to Don Cragun For This Post:
Adyan Faruqi
# 6  
Old 08-11-2015
Well, for whatever reason, if it is not working on AIX - asis - change it.

The problem is, assuming, finding the file or files with a certain string, not that one command can be used everywhere, regardless.

Break it into components - e.g., get a list of all the files you want to scan, e.g., skip the -exec part and as a second command process the files where - if the string is found - output the filename and/or repeat the search with output to stdout.

My advice is not to spend too much time on find per se (and we have assumed it is AIX find, not a find from a RPM package of coreutils (or even my installp packaging).

If you are clever enough to write a single find command to do this task, I am sure it is child's play to break it into components.

---------- Post updated at 01:19 PM ---------- Previous update was at 01:13 PM ----------

One more thought - try prefixing (or use export) command with
Code:
LDR_CNTRL=MAXDATA=0x8000000 find ...

or
Code:
export LDR_CNTRL=MAXDATA=0x8000000
find ...

The value 0x80000000 assigns 2G of data to the application, rather than the default 256MByte. This value can be enlarged, but hopefully - 2G will be enough.
# 7  
Old 08-11-2015
Thanks Don/Michael .. using Exec option with find command worked well and now I don't get the out of memory error. The script executed well.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

0403-057 Syntax error

I am getting the error : rocfm/wls_subload/in/processed_files/tel_input_additional_checked_all_mandatory.txt: 0403-057 Syntax error at line 1 : `|' is not expected. >>>>ALL MANDATORY FIELDS CHECKING IS SUCCESSFUL count is 0 ... (3 Replies)
Discussion started by: princetd001
3 Replies

2. Shell Programming and Scripting

Syntax error in sh script execution

Script: #!/sbin/sh echo "Welcome to my First Script" echo "Enter a word" read PASS if then echo "You are correct" elif then echo "Thats incorrect" else echo "Bye" fi When i run the script shell says: Syntax error at line 7:'elif' is not expected I ran through some old posts and... (3 Replies)
Discussion started by: Amit Kulkarni
3 Replies

3. Shell Programming and Scripting

Execution error with awk script

Hi, I run an awk script and I got the error attached below: here are the lines that the compiler point to as an error: duration = timeEnd1-timeBegin1; print "Transmission: type of traffic " flow1 ; print “ - Total transmitted bits = ” totalBits1 ” bits”; print “ - duration = ”... (2 Replies)
Discussion started by: ENG_MOHD
2 Replies

4. AIX

backup script failed with error '0403-005 Cannot create the specified file'

In AIX 5.1, a daily run script that backing up oracle data failed yesterday with following errors: The Tivoli backup of DBPROD failed. What could be the issue, OS, backup or Oracle? (3 Replies)
Discussion started by: jalite19
3 Replies

5. AIX

AIX KSH: 0403-029 There is not enough memory available now

Hi guys, I hope you can help me out with this one. I am getting an error in AIX when running my KSH script 0403-029 There is not enough memory available now. It is getting this error at the point where I have a PL/SQL Script executed. After executing, I wanted to put it in the log file.... (4 Replies)
Discussion started by: 3vilwyatt
4 Replies

6. Shell Programming and Scripting

Receiving error: ./ang.ksh[35]: 0403-057 Syntax error at line 116 : `done' is not expected.

Hi All I am quite new to Unix. Following is a shell script that i have written and getting the subject mentioned error. #!/bin/ksh #------------------------------------------------------------------------- # File: ang_stdnld.ksh # # Desc: UNIX shell script to extract Store information.... (3 Replies)
Discussion started by: amitsinha
3 Replies

7. Shell Programming and Scripting

Problem with shell script - Error: 0403-057

Hi, I am new to shell scripts may be the error is very very small and i am unable to catch hold of it, any suggestion would be appreciated....error is at the bottom: +210 # get file type +211 filetype=`tail -1 "$inputdir"/"$i"|cut -d"|" -f2` +212 # +213 # get the record count as specified... (4 Replies)
Discussion started by: ravi0435
4 Replies

8. AIX

allocate memory for shell script at runtime during execution--urgent critical help!!

How to allocate memory for a shell script on aix box at the time of execution i.e at runtime Are there any commands for AIX in specific Thanks in Advance (1 Reply)
Discussion started by: aixjadoo
1 Replies

9. Shell Programming and Scripting

Script Error: 13192.sumr: 0403-016 Cannot find or open the file.

Hello, i am familiar enough with unix to do some damage but thats about it. We have a set of RS/6000 43P Model 150's running AIX for our Catia V4 programmers. back in the 90's a script was written to automate the conversion of files into machine code. that script has started giving us... (0 Replies)
Discussion started by: jgruenwald
0 Replies

10. Shell Programming and Scripting

error during the execution of script

Hi, I have a cron job which executes daily once 9 PM. The script is like if then TYPE=OC elif then TYPE=i elif then TYPE=mmc elif then TYPE=CB elif then TYPE=oth fi (1 Reply)
Discussion started by: surjyap
1 Replies
Login or Register to Ask a Question