exec w/input file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers exec w/input file
# 1  
Old 01-02-2003
exec w/input file

Why is my code not doing what I want it to do? I have 3 files that I want to process
that are in format file123, file456, file789.
Code:
echo "Processing each file one at a time..."

cd $dir
ls -r1 file???.dat > input.dat

exec < input.dat
    IFS='|'
    while read INPUT_FILE ; do
          echo INPUT_FILE = $INPUT_FILE 1>/dev/null
          echo "Processing file $INPUT_FILE..."

          if [[ -s  $INPUT_FILE ]]; then
              ksh script1.ksh
          else
             echo "$INPUT_FILE was empty."
             exit 1
          fi

          echo "Renaming processed file $INPUT_FILE..."
          mv $INPUT_FILE $INPUT_FILE.done
    done

The problem is that as soon as the script1.ksh launches, this main script doesn't wait for all 10 scripts
to finish (script1.ksh ends then calls script2.ksh, which calls script3.ksh, etc..all the
way to script10.ksh) before it attempts to process the second input file. Why?? I was thinking that it would
only execute the second file once I reach the rename part (and that's after it had finished all 10 scripts)
but it doesn't. It kicks off script1.ksh and then it renames my files before, say script5.ksh even had
a chance to work with the input file.

Any ideas on how to make sure all 10 scripts execute for each iteration/each file? Right now, things are getting clobbered...and I want things done in sequence...

Gianni

added code tags for readability --oombera

Last edited by oombera; 02-19-2004 at 04:15 PM..
# 2  
Old 01-03-2003
look the wait man page

See the wait man page.

Note: You can also use the for command

cd $DIR

for INPUT_FILE in ./file???.dat
do

....
done

In order to debug the scripts use :
set -x

Regards. Hugo
# 3  
Old 01-03-2003
Computer Alternative to your script

I dont think you can use the 'wait' command unless you submit the script as a background process (see man wait). You can perform a similar operation using a "for" loop rather than the while loop. From what I gather, you want to create a list of file names, loop through them one at a time, execute script1.ksh for each iteration then rename. Exit program if any file is empty.

Note: In your script, if you were to run it more than one time then all of your files will be named file???.dat.done.done - you may want to consider a check to prevent this.

I didnt debug, so excuse me if there are syntax errors, but try this....
Assuming that DIR is set as some env var
Code:
#!/bin/ksh

cd $DIR
SUCCESS=0
FAILURE=1
MYFILES=`ls -r1 file???.dat` 
LOGFILE="| tee -a $LOG_PATH/loop_and_rename.log"
SCRIPT_NAME=script1.ksh

#Wait Function
wait_for_me () {
   WAIT_TIME=$1
   SCRIPT_NAME=$2
   TIMER_COUNT=0
while [ "${AM_I_RUNNING}" ] ; 
   do
       if (( $TIMER_COUNT >= $WAIT_TM ));
           then 
	echo "Took To Much Time: Tired of Waiting" 
	echo $FAILURE $LOGFILE
	exit $FAILURE	 
           else
	echo "Waiting!  $SCRIPT_NAME is currently running."
	let TIMER_COUNT="TIMER_COUNT+1"
	sleep 60
       fi
         AM_I_RUNNING=`ps -e | grep $SCRIPT_NAME`
 done
return
}


echo "Processing each file one at a time..."  $LOGFILE
for i in $MYFILES
do
echo "Processing file $i..."  $LOGFILE

wait_for_me 100 $SCRIPT_NAME

if [[ -s $i ]]
  then
       echo "Running Script For File [ $i ]" $LOGFILE
       ksh script1.ksh 
  else  
       echo "$i was empty."  $LOGFILE
       exit 1 
fi 

  echo "Renaming processed file $i..." $LOGFILE
  mv $i $i.done 
done

added code tags for readability --oombera

Last edited by oombera; 02-19-2004 at 04:16 PM..
# 4  
Old 01-04-2003
I see nothing in posted script that would explain the behavior that giannicello describes. So this leads me to suspect one of the 10 other scripts that were not posted.
# 5  
Old 01-06-2003
This script was a driver script that I was supposed to create because I was told that instead of ONE input file w/specific name, I would be given up to five files of format file????.dat where the ???? is a location.
Initially, I created just the 10 scripts to handle the ONE input file and it ran fine and was in production for a while. Then they decided it would be better to split the files before hand...yada yada yada. I didn't want to change script one to hard code the files of format file????.dat because it could be 5 today but 10 tomorrow...so I thought it would be faster to create one driver file which would do an ls of the file w/file???.dat, put the list into a file, input.dat, and then work each one, one at a time.
But whenever the driver file kicks off the first script, it acts like all's good, then it renames the current file and loops thru the while statement and clobber everything...weird....but I don't know why it's doing that.
The wait approach may not work because I have one script that can take between 20 - 40 minutes to run depending on how busy the server is running when it's executing an embedded query...

Thanks for everyone's input so far.

Gianni
# 6  
Old 01-06-2003
well, if all else fails, you can make your wait sufficiently large (its currently set to 100 minutes) enough to allow the longest script to complete.

Can you post your other scripts? or examples of them

Last edited by google; 01-06-2003 at 04:52 PM..
# 7  
Old 01-06-2003
Can you try with a lock file?

Code:
echo "Processing each file one at a time..." 

cd $dir 
ls -r1 file???.dat > input.dat 

exec < input.dat 
IFS='|' 
while read INPUT_FILE ; do 
   while [ ! -f /tmp/my_lockfile ]
   do
      echo "Waiting for the end of the previous INPUT_FILE"
   done

   touch /tmp/my_lockfile 
   echo INPUT_FILE = $INPUT_FILE 1>/dev/null 
   echo "Processing file $INPUT_FILE..." 

   if [[ -s $INPUT_FILE ]]; then 
       ksh script1.ksh 
   else 
       echo "$INPUT_FILE was empty." 
       exit 1 
   fi 

    echo "Renaming processed file $INPUT_FILE..."  
    mv $INPUT_FILE $INPUT_FILE.done 

done

Note: In the footer of script10.ksh add
rm -f /tmp/my_lockfile

added code tags for readability --oombera

Last edited by oombera; 02-19-2004 at 04:17 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

Removing punctuations from file input or standard input

Just started learning Unix and received my first assignment recently. We haven't learned many commands and honestly, I'm stumped. I'd like to receive assistance/guidance/hints. 1. The problem statement, all variables and given/known data: How do I write a shell script that takes in a file or... (4 Replies)
Discussion started by: fozilla
4 Replies

2. Shell Programming and Scripting

Read input files and merge them in given order and write them to input one param or one file

Dear Friends, I am looking for a shell script to merge input files into one file .. here is my idea: 1st paramter would be outfile file (all input files content) read all input files and merge them to input param 1 ex: if I pass 6 file names to the script then 1st file name as output file... (4 Replies)
Discussion started by: hyd1234
4 Replies

3. Shell Programming and Scripting

XML variable for input in same input file

Dear All , i stuck in one problem executing xml .. i have input xml as <COMMAND name="ARRANGEMENT.WRITE" timestamp="0" so="initial"> <SVLOBJECT> <LONG name="CSP_PMNT_ID" val="-1"/> <MONEY name="CSP_CEILING" amount="0.0" currency="AUD"/> ... (6 Replies)
Discussion started by: arvindng
6 Replies

4. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

5. Shell Programming and Scripting

Script Variables Inquiry, Values Okay in Standalone Exec, No-Show in Cron Exec

I have the following bash script lines in a file named test.sh. #!/bin/bash # # Write Date to cron.log # echo "Begin SSI Load $(date +%d%b%y_%T)" # # Get the latest rates file for processing. # d=$(ls -tr /rms/data/ssi | grep -v "processed" | tail -n 1) filename=$d export filename... (3 Replies)
Discussion started by: ginowms
3 Replies

6. UNIX for Dummies Questions & Answers

File redirect with find -exec

I'm trying to do an in-place encoding conversion in bash on a new Cygwin install, and this is the one-liner I've tried: $ find . -name '*.txt' -exec iconv -f UTF-8 -t CP932 {} > {} \;While the brace expansion works for multiple parameters (for example, -exec mv {} {}.bk), the redirect seems to... (1 Reply)
Discussion started by: R_Frankum
1 Replies

7. Shell Programming and Scripting

Need script to take input from file, match on it in file 2 and input data

All, I am trying to figure out a script to run in windows that will allow me to match on First column in file1 to 8th Column in File2 then Insert file1 column2 to file2 column4 then create a new file. File1: 12345 Sam 12346 Bob 12347 Bill File2:... (1 Reply)
Discussion started by: darkoth
1 Replies

8. HP-UX

exec and file descriptors

Hi, I speak and write english more or less, so I hope my asking be clear. :) In the company I am working, they are using control-m software to lunch shell scripts. So i put this command in all shell scripts: export LOGFILE_tmp=$PRODUC_DATA/tmp/${SCRIPT}_${PAIS}_`date... (0 Replies)
Discussion started by: anamcara
0 Replies

9. Shell Programming and Scripting

Reading specific contents from 1 input files and appending it to another input file

Hi guys, I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help. I have 2 input files (example given below) Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies

10. Shell Programming and Scripting

Help with exec command and file descriptors II

Need to close files which descriptor number are larger than 9 in ksh. 'exec 10>&-' fails with 'ksh: 10: not found'. How do you specify file descriptors which occupies two or more digits in ksh script? Thanks, Masaki (2 Replies)
Discussion started by: masaki
2 Replies
Login or Register to Ask a Question