UNIX for Dummies Questions & Answers

Hi,

I have a script which has a for loop in it. In that for loop are names of files that are going to be processed. For example,

for file in file1 file2 file3 file4
do
(something)
done

Let's say that this script gets executed and it fails at 'file2'. Is there a way that I can actually tell this script where to restart from (file3), to avoid going into this script and modifying it? Any suggestions would be appreciated.

As always,thanks!

create step file before processing the file. ie.. the file
run.step should contain the file which it processed.

when you re-run check the last run file and continue from there.

Thanks for replying.

Can you please provide an example so that I understand what you mean?

Thanks in advance!

I think abcd122 just meant to keep track of which files you have finished, so you have a way of knowing which files still need processing. That's a start, but doesn't really solve the problem, as such.

I usually run GNU make for stuff that might take long and might bomb out; it will delete the output form a botched run (if you specify .DELETE_ON_ERROR) and can keep track of what's been done and what still needs to be done. (Still missing from the picture is some sort of concurrency control, to be able to see that a particular pending result is already running on another host.) It takes a bit of getting used to but it has paid itself back handsomely a number of times.

That's a more radical refactoring than you were thinking of, I'm sure, but I'm offering it nevertheless; if interruptions are a scenario you need to take seriously, there's no way you can code a for loop which simply "knows" when it's done.

But of course, if the presence of a file is enough, then the really simple thing will work:

Code:

for f in file1 file2 file3 file4; do
  if [ -e $f.out ]
  then
    echo $f.out already exists -- not rerunning
  else if  long and winding and painstaking command to calculate ruptures in space time fabric <$f>$f.tmp
   then
    # only commit when it really finished, notice this is also conditional on exit code
    mv $f.tmp $f.out
  else
    echo "oh dear, $f failed (exit code $?), leaving output in $f.tmp" >&2
  fi
done

Thank you for your post. I can tell you are really experienced with this stuff. Going back and reading my initial post I realized that I did not do a good job at explaining what my goal is. At least, I don't think so. So, let me explain myself in a better fashion.

I have three .ctl files:

A.ctl
B.ctl
C.ctl

In all of these .ctl files, I have, among other things, something called "BATCH". It looks like this: BATCH = 123-133. For each of these files I am grepping the "BATCH" value. Which is where my for loop comes to play.

grep 'BATCH" /somewhere/in/a/dir/${filename}.ctl | read a1 a2 BATCH

for filename in A B C
do
(executing another script which will use the "BATCH" to load some data)
done

This script will be running via kron scheduler. So let's say that for the A.ctl, everything went smoothly, but when it came to the B.ctl we got an error. We would like to re-run B.ctl, without going into the script and deleting A.ctl from the script and letting the for loop continue with B and C.

Is there a way where I can, from the command line pass the $filename parameter to this script so it can start the for loop from where I want it to?

THANKS!

Frankly, I think you have all the pieces you need, you just need to do it.

It is easier said than done, my friend.

I just need to gather my thoughts. I definately have great examples to follow.