Shell Programming and Scripting

I have a script that if it crashes I want it to start again where it stopped.

The way to identify the tasklist is using a find command that list all the files that need the process performed on them

Code:

TASKLIST=`find /path/to/directory -iname *.dummy | sort`

The way to identify the ones that have already been processed is

Code:

FINLIST=`find /path/to/directory -iname *.txt | sort`

So I'm then trying to delete entries in TASKLIST up to the last FINLIST entry.

Code:

LASTDONE=`echo "$FINLIST" | tail -n1 | sed 's/\.txt//'
NEWJOBS=`echo "$TASKLIST" | sed "1,/${LASTDONE}/d"

I've tried syntax ${LASTDONE} and $LASTDONE as well as sed "1,|${LASTDONE}|d" to try a different delimeter.

I keep getting unknown command errors.

Could someone please point out the probably glaringly obvious mistake I'm making so this Frenchman stops tapdancing inside my head.

Thanks

I like implicit restartable processes -- no code to mess up when first it is stressed!

Move the input files to another dir when they are done, or rename them (mv)? This is cheap and easy. Putting status and time stamps on the file name is a relatively reliable way to signal. Even if ftp'ing files, ftp to one name and if it is OK, do a rename to the final name, which tells the other end that, by its very existence, it is ok to use. Using dirs means there is less conflict over the locks on the directory inode, and everything in there says work needs to be done! As long as a mv does not change volumes/mounts/devices, it is cheap, just two writes to dir inodes, at least on most UNIX file systems and WinDos.

---------- Post updated at 04:41 PM ---------- Previous update was at 04:22 PM ----------

On a more cooperative vein, find all the files of each type without conflicting path, use sed to remove the extension differences, sort them, and use "comm -13 in_file_list_file out_file_list_file" to tell you what is undone. With Solaris you can pipe it all:

Code:

#!/usr/bin/ksh

cd path
comm -13 <(
  ls *.dummy | sed 's/.dummy//' | sort
 ) <(
  ls *.txt | sed 's/.txt//' | sort
 )

Thanks DGP but I'm looking for what's wrong with the syntax to the sed command (I'm pretty certain the logic is OK) else I'll end up having to debug other code.

If someone could point out what I seem to be missing I learn from my mistake for next time.

Regards

Slashes in the env var? substitute with pipes? Missing '`' (always use $() myself):

Code:

LASTDONE=$(
 echo "$FINLIST" | tail -n1 | sed '
   s/\.txt//
   s/\//\\//g
  '
)

Agree on using $(), definitely easier to spot command substituions that way and if you're missing the closing end.

But replace this:

Code:

NEWJOBS=`echo "$TASKLIST" | sed "1,/${LASTDONE}/d"`

with this:

Code:

NEWJOBS=`echo "$TASKLIST" | sed "1,\%${LASTDONE}%d"`

When the literal '/' is a part of your address pattern, you can change the delimeters by using any other character as long as the first one is preceded with '\'

Thanks RWE

I'd forgotten that you need the backslash in front of the substitution, mainly because I usually do just substitution where the syntax structure is different.

Problem fixed.

$() nests, too!