Shell Programming and Scripting

Need some fine tuning advice:

I have a KSH programs croned to execute once a day. Which basically does the following simple operation:

for i in `ls`

do
cp $i $i.backup
>$i
gzip $i.backup

done

Basically cleanup the file after taking a backup.

Now the issue here is this directory will contain more than 12 files with >1GB each file and gets very nasty all time.

I considered an approach like

1. tar -cvf debug.tar ~/debug
2. > [All the files]

This approach also creates problem because there will be a big time gap between step 1 and 2. so all the info written between 1 and 2 will be lost.

Any advice on this?

maybe do a mv rather than cp, then gzip, and then "touch $i" to create a new file?

Hey, use the "FIND" command along with tar command to zip the files greater than the specified size.
Go to the specified directory and execute the following command.....

Here is an eg :
find . -type f -size +1000 2>/dev/null | xargs tar -cvf testingtar.tar


What this basically does it
1> It finds all the files that are greater than 1000 blocks.
2> We collect all those through xargs (which is the faster than others)
3> We tar all those files.

2>/dev/null is used so that the error messages are not displayed.....


It works quickly and i believe this is the better way to use it.


Let me know if you need any.......

Age 79/Helper, Thanks for response. Both the ideas suggested gone in my mind already, especially the Age79 approach.

However, what I'm concerned is that If I do

mv $i
touch $i

This may have an impact on the C++ program as the file handler changes internally? Am I talking sense?

Helper,
find . -type f -size +1000 2>/dev/null | xargs tar -cvf testingtar.tar

Is good but still it's not recovering the demerit I've mentioned. As I have to cleanup the files also using ">" probably, I'll loose some data when I cleanup after the above operation.


Anyway, goog points. Appreciated.

Ideally the application should have a way to make it close and reopen its log file precisely for this reason.

100% agreeing to it. This is like a legacy application running in production and don't have a way to alter it. Otherwise I would prefer a log rotation kind of mechanism which creates new log file everyday! :-) We can archive the rest of stuffs safely then.

Anyway.. This is what it is. :-)

Cheers
F