Shell Programming and Scripting

I'm sure this has been asked many times, but a search didn't turn up a definitive best method for this (if there ever is such a thing).

I have been using rsync to back up my main data directory, but I have accumulated a large number of older backups that I don't need. All of the files I don't need anymore have the extension .back, so I need to troll through all of the folders and sub-folders and delete everything with the .back extension. I thought I would need to do some kind of recursive ls and pipe the results to rm, but I'm not sure what that would look like so I did a search.

Many of the solutions I found use find and look like,

Code:

find . -name *.back -exec file {} \; -exec rm -i {} \;

or

Code:

find /path . -name '*.back' -type f -delete

or

Code:

find /path -iname "*.back" -type f -delete

As usual, there appear to be many ways of doing things and I have no basis on which to make a choice. These files are copies, so I could always rebuild the backup if there was a disaster with the cleaning, but that would take time and I try to avoid putting my foot in it to that extent.

Any suggestions?

LMHmedchem

Go to the main directory and run

Code:

find . -name "*.back" -type f -exec rm -f {} \;

Alright, I will give that a go. If you have a minute to answer, what is the difference between the method you posted and the other examples I gave in my original post?

LMHmedchem

IMO, the first command will not work because the o/p of 'find' command is not piped in any way to the exec command. But the second and third should work for sure.
I wasn't sure about the -delete' action. That is why I gave you the command I am familiar and experienced with.

Any better explanations from geeks are welcome.

Oh dear, I'm classing myself as a geek. Well, if the name fits, ......

The way you have tried, the shell will expand *.back before trying to run the command. If you happend to have a file at the top level called this.back then the command actually run will become:-

Code:

find . -name this.back -exec file {} \; -exec rm -i {} \;

so you will not actually match anything other than the file at the top level.

The others are various errors. What jaiseaugustine has suggested is the correct format for you. It will pass in *.back as it is to the find command and then it can be used for pattern matching.

If there are no files at the top level, you might get away with it depending how your shell reacts, but if there is more that one file called *.back, e.g. this.back & that.back, then you will probably get the error:-

Code:

find: There is a missing conjunction

because the shell will try to run:-

Code:

find . -name that.back this.back -exec rm {} ;

I hope that this clarifies things a bit.



Robin
Liverpool/Blackburn
UK

Further, the -exec file {} \; -exec rm -i {} \; is tailored for interactive use,
while -type f -exec rm -f {} \; is tailored for scripts.

BTW -delete is a new option in new find versions. Would be more efficient if thousands of files are deleted.

This is more or less always what I end up doing and I guess it is a reasonable way to proceed in most cases. I keep notes on what I have used for various situations, especially those methods that worked well.

The command,

find . -name "*.back" -type f -exec rm -f {} \;

worked well and cleared out about 50GB or older incremental versions. I ran this while I was out for a while and I didn't run it under time, so I can't comment on how fast the method is compared to other possibilities. I generally presume that there is no fast script based method to process a directory tree with 4+ million files.

I also did a defrag/optimize (auslogics) and clean out of MFT records. All told it took almost 24 hours to run, but I find I need to keep these backups well maintained, or they eventually bork and you have to reformat and start again. It seems as if rsync tends to lead to very fragmented repositories. I have never quite understood why you get lots of fragmenting on a drive with 500GB of empty space.

Thanks for all the additional explanations. I do always try to understand what a script is doing and why you would choose one method over another. I think I need to read a bit about exec.

LMHmedchem