Shell Programming and Scripting

Hi,

I have here a script which is used to purge older files/directories based on defined purge period. The script consists of 45 find commands, where each command will need to traverse through more than a million directories. Therefore a single find command executes around 22-25 mins approximately . The entire script runs approximately 18.30hrs which is very very huge. Could someone suggest a better idea/logic to achieve this purpose and also reducing the execution time as much as possible. Thanks!

Sunday morning guesswork (with by far not enough files available for serious testing, based on experience with a similar but smaller problem though):

- First, I'd separate the finding from the killing, i.e. generate a shitlist (of data to be removed) to be processed as a background task of lesser priority.

- Second, I'd rewrite thus reduce the number of searches by means of (more) regular expressions, maybe even using 'ls' and 'grep' instead of 'find' (?).

Thank you for responding dr.house, and yes I did think about the option of separating the search logic and redirect them into a dump file. Later pick the entries from the dump file and proceed to delete/purge. But this one too had a negative impact on the size of dump file created(very huge), and the filesystem reached 100% before the search was complete.

Honestly speaking, I am very weak in shell scripting especially involving complex logic. It should be grateful if you could post the script with the logic you suggested. Thanks!

Put together all those lines where rule is same, use find -o option to put together name rules using one find.

There is also meny different kind of finds, some days old or one year old. Think carefully how often you really need to run different kind of finds+rm. Make more than find script file, run once in month, run once in ...

Example something like this:

Code:

find . \( -name '*(MON)*'  -o -name '*(TUE)*' \) -type f

Thanks kshji, seems to be quite a good suggestion. If I group together all the search patterns having common purge period, I am left with only 13 find commands. If the find command with multiple search patterns also run for approximately 22-25 mins, your logic would drastically reduce the execution time. Let me try it and post a reply.

One quick question, may I precede using "-type f" before "-name"

The answer is yes

Thank you