Script to go Into Directories and Find/Delete files

04-03-2013

Registered User

34, 1

Join Date: Mar 2012

Last Activity: 18 September 2015, 9:47 AM EDT

Location: Baltimore, MD

Posts: 34

Thanks Given: 18

Thanked 1 Time in 1 Post

Script to go Into Directories and Find/Delete files

I have a task, I usually do manually, but with growing responsibilities I tend to forget to do this weekly, I want to write a script that automates this, but I cant seem to work it out in my head, I have the shell of it out, but need help, and you guys have helped me with EVERY problem I have posted, and have found help in others posts as well. Please see below shell code I have started. I want to go to the ORDATA which is the variable for /opt/data in this case directory, find the folders with most stuff in it and cd into each one and then cd into the backup folder and do a couple of those find commands.

How do I make it go out of the first directory and into the next and so on and so forth? I know i can use the while loop in some capacity, but not quite sure.

for example /opt/data/ has 4 directories /mpw, /docprep, /edfiles, /vgdata and under each of those directories is a /backup directory as well, I want to start at /opt/data, go into each the 4 directories, go into the backup directories and do those find and delete commands i came with until its went into all 4 backup directories.

Below is what I have so far.

Code:

#!/bin/ksh
#Author:Emmanuel Iroanya Jr
#Date:April 2nd, 2013
#Purpose: The purpose of this is to check data and rollout directory and keep file space optimal. 

if [ "$1x" = "x" ]
	then
	echo "You must supply an environment parameter to this script"
	echo "example:  $0 tb82"
	exit
fi

. /opt/origenate/$1/config/usrconfig.sh 	#Sources Environments Admin Config Variables

cd $ORDATA 								#Environments Data Storage Directory
du -sk */ | sort -n -r | head -n 10 |awk '{print $2}' |while read d
do 
	cd "${d}" 							
	cd backup
	find *.zip -type f -a -mtime +3 -exec rm {} \;
	find *.pdf -type f -a -mtime +3 -exec rm {} \;
	find *.CSV -type f -a -mtime +3 -exec rm {} \;
	find *.txt -type f -a -mtime +3 -exec rm {} \;
done

Last edited by gkelly1117; 04-03-2013 at 12:08 PM.. Reason: Bad title

gkelly1117

View Public Profile for gkelly1117

Find all posts by gkelly1117

04-03-2013

Registered User

4,996, 477

Join Date: Dec 2003

Last Activity: 12 June 2016, 11:03 PM EDT

Location: /dev/ph

Posts: 4,996

Thanks Given: 73

Thanked 477 Times in 439 Posts

Hello,

Per our forum rules, all threads must have a descriptive subject text. For example, do not post questions with subjects like "Help Me!", "Urgent!!" or "Doubt". Post subjects like "Execution Problems with Cron" or "Help with Backup Shell Script".

The reason for this is that nearly 95% of all visitors to this site come here because they are referred by a search engine. In order for future searches on your post (with answers) to work well, the subject field must be something useful and related to the problem!

In addition, current forum users who are kind enough to answer questions should be able to understand the essence of your query at first glance.

So, as a benefit and courtesy to current and future knowledge seekers, please be careful with your subject text. You might receive a forum infraction if you don't pay attention to this.

Thank you.

The UNIX and Linux Forums

This User Gave Thanks to fpmurphy For This Post:

fpmurphy

View Public Profile for fpmurphy

Find all posts by fpmurphy

04-03-2013

Registered User

34, 1

Join Date: Mar 2012

Last Activity: 18 September 2015, 9:47 AM EDT

Location: Baltimore, MD

Posts: 34

Thanks Given: 18

Thanked 1 Time in 1 Post

My apologies, that makes sense, I have changed the title of the post to something more appropriate.

Also, I put comments in the code/query so people could understand my thought process, does it not come across as a clear thought?

gkelly1117

View Public Profile for gkelly1117

Find all posts by gkelly1117

04-03-2013

Registered User

23,310, 4,623

Join Date: Aug 2005

Last Activity: 7 July 2020, 11:47 AM EDT

Location: Saskatchewan

Posts: 23,310

Thanks Given: 1,331

Thanked 4,623 Times in 4,217 Posts

A few changes to suggest:

Code:

#!/bin/ksh
#Author:Emmanuel Iroanya Jr
#Date:April 2nd, 2013
#Purpose: The purpose of this is to check data and rollout directory and keep file space optimal. 

if [ -z "$1" ] # -z is simply "if string is blank".
then
	echo "You must supply an environment parameter to this script"
	echo "example:  $0 tb82"
	exit
fi

. /opt/origenate/$1/config/usrconfig.sh 	#Sources Environments Admin Config Variables

cd $ORDATA 								#Environments Data Storage Directory
du -sk */ | sort -n -r | head -n 10 |awk '{print $2}' |while read d
do
        # find's first parameter is a path, not a wildcard
        # You give it wildcards with -name 'string'
        # You can group them with ( ) and or them with -o
        # Remove the echo once you've tested and are sure it does what you want.
	find ${d}/backup '(' -name '*.zip' -o -name '*.CSV' -o -name '*.txt' ')' \
                -type f -a -mtime +3 -exec echo rm {} \;
done

This User Gave Thanks to Corona688 For This Post:

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

04-03-2013

Registered User

34, 1

Join Date: Mar 2012

Last Activity: 18 September 2015, 9:47 AM EDT

Location: Baltimore, MD

Posts: 34

Thanks Given: 18

Thanked 1 Time in 1 Post

Quote:

Originally Posted by Corona688

A few changes to suggest:

Code:

#!/bin/ksh
#Author:Emmanuel Iroanya Jr
#Date:April 2nd, 2013
#Purpose: The purpose of this is to check data and rollout directory and keep file space optimal. 

if [ -z "$1" ] # -z is simply "if string is blank".
then
	echo "You must supply an environment parameter to this script"
	echo "example:  $0 tb82"
	exit
fi

. /opt/origenate/$1/config/usrconfig.sh 	#Sources Environments Admin Config Variables

cd $ORDATA 								#Environments Data Storage Directory
du -sk */ | sort -n -r | head -n 10 |awk '{print $2}' |while read d
do
        # find's first parameter is a path, not a wildcard
        # You give it wildcards with -name 'string'
        # You can group them with ( ) and or them with -o
        # Remove the echo once you've tested and are sure it does what you want.
	find ${d}/backup '(' -name '*.zip' -o -name '*.CSV' -o -name '*.txt' ')' \
                -type f -a -mtime +3 -exec echo rm {} \;
done

Appreciate it alot, let me play with this a little bit, question, Can I feed this multiple directories, I remembered I also have a tmp directory and files under ordata in a different structure tree other than /backup that fills quickly, so can i add

Code:

find ${d}/tmp '('

etc.. after that first find? It makes sense to me that it could, just want confirmation.

gkelly1117

View Public Profile for gkelly1117

Find all posts by gkelly1117

04-04-2013

Registered User

5,091, 1,931

Join Date: May 2012

Last Activity: 15 July 2020, 4:46 AM EDT

Location: Simplicity

Posts: 5,091

Thanks Given: 565

Thanked 1,931 Times in 1,668 Posts

Yes, we can.

If your find statements are exactly the same otherwise, you can combine them to one

Code:

find ${d}/backup ${d}/tmp -type f -mtime +3 '(' -name '*.zip' -o -name '*.CSV' -o -name '*.txt' ')' -exec echo rm -f {} \;

I changed the order, so the more "expensive" name comparison is after the simple file type and age checks. -a (AND) is only for readablity - can be omitted.
Last but not least, this should be secured:

Code:

cd $ORDATA || exit #fatal - directory must exist!

This User Gave Thanks to MadeInGermany For This Post:

MadeInGermany

View Public Profile for MadeInGermany

Find all posts by MadeInGermany

Shell Programming and Scripting