Exclude certain folders


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Exclude certain folders
# 1  
Old 08-05-2014
Exclude certain folders

Hi Experts,
Below is my shell script and it will move the files older than 90 days to archive mount. Now my new requirement is , I need to move some of the directory files older than 365 days. How can I achieve this.

Simply I have
DIR1
DIR2
DIR3
DIR4
I need to exclude DIR 2 and DIR 2 directory older than 365 days files should be moved to another directory. Remaining directory will move the files older than 90 days as usual
Code:
#!/bin/ksh
#
#
# Backup Script to move older than 90 days files
DATE=`date +%Y%m%d`
#
#SRC Directories and Location
#-----------------------------------------------------
SRC_CDR_OUTBOUND=/omount/outbound
SRC_CDR_INBOUND=/omount/inbound
DEST_SRC_BILG=/omount/archive/
#-----------------------------------------------------
#
DATE=`date +%Y%m%d`
#Counting Number of files
#--------------------------------------------------------
BK_FILECOUNT_OUT=`find $SRC_CDR_OUTBOUND -type f -mtime +90 |wc -l`
BK_FILECOUNT_IN=`find $SRC_CDR_INBOUND -type f -mtime +90 |wc -l`
#--------------------------------------------------------
#
#Finding OUTBOUND files older than "90" days and Move the files to archive folder
#-----------------------------------------------------------------------------
find $SRC_CDR_OUTBOUND -type d -name '*' -exec mkdir -p $DEST_SRC_BILG{} \;
find $SRC_CDR_OUTBOUND -type f -name '*' -mtime +90 -exec mv {} $DEST_SRC_BILG{} \;
#-----------------------------------------------------------------------------
#
#Finding INBOUND files older than "90" days and Move the files to archive folder
#-----------------------------------------------------------------------------
find $SRC_CDR_INBOUND -type d -name '*' -exec mkdir -p $DEST_SRC_BILG{} \;
find $SRC_CDR_INBOUND -type f -name '*' -mtime +90 -exec mv {} $DEST_SRC_BILG{} \;
#-----------------------------------------------------------------------------
#Generating the log with date and file count for refernece
#-------------------------------------------------------------------------------------------
echo $BK_FILECOUNT_OUT "Outbound files moved to archive on"$DATE >>/omount/backup_logs.txt
echo $BK_FILECOUNT_IN "Inbound files moved to archive on"$DATE >>/omount/backup_logs.txt
#-------------------------------------------------------------------------------------------

Moderator's Comments:
Mod Comment Please use code tags next time for your code and data. Thanks

Last edited by vbe; 08-05-2014 at 12:43 PM..
# 2  
Old 08-05-2014
Welcome arumugavelvelu ,

Firstly, you don't need -name '*' which says "only objects called anything" so leaving it out will ignore name matching and find objects that match the other criteria giving you the same result.

If you have a fixed set of input directories, then you could set your variables at the top to refer to them, e.g.:-
Code:
SRC_CDR_OUTBOUND="/omount/outbound/DIR1 /omount/outbound/DIR3 /omount/outbound/DIR4"
SRC_CDR_INBOUND="/omount/inbound/DIR1 /omount/inbound/DIR3 /omount/inbound/DIR4"

These will then get used by your find command as the directories to search.

If the directories may change over time and you don't want to recode them, you may need to do something a bit more elaborate like:
Code:
SRC_CDR_OUTBOUND="`ls -1d /omount/outbound/* | grep -v DIR2"
SRC_CDR_INBOUND="`ls -1d /omount/inbound/* | grep -v DIR2"


This will get you all the objects at that level, so if you want to exclude files, then you have to be more elaborate still:-
Code:
SRC_CDR_OUTBOUND="`find /omount/outbound -type d ! -name DIR2 | cut -f-4 -d\"\/\" |sort -u`"
SRC_CDR_INBOUND="`find /omount/inbound -type d ! -name DIR2 | cut -f-4 -d\"\/\" |sort -u`"


Do these achieve what you need, or have I missed the point somewhere?



Robin

Last edited by rbatte1; 08-05-2014 at 02:03 PM.. Reason: Line spacing to make it easier to read
# 3  
Old 08-05-2014
Hi Robin
Thank so much for your explanation. That works for me ..
But what i am planning is i am going to put one text file named as 365.txt to the directory which i want to exclude.so my shell script will find whether 365.txt file is there or not in each directory. If it exist then it will move the file older than 365 days..... if not it will move the files older than 90 days.

So in future if new request comes to me to exclude this directory I will put 365.txt file to that directory then it will move older than 365 days files
I know its possible but i tried many ways .. Unfortunately could not achieve it. Could you please put some light on this.

Thanks in advance....

Last edited by rbatte1; 08-06-2014 at 06:02 AM.. Reason: Spelling
# 4  
Old 08-06-2014
That makes it a little more tricky, but as they say, "Anything is possible."

You can test if a file exists with this:-
Code:
if [ -f filename ]
then
   echo "Found it!"
else
   echo "File is not there."
fi

Does that help? You could set a variable to use in the find as the value assigned to -mtime. I'm still not clear on the process you would want it to take and how you would know where to find the file to make your decision. Can you write out a set of logical steps so we can plan it. If you put the phrases one on a line, then highlight them and press a list button (same icon choice as MS word) it will format it a bit better. To show a loop, just highlight the section and again press the list button of your choice.

This:-
Quote:
[LIST]
[*]List line1
[*]List line2
[*]List line3
[LIST]
[*]2nd line1
[*]2nd line2
[/LIST]
[*]List line4
[/LIST]
.... generates this:-
  • List line1
  • List line2
  • List line3
    • 2nd line1
    • 2nd line2
  • List line4
If you can articulate what you need as a set of steps, then I'm sure someone here can help.



Regards,
Robin
# 5  
Old 08-06-2014
Hi Robin,

Thank you for your reply.

Can you please tell me what the following command will do Smilie
Code:
cut -f-4 -d\"\/\" |sort -u

First I'm trying to get the directory name which has 365.txt file and accordingly, I will execute the command to move the files to Archive

Here I'm getting directory with file name as output, I would like to cut the 365.txt from the output.

Now I'll store this directory in a variable and execute the mv command.

Code:
[psoft@shdpsap1lql ~]$ find $src -type f -name 365.txt
/omount/zy_custom/outbound/archive/velu/365.txt
[psoft@shdpsap1lql ~]$


Moderator's Comments:
Mod Comment Please use CODE tags to mark code, input & output/errors

Last edited by rbatte1; 08-06-2014 at 08:38 AM.. Reason: Added CODE tags
# 6  
Old 08-06-2014
The partial command in question: cut -f-4 -d\"\/\" |sort -u splits the values from the preceding command based on a forward slash /. The backslashes \ are needed because the quotes " and forward slash / are special to the shell and might get interpreted as meaning something rather than just being plain text.

The sort -u is a sort-unique, so having cut off the first 4 fields (null and three layers of directory split by /) there will be duplicates. This ensures that there is only one of each to be assigned to the variable in question.

I'm not sure if that makes it any clearer though. Maybe an example is better. Consider a directory structure like this:-
Code:
/a/b/c/d/e/1
/a/b/c/d/e/2
/a/b/c/d/e/3
/a/b/c/d/Z/1
/a/b/c/d/Z/2
/a/b/9/d/e/1
/a/b/9/d/e/2

If the first three layers of directories are significant, you want to get the result like this:-
Code:
/a/b/c
/a/b/9

To do this, my code would first get the first four fields (and delimiters / with the cut command to give you this:-
Code:
/a/b/c/d/e/1
/a/b/c
/a/b/c
/a/b/c
/a/b/c
/a/b/9
/a/b/9

..... then the sort would reduce it to the required list.

I hope that is a little better.


On the question of finding directories and the '365' file, you might need more steps:
  • Find all the significant directories as above (storing them in $SRC_CDR_OUTBOUND etc.)
  • Change your find to a loop for each directory in $SRC_CDR_OUTBOUND:-
    • Check if the file 365.txt exists with if [ -f $directory/365.txt ] and if it does set the value of days as 365, else 90.
    • Run the find and mv for the specific directory using the value of days to define the -mtime limit
  • Go round the loop again for the next directory in $SRC_CDR_OUTBOUND etc.
Does this logic work for you, or have I missed the point?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Please remember to use CODE tags. Simply wrap code and data input/output in CODE tags, like this:-
Quote:
[CODE]This is my code[/CODE]
to produce the following (fixed character width, space respected):-
Code:
This is my code

Not only does it make posts far easier to read, but CODE and ICODE sections respect multiple space and have fixed width characters, which is important for easily seeing input/output requirements.

Last edited by rbatte1; 08-06-2014 at 09:02 AM.. Reason: Correcting LIST formatting
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to copy files/folders and show the files/folders?

Hi, So i know we use cp -r as a basic to copy folders/files. I would like this BUT i would like to show the output of the files being copied. With the amazing knowledge i have i have gone as far as this: 1) find source/* -exec cp -r {} target/ \; 2) for ObjectToBeCopied in `find... (6 Replies)
Discussion started by: Imre
6 Replies

2. UNIX for Dummies Questions & Answers

Archive folders and sub folders

Hi Can i archive folder and folders in with the tar command My files are located in subfolders Eg: Folder1/Folder1_1/*.pdf Folder1/Folder1_2/*.pdf Folder1/Folder1_3/*.pdf so i would like to tar all the files in Folder1_1 and Folder1_2 only not Folder1_3 that should be done next... (2 Replies)
Discussion started by: cnrj
2 Replies

3. Shell Programming and Scripting

Copy between two different folders containing same sub-folders

I have a folder like this ls input1 dir1 dir2 dir3 file1 file2 file3 dir1, dir2 and dir3 are sub-folders inside the folder input1 ls input2 dir1 dir2 dir3 file1 file2 file3 My dir1 in input1 folder has files f1, f2, f3 and f4. My dir1 in input2 folder has file f4 and f5. ... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

4. Shell Programming and Scripting

Grep for a srting & exclude two folders

Hi, Below is the command to grep for a string under grep -r "redeem" /home/tom Need to make it case insensitive and exclude logs & tmp folders under /home/tom directory in my Search. Need this in Linux. (1 Reply)
Discussion started by: mohtashims
1 Replies

5. Ubuntu

[Solved] Using Find with an exclude/exclude file

I am familiar with using tar and exclude/include files: tar zcf backup.dirs.tgz --files-from=include.mydirs --exclude-from=exclude.mydirs --no-recursion but was wondering if I could use find in the same way. I know that you can just specify the directories to exclude but my list is... (2 Replies)
Discussion started by: metallica1973
2 Replies

6. UNIX for Dummies Questions & Answers

Searching for folders/parent folders not files.

Hello again, A little while back I got help with creating a command to search all directories and sub directories for files from daystart of day x. I'm wondering if there is a command that I've overlooked that may be able to search for / write folder names to an output file which ideally... (2 Replies)
Discussion started by: Aussiemick
2 Replies

7. Shell Programming and Scripting

Making 99 folders 99 folders deep

I am trying to make a unix shell script that will make 99 folders 99 deep (counting the first level folders). So far i have made it make the first 99 folders and 99 more in all of the folders. The only problem is the only way i have found is copying and pasting part of the script over and over and... (18 Replies)
Discussion started by: YukonAppleGeek
18 Replies

8. Shell Programming and Scripting

How to exclude folders/files in search?

I have a directory with about 20 folders and many different types of files. I need to search for files and gzip in all the directories except for 1 directory. How do you exclude a directory? (2 Replies)
Discussion started by: bbbngowc
2 Replies

9. UNIX for Dummies Questions & Answers

Copying Folders without some folders... ;-)

I am in a fix....... I have to write a backup script to backup say Folder A. Folder A contains n folders 1,2 ,3 .....n. my script should copy A without folder 2 & 3. Is there anyway I can do it without writing individual copy commands???? Please help.... (5 Replies)
Discussion started by: chimpu
5 Replies

10. Shell Programming and Scripting

Backing up Folders without some folders...;)

I am in a fix....... I have to write a backup script to backup say Folder A. Folder A contains n folders 1,2 ,3 .....n. my script should copy A without folder 2 & 3. Is there anyway I can do it without writing individual copy commands???? Please help.... (1 Reply)
Discussion started by: chimpu
1 Replies
Login or Register to Ask a Question