Script For Folder Management Automation


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script For Folder Management Automation
# 1  
Old 01-18-2012
Script For Folder Management Automation

Let me start off by saying I am not a linux guy nor do I script much (I modify scripts a bit) but I am looking for a code to help automate file management on usb sticks with clonezilla images.

I'm looking for a script to recognize folder names and allow a max of 2 folders with the same text in each name.

For example:

/home/partimag$ ls

apple-1-14-2012
apple-1-15-2012
apple-1-16-2012
apple-1-18-2012
banana-1-15-2012
banana-1-16-2012
crape-1-13-2012
crape-1-15-2012
crape-1-16-2012
crape-1-17-2012

=================================

The script would delete images if I had more than 2 of each listed.

/home/partimag$ ls

apple-1-14-2012 <---delete
apple-1-15-2012 <---delete
apple-1-16-2012
apple-1-18-2012
banana-1-15-2012
banana-1-16-2012
crape-1-13-2012 <---delete
crape-1-15-2012 <---delete
crape-1-16-2012
crape-1-17-2012

Keep the newest and delete those that exceed folder limitations for each of the three types. 'apple', 'banana', and 'crape'.
==================================

Thanks for reading and look forward to ideas.

Last edited by traustic; 01-19-2012 at 02:29 AM..
# 2  
Old 01-18-2012
these are duplicates:
Code:
banana-1-16-2012
banana-1-16-2012

These are not, right?
Code:
crape-1-13-2012 <---delete
crape-1-15-2012 <---delete

Assuming I got what you want:
Code:
cd /path/to/usb
find . -type d | 
     awk ' arr[$0]++; END {for (i in arr){if(arr[i]>1){print i}  } } > tmp.tmp
while read dir
do
   rm -R $dir
done < tmp.tmp

BE careful. This can clobber lots of stuff, and I may not have understood what you really want. It deletes BOTH duplicate directories, as you specified.
# 3  
Old 01-19-2012
Thanks for responding.
=====================
these are duplicates:

Code:
banana-1-16-2012 banana-1-16-2012===========================
Those were a mistake - all folder names are different, beginning is the same and the date is different for all.

Basically I save images of live systems and name them:
Apple-date-i-made-image
Banana-date-i-made-image
Crape-date-i-made-image

Over time I create:
apple-date-i-made-image 4-5 times with different dates but same beginning.
banana-date-i-made-image 6 times with different dates but same beginning.
crape-date-i-made-image 3 times with different dates but same beginning.

I'm going to distribute this cloner configuration to coworkers and need something that cleans up extra images they aren't using. So a max of 2 images with the same 'intro name' exist on the usb. Apple only works on machine 1, Banana works on machine 2, and Crape works on machine 3 and they are not interchangeable.

---------- Post updated 01-19-12 at 09:39 PM ---------- Previous update was 01-18-12 at 10:56 PM ----------

any ideas?

Want to keep two images available on usb for each of three systems.

Windows XP image
Windows 7 image
Windows NT Server image

All three computers contain different hardware settings and want to keep the two latest and delete the rest.

---------- Post updated at 11:56 PM ---------- Previous update was at 09:39 PM ----------

Found something:

Code:
#!/bin/bash
# edit above line to the path to bash
#command to find is: "which bash"
#
##############set your variables below###########
# prefix is the prefix of the filename,
# example filename: server1-07-20-2011.bak, the prefix would be: server1
# numtokeep is the number of backups you would like to preserve
# backupFolder is the location of your backup files, exclude trailing slash
#
prefix=poop
numtokeep=3
backupFolder=/home/dustin/test
#################################################
#
#  go to backup folder just to be safe
cd $backupFolder
#
# count how many files are in backup folder, and set to a variable
filecount=`ls -l $backupFolder/$prefix* | wc -l`
#
# checking to see if number of backups is greater than number to keep
if [ $filecount -gt $numtokeep ]
        then
                extras=$(($filecount - $numtokeep))
                echo "leaving $numtokeep, deleting $extras"
                for counter in $(ls -ltr $backupFolder/$prefix* |  head -n $extras | awk '{ print $8 }')
                do
                        if [ "$1" == "--test" ]
                        then
                                echo "Will delete: $counter"
                        else
                                echo "deleting: $counter"
                                rm -R $counter
                                if [ $? != 0 ] ; then
                                echo "Error deleting: $counter"
                                else
                                echo "Successfully deleted: $counter"
                                fi
                        fi
                done
        else
                echo "not enough backups to delete"
fi


from:
dustinmihalko.com/2011/07/bash-script-to-delete-old-files-leaving-last-few/

Last edited by Franklin52; 01-20-2012 at 03:11 AM.. Reason: Please use code tags for code and data samples, thank you
# 4  
Old 01-20-2012
run below command without red parts first.
Code:
ls |sort -t \- -k1,1 -k4,4nr -k2,2nr -k3,3nr |awk -F \- '++a[$1]>2' |xargs rm -rf

# 5  
Old 01-20-2012
got the script sorta working but having issues....

set
Code:
filecount=`ls -l $backupFolder/$prefix* | wc -l`

to
Code:
filecount=`ls -d $backupFolder/$prefix* | wc -l`

so it will list only directorys in path with prefix and works good

script runs and stops at 'do' after the following:
Code:
for counter in $(ls -ltr $backupFolder/$prefix* |  head -n $extras | awk '{ print $8 }')

the console shows 'leaving 4, deleting 1"' and does nothing else

---------- Post updated at 02:21 AM ---------- Previous update was at 02:17 AM ----------

nice rdcwayx - lists four oldest directories out of 8 there for deletion

---------- Post updated at 02:30 AM ---------- Previous update was at 02:21 AM ----------

rdcwayx

code posted sorts by numbers in the directory name rather than the creation/modifed date?
if I have an image named winxp-1-27-2012 and name a new one 2-1-2012
won't it delete 2-1-2012 and keep the old directory?

Last edited by Franklin52; 01-20-2012 at 03:11 AM.. Reason: Please use code tags for code and data samples, thank you
# 6  
Old 01-20-2012
sort the folder name with date.

You can try to create a folder name apple-2-1-2012, and run the sort/awk command only, you should see it is not list for deleting
# 7  
Old 01-20-2012
thanks for your help - it works well

I also managed to get this to work:
Code:
#!/bin/bash
# edit above line to the path to bash
#command to find is: "which bash"
#
##############set your variables below###########
# prefix is the prefix of the filename,
# example filename: server1-07-20-2011.bak, the prefix would be: server1
# numtokeep is the number of backups you would like to preserve
# backupFolder is the location of your backup files, exclude trailing slash
#
prefix=poop
numtokeep=3
backupFolder=/home/partimag
#################################################
#
#  go to backup folder just to be safe
cd $backupFolder
#
# count how many files are in backup folder, and set to a variable
filecount=`ls -d $backupFolder/$prefix* | wc -l`
#
# checking to see if number of backups is greater than number to keep
if [ $filecount -gt $numtokeep ]
        then
                extras=$(($filecount - $numtokeep))
                echo "leaving $numtokeep, deleting $extras"
                for counter in $(ls -dlt $backupFolder/$prefix* |  head -n $extras | awk '{ print $9 }')
                do
                        if [ "$1" == "--test" ]
                        then
                                echo "Will delete: $counter"
                        else
                                echo "deleting: $counter"
                                rm -R $counter
                                if [ $? != 0 ] ; then
                                echo "Error deleting: $counter"
                                else
                                echo "Successfully deleted: $counter"
                                fi
                        fi
                done
        else
                echo "not enough backups to delete"
fi

Moderator's Comments:
Mod Comment How to use code tags


---------- Post updated at 03:39 AM ---------- Previous update was at 03:07 AM ----------

found a problem in above code - it deletes the newest and keeps the oldest - just reverse the order of the ls:

Code:
 for counter in $(ls -dltr $backupFolder/$prefix* |  head -n $extras | awk '{ print $9 }')


Last edited by traustic; 01-20-2012 at 03:21 AM.. Reason: Please use code tags for code and data samples, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Request for Shell script to move files from Subfolder to Parent folder and delete sub folder

Hi Team, I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder. Eg: parent folder --> /Interface/data/test/IN Sub folder -->/Interface/data/test/IN/Invoice20180607233338 Subfolder will be always with timestamp... (6 Replies)
Discussion started by: srivarun15
6 Replies

2. Shell Programming and Scripting

Automation Script for Oracle

Hi, As a Oracle Developer, I am writing many Procedures,Functions and Packages. Facing Many optimization issue after writing these Database objects. Trying to tune it manually. Can we write any Shell/Perl/Python script to Optimize these Database objects instead of doing manual check and... (1 Reply)
Discussion started by: vasuvv
1 Replies

3. Shell Programming and Scripting

Automation script

Hello All , I came across a tricky solution to devolop . Here is a part of the requirement automation . I have different set of server say : Web ( has 4 servers under it ) , App ( has 4 servers under it ) , DB ( has 2 servers under it ) Above each i have different load balancers , Say : Web... (4 Replies)
Discussion started by: radha254
4 Replies

4. Shell Programming and Scripting

awk script automation

I have the below code which calculates the time difference between src and dst from a large trace file. The code works for a given source and destination. However, I want to automate the code to go over any src and destination. The format of the source is like that: X.Y where x is always =2 and Y... (10 Replies)
Discussion started by: ENG_MOHD
10 Replies

5. Shell Programming and Scripting

Script to move one folder to multiple folder...

Hi All, I have to requirement to write a shell script to move file from one folder (A) to another five folder (B,C,D,E,F) and destination folder should be blank. In not blank just skip. This script will run as a scheduler every 2 minutes. It will check number of files in folder A and move 1 to... (9 Replies)
Discussion started by: peekuabc
9 Replies

6. Shell Programming and Scripting

Script Automation

Hi Gurus, I have a clearcase script that i use to check in a single file at time on my clearcase server. the script is as follows setmyview settask 75098_MSI_TRILOGY_EIM cd /vobs/Trilogy_R12/custom/msieim/12.0.0/sql/ cleartool co -nc . ct mkelem -nc Filename_1.sql cp... (3 Replies)
Discussion started by: r_t_1601
3 Replies

7. Shell Programming and Scripting

Help with Shell Script automation

can someone look into this one please... I am struck at this point. I do not know what logic to be followed here. I can go ahead with my work only, if this step is done. Please Help. I have a process X in a shell script. Once the process X is done, it generates a log file. Process X is basically... (1 Reply)
Discussion started by: ss3944
1 Replies

8. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

9. Shell Programming and Scripting

FTP automation script

Hi, I have got a requirement like this. a parameterized function custFtp which will take 5 i/ps and will do the following tasks. p1) server name p2) username p3) password p4) path name of the server where the file resides p5) file name pattern the function will work like this. ... (1 Reply)
Discussion started by: ani_datta
1 Replies

10. Shell Programming and Scripting

script for Finding files in a folder and copying to another folder

Hi all, I have a folder '/samplefolder' in which i have some files like data0.txt, data1.txt and data2.txt. I have to search the folder for existence of the file data0.txt first and if found have to copy it to some other file; next i have to search the folder for existence of file... (5 Replies)
Discussion started by: satish2712
5 Replies
Login or Register to Ask a Question