Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Script to tar/rsync/rm multiple folder names

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to tar/rsync/rm multiple folder names
# 15  
Old 04-16-2016
so it will continue with my script even if i dont specify an ELSE statement and just use an IF statement, unless i include in my IF statement an EXIT statement?

because all im going to care about is if it doesnt succed in doing the tar, rsync and, rm commands so im just going to do if doesnt equal 0 email me and exit the script

---------- Post updated at 07:00 PM ---------- Previous update was at 05:27 PM ----------

using just the if statements for all the commands im doing as i want to to notified if any commands fail but the last command i have done a else aswell

its working great

Code:
#!/bin/bash
cd /to_be_archived/
for DIR in * ; do
fSaveDir="${DIR##*/}"
tar -cf "${fSaveDir}".tar "${fSaveDir}"
if [ $? -ne 0 ]
then
sendmail "${fSaveDir}" robertkwild@gmail.com <<< "creating of tar "${fSaveDir}" failed due to error"
exit
fi
rsync -a "${fSaveDir}".tar /archived_projects/
if [ $? -ne 0 ]
then
sendmail "${fSaveDir}" robertkwild@gmail.com <<< "rsync of "${fSaveDir}" failed due to error"
exit
fi
rm -f "${fSaveDir}".tar
if [ $? -ne 0 ]
then
sendmail "${fSaveDir}" robertkwild@gmail.com <<< "removing of tar "${fSaveDir}" failed due to error"
exit
fi
rm -rf "${fSaveDir}"
if [ $? -ne 0 ]
then
sendmail "${fSaveDir}" robertkwild@gmail.com <<< "removing of "${fSaveDir}" failed due to error"
exit
else
sendmail "${fSaveDir}" robertkwild@gmail.com <<< "successfully completed archiving "${fSaveDir}""
fi
done

Last edited by robertkwild; 04-16-2016 at 09:08 PM..
# 16  
Old 04-17-2016
A few notes on your code and a possible alternative for you to consider...
  1. Learn to indent your code as bakunin has suggested several times. It not only looks more professoinal, it makes it easier for anyone (including you) to read and understand what your code is trying to do, makes it easer to spot missing syntax elements (like a do or a done or a then or a fi). I have a coding style that is a little bit different form bakunin's style. I don't care what style you use as long as you pick one and use it consistently.
  2. Since you cd into the directory containing the directories you want to process and use for DIR in *, $DIR will never expand to a string containing a <slash> character and, therefore, fSaveDir will always contain exactly the same string as DIR. Therefore, I have gotten rid of fSaveDir and just use DIR.
  3. Since you are creating tar archives in the directory containing your project directories, and you don't remove them if something fails, you need to verify that the file you are processing ($DIR) is a directory. There are several ways to do that. I have chosen to simply test for non-directory file in an if statement and use a continue to silently skip over non-directory files.
  4. The sendmail utiltiy (at least on my system) does not treat its 1st operand as a subject line (and has no option to set a subject line on the command line). I don't see why you would want a separate e-mail message for each project processed, and I don't see why you would want to stop all processing if you hit one error. The code below uses mailx instead of sendmail and only sends one message containing the status for each project directory processed by one invocation of your script.
  5. As noted before by bakunin, c and f are not options to tar and should not be preceded by a hyphen.
Please note that none of this has been tested in any way, but I think it comes close to doing what you are trying to do:
Code:
#!/bin/bash
cd /to_be_archived/
for DIR in *
do	if [ ! -d "$DIR" ]
	then	# This file is not a directory, skip to next file...
		continue
	fi
	printf 'Processing proejct: %s\n' "$DIR"
	if ! tar cf "${DIR}".tar "${DIR}"
	then	printf 'Creating "%s.tar" failed.\n' "$DIR"
		continue
	fi
	if ! rsync -a "${DIR}".tar /archived_projects/
	then	printf 'rsync of "%s.tar" failed.\n' "$DIR"
		continue
	fi
	if ! rm -f "${DIR}".tar
	then	printf 'Removing "%s.tar" failed.\n' "$DIR"
		continue
	fi
	if ! rm -rf "${DIR}"
	then	printf 'Removing project "%s" failed.\n' "$DIR"
		continue
	fi
	printf 'Project '%s" successfully archived and removed.\n' "${DIR}"
done 2>&1 | mailx -s "${0##*/} Status report $(date)" robertkwild@gmail.com

# 17  
Old 04-17-2016
But surely i dont want to continue the script if it fails on the taring rsync or removing

Thats why i put an exit statement instead so if fails it can let me know via email and exit the script

Also lets say i have multiple folders i want the script to run lets say it exits on the first tarring of folder 1, will it try and do folder 2 and so on so forth or will it exit altogther
# 18  
Old 04-17-2016
Quote:
Originally Posted by robertkwild
But surely i dont want to continue the script if it fails on the taring rsync or removing

Thats why i put an exit statement instead so if fails it can let me know via email and exit the script

Also lets say i have multiple folders i want the script to run lets say it exits on the first tarring of folder 1, will it try and do folder 2 and so on so forth or will it exit altogther
Please look at post #12 in this thread again!

If you have five projects to archive and remove, and the tar on the first one fails; if you use exit, your script will exit and there will be absolutely no attempt to tar, rsync, and rm the other four projects. That is why I used continue instead of exit. Using continue, the script I suggested will stop processing the first project (if any step fails) but will then continue processing other projects until all five projects have been processed. When it is done processing all five projects (successfully or unsuccessfully), it will send you one e-mail giving you the status of all five projects (including the normal and diagnostic output produced by each command run in the script for each project). Isn't that what you want to do?
# 19  
Old 04-17-2016
ok thanks,

i thought exit would exit folder 1 if it failed any steps and continue doing folder 2

so when you say continue, it wont continue with folder 1 if it failed any steps but instead it will go to folder 2 and once its done all 5 it will give you an email of all the folders its succedded or failed?

so my coding

Code:
tar -cf "${fSaveDir}".tar "${fSaveDir}"
if [ $? -ne 0 ]
then
sendmail "${fSaveDir}" robertkwild@gmail.com <<< "creating of tar "${fSaveDir}" failed due to error"
exit
fi

if i change the exit to a continue, will it stop working on folder 1 and not even bother doing the other commands (rsync and rm) and start on working on folder 2?
Last edited by robertkwild; 04-17-2016 at 05:29 PM..
# 20  
Old 04-17-2016
Quote:
Originally Posted by robertkwild
ok thanks,

i thought exit would exit folder 1 if it failed any steps and continue doing folder 2
No, No, NO! Calling exit terminates your script; not just the current iteration of the loop containing it.

Quote:
so when you say continue, it won't continue with folder 1 if it failed any steps but instead it will go to folder 2 and once its done all 5 it will give you an email of all the folders its succeeded or failed?
Yes. Calling continue causes the script to skip any remaining steps in the current iteration of the loop and continue processing starting at the beginning of the next iteration of the loop that contains it.
# 21  
Old 04-18-2016
when you say the current iteration of the loop, do you mean any if/else statements inbetween the do and done commands?

---------- Post updated at 05:14 AM ---------- Previous update was at 04:35 AM ----------

i will make it neater near when im nearly done finalizing it but its looking good

Code:
#!/bin/bash
cd /to_be_archived/
for DIR in * ; do
fSaveDir="${DIR##*/}"
tar -cf "${fSaveDir}".tar "${fSaveDir}"
if [ $? -ne 0 ]
then
mail -s "${fSaveDir}" robertw@molinare.co.uk <<< "creating of tar "${fSaveDir}" failed due to error"
rm -f "${fSaveDir}".tar
continue
fi
rsync -a "${fSaveDir}".tar /archived_projects/
if [ $? -ne 0 ]
then
mail -s "${fSaveDir}" robertw@molinare.co.uk <<< "rsync of "${fSaveDir}" failed due to error"
continue
fi
rm -f "${fSaveDir}".tar
if [ $? -ne 0 ]
then
mail -s "${fSaveDir}" robertw@molinare.co.uk <<< "removing of tar "${fSaveDir}" failed due to error"
continue
fi
rm -rf "${fSaveDir}"
if [ $? -ne 0 ]
then
mail -s "${fSaveDir}" robertw@molinare.co.uk <<< "removing of "${fSaveDir}" failed due to error"
continue
else
mail -s "${fSaveDir}" robertw@molinare.co.uk <<< "successfully completed archiving "${fSaveDir}""
fi
done

basically what ive done is if fails on the tarring process i want it to delete the tar file straight away
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Checking Multiple File existance in a UNIX folder(Note: File names are all different)

HI Guys, I have some 8 files with different name and extensions. I need to check if they are present in a specific folder or not and also want that script to show me which all are not present. I can write if condition for each file but from a developer perspective , i feel that is not a good... (3 Replies)
Discussion started by: shankarpanda003
3 Replies

2. Homework & Coursework Questions

Bash script for printing folder names and their sizes

1. The problem statement, all variables and given/known data: The task is to create a script that would reproduce the output of 'du' command, but in a different way: what 'du' does is: <size> <folder name>and what is needed is <folder name> <size>We need to show only 10 folders which are the... (3 Replies)
Discussion started by: UncleIS
3 Replies

3. Shell Programming and Scripting

Bash script for printing folder names and their sizes

Good day, everyone! I'm very new to bash scripting. Our teacher gave us a task to create a script that basically does the same job the 'du' command does, with the difference that 'du' command gives an output in the form of <size> <folder name>and what we need is <folder name> <size>As for... (1 Reply)
Discussion started by: UncleIS
1 Replies

4. Shell Programming and Scripting

Script to move one folder to multiple folder...

Hi All, I have to requirement to write a shell script to move file from one folder (A) to another five folder (B,C,D,E,F) and destination folder should be blank. In not blank just skip. This script will run as a scheduler every 2 minutes. It will check number of files in folder A and move 1 to... (9 Replies)
Discussion started by: peekuabc
9 Replies

5. UNIX for Dummies Questions & Answers

Do I need to extract the entire tar file to confirm the tar folder is fine?

I would like to confirm my file.tar is been tar-ed correctly before I remove them. But I have very limited disc space to untar it. Can I just do the listing instead of actual extract it? Can I say confirm folder integrity if the listing is sucessful without problem? tar tvf file1.tar ... (1 Reply)
Discussion started by: vivien_chu
1 Replies

6. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

7. Shell Programming and Scripting

editing names of files in multiple folder

I have 1000's of directories which is named as numbers. Each directory contains multiple files. Each of these directories have a file named "att". I need to rename all the att files by adding the directory name followed by "_" then att for each of the directories. Directories 120 att... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

8. Shell Programming and Scripting

Script to move files with similar names to folder

I have in directory /media/AUDIO/WAVE many .mp3 files with names like: my filename_01of02.mp3 my filename_02of02.mp3 Your File_01of06.mp3 Your File_02of06.mp3 etc.... In the same directory, /media/AUDIO/WAVE, I have many folders with names like 9780743579490 9780743579491 etc.. Inside... (7 Replies)
Discussion started by: glev2005
7 Replies

9. Shell Programming and Scripting

lvm/tar/rsync backup script feedback/criticism

I have written a shell script to perform backups using tar, rsync and optionally utilise lvm snapshots. The script is not finished but is in a working state and comments/descriptions are poor. I would greatly appreciate any criticism and suggestions of the script to help improve my own learning... (0 Replies)
Discussion started by: jelloir
0 Replies

10. UNIX for Dummies Questions & Answers

Copying multiple folders to local machine (don't know folder names)

Hi. I'm trying to copy multiple folders from the remote machine to the local machine. I wrote a batch file to run an ftp window. The problem I am having is that the only command to copy files is mget *, and this copies only files, not folders. For example, ftp ts555 cd ts555/test ' test... (5 Replies)
Discussion started by: leenyburger
5 Replies
Login or Register to Ask a Question