Script to tar/rsync/rm multiple folder names


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to tar/rsync/rm multiple folder names
# 29  
Old 04-21-2016
sorry for the late reply, just been busy work and all

i have added to my script if the tar file already exists in the "archived_projects" folder do not over write and also the size and file count

Code:
#!/bin/bash
cd /to_be_archived/
for DIR in * ;

    do
    MailAddress="robertw@molinare.co.uk"

tar -cf "$DIR".tar "$DIR"

    if [ $? -ne 0 ]
    then
    mail -s "tar failed $DIR" $MailAddress <<< "creating of tar $DIR failed due to error, removing $DIR.tar"
    rm -f "$DIR".tar
    continue
    fi

    if [ -f /archived_projects/"$DIR".tar ]; then
      mail -s "duplicate exists $DIR" $MailAddress <<< "$DIR.tar already exists"
    rm -f "$DIR".tar
    continue
    fi

rsync -a "$DIR".tar /archived_projects/

    if [ $? -ne 0 ]
    then
    mail -s "rsync failed $DIR" $MailAddress <<< "rsync of $DIR failed due to error, removing $DIR.tar"
    rm -f "$DIR".tar
    continue
    fi

rm -f "$DIR".tar

    if [ $? -ne 0 ]
    then
    mail -s "remove tar failed $DIR" $MailAddress <<< "removing of tar $DIR failed due to error"
    continue
    fi

du -h "$DIR" >> /archive_details
ls -1 /to_be_archived/"$DIR" | wc -l >> /archive_details

rm -rf "$DIR"

    if [ $? -ne 0 ]
    then
    mail -s "remove folder failed $DIR" $MailAddress <<< "removing of $DIR failed due to error"
    continue
        else
        mail -s "success $DIR" $MailAddress <<< "successfully completed archiving $DIR"
    fi

    done

---------- Post updated at 08:27 AM ---------- Previous update was at 06:10 AM ----------

i think thats a good idea condensing the remove commands to one, i will do that in my next revision, but what bakunin said about you dont need to add the return code to the if statement, i dont seem to get it as for every command you type if it returns 0 (success) or 1-255 (failed) it will either do then or the else statement

but you havnt put in any return code so how does it know whether the command before has succeeded or failed?

Last edited by robertkwild; 04-21-2016 at 11:52 AM..
# 30  
Old 04-21-2016
Quote:
Originally Posted by robertkwild
i think thats a good idea condensing the remove commands to one, i will do that in my next revision, but what bakunin said about you dont need to add the return code to the if statement, i dont seem to get it as for every command you type if it returns 0 (success) or 1-255 (failed) it will either do then or the else statement

but you havnt put in any return code so how does it know whether the command before has succeeded or failed?
OK, short introduction to the intractes of "if":

"if" is followed by a (any) command and it executes first this command, then the "then...."-part if the command returns RC 0 or the "else..."-part, if not. This is always the case. For instance:

Code:
if some_command ; then
     echo "some_command returned 0"
else
     echo "some_command returned something else"
fi

Here, the command "some_command" is executed. If it returns 0 the then-part is executed, if it returns something else the "else"-part is executed.

You may ask yourself where this:

Code:
if [ $? -eq 0 ] ; then

now comes from because there seems no command to be involved: wrong. In fact there is a command involved and it is called "test". "test" does all kinds of tests: integer comparisons, string comparisons and tests for the existence (or certain attributes) of files and it returns a RC of 0 for the comparisons being (logically) true and an RC <> 0 for comparisons being logically false.

Historically, in the first UNIX versions the construct looked like this (and you may see this still in some very old scripts):

Code:
if test $? -eq 0 ; then

or maybe:
Code:
if /usr/bin/test $? -eq 0 ; then

You see, the expression to be evaluated consists just of arguments to "test". "test", is still there, i suggest to read the man page of it.

But because this didn't look like the programming languages the inventors of UNIX knew (they were all programmers, after all), they created a device which made it look more like what they were accustomed to: they created a symbolic link from /usr/bin/test to /usr/bin/[ - in fact "[" is a possible filename - and the line now looked like this:

Code:
if [ $? -eq 0 ; then

In some variants of UNIX (HP-Ux, if i remember right) this link still exists.

That was actually better but it looked like a bracket was opened and never closed. So the final change to /usr/bin/test was that it expected a "]" as the last argument if it was called as "[" - and this created the look we know today:

Code:
if [ $? -eq 0 ] ; then

Notice, that for this reason the following won't work:

Code:
if [ $? -eq 0] ; then

because the last parameter now would be "0]", and the expected final "]" would be missing. Try it, you will see that it leads to a syntax error - for the reasons stated above. The following weird-looking construct, though, works perfectly:

Code:
if [ "$?" "-eq" "0" "]" ; then

The last step in the development was that programmers of UNIX shells (and i think David Korn with his Korn shell was the first) noticed that "test" and its equivalent was called so often that they made it a shell built-in. This will prevent the necessity of loading and starting an external executable (time-consuming activities) and is a speed optimisation. Still, the buil-in "[" works the same as the external /usr/bin/test (again, with the exception of expecting "]" as the last argument) as the man page for the shell will tell you.

I hope this helps.

bakunin

Last edited by bakunin; 04-21-2016 at 12:48 PM..
# 31  
Old 04-21-2016
Quote:
Originally Posted by bakunin
... ... ...

But because this didn't look like the programming languages the inventors of UNIX knew (they were all programmers, after all), they created a device which made it look more like what they were accustomed to: they created a symbolic link from /usr/bin/test to /usr/bin/[ - in fact "[" is a possible filename - and the line now looked like this:

Code:
if [ $? -eq 0 ; then

In some variants of UNIX (HP-Ux, if i remember right) this link still exists.

That was actually better but it looked like a bracket was opened and never closed. So the final change to /usr/bin/test was that it expected a "]" as the last argument if it was called as "[" - and this created the look we know today:

Code:
if [ $? -eq 0 ] ; then

... ... ...

I hope this helps.

bakunin
The POSIX standards require that both test and [ (and all other standard utilities except special built-ins) exist as stand-alone utilities (not just as shell built-ins). This allows them to be used as command arguments to the env, find -exec, nice, nohup, time, and xargs utilities.

The [ expression ] form of the test utility was actually added to UNIX systems before symlinks. On many UNIX and UNIX-like systems, the test and [ utilities are hard links to the same executable file instead of one of them being a symlink or both of them being separate executables.
# 32  
Old 04-24-2016
so am i right in thinking -

Code:
tar -cf "$DIR".tar "$DIR"
if [ $? -ne 0 ]
then
mail -s "tar failed $DIR" $MailAddress <<< "creating of tar $DIR failed due to error, removing $DIR.tar"
rm -f "$DIR".tar
continue
fi

is the same as this -

Code:
if tar -cf "$DIR".tar "$DIR" ; else
mail -s "tar failed $DIR" $MailAddress <<< "creating of tar $DIR failed due to error, removing $DIR.tar"
rm -f "$DIR".tar
continue
fi

as you said then is equal to 0 and else is not equal to 0?

many thanks

rob

Last edited by robertkwild; 04-24-2016 at 06:39 PM..
# 33  
Old 04-24-2016
Quote:
Originally Posted by robertkwild
so am i right in thinking -

Code:
tar -cf "$DIR".tar "$DIR"
if [ $? -ne 0 ]
then
mail -s "tar failed $DIR" $MailAddress <<< "creating of tar $DIR failed due to error, removing $DIR.tar"
rm -f "$DIR".tar
continue
fi

is the same as this -

Code:
if tar -cf "$DIR".tar "$DIR" ; else
mail -s "tar failed $DIR" $MailAddress <<< "creating of tar $DIR failed due to error, removing $DIR.tar"
rm -f "$DIR".tar
continue
fi

as you said then is equal to 0 and else is not equal to 0?

many thanks

rob
No! You can't have an if statement without a then clause. But, you can do it as I suggested back in post #16 in this thread:
Code:
	if ! tar cf "${DIR}".tar "${DIR}"
	then	mail -s "tar failed $DIR" $MailAddress <<< "creating of tar $DIR failed due to error, removing $DIR.tar"
		rm -f "$DIR".tar
		continue
	fi

Note the exclamation point in red in the if statement. That reverses the exit status of the given command (tar in this case) so that if the command fails (exits with a non-zero exit status) it will execute the then clause and if the command succeeds (i.e., exits with zero exit status) it will execute the else clause if there is one.
# 34  
Old 04-24-2016
Ohh so an ! Means not equal to 0
# 35  
Old 04-24-2016
Quote:
Originally Posted by robertkwild
Ohh so an ! Means not equal to 0
No. The ! keyword in the shell doesn't MEAN anything. It DOES something. As I said before, it "reverses the exit status of the given command (tar in this case) so that if the command fails (exits with a non-zero exit status) it will execute the then clause and if the command succeeds (i.e., exits with zero exit status) it will execute the else clause if there is one.

Look at each of the following if statements and try to determine what the output will be. Then run the command and see if it did what you expected:
Code:
if true; then echo then; else echo else; fi

if ! true; then echo then; else echo else; fi

if ! ! true; then echo then; else echo else; fi

if false; then echo then; else echo else; fi  
 
if ! false; then echo then; else echo else; fi

if ! ! false; then echo then; else echo else; fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Checking Multiple File existance in a UNIX folder(Note: File names are all different)

HI Guys, I have some 8 files with different name and extensions. I need to check if they are present in a specific folder or not and also want that script to show me which all are not present. I can write if condition for each file but from a developer perspective , i feel that is not a good... (3 Replies)
Discussion started by: shankarpanda003
3 Replies

2. Homework & Coursework Questions

Bash script for printing folder names and their sizes

1. The problem statement, all variables and given/known data: The task is to create a script that would reproduce the output of 'du' command, but in a different way: what 'du' does is: <size> <folder name>and what is needed is <folder name> <size>We need to show only 10 folders which are the... (3 Replies)
Discussion started by: UncleIS
3 Replies

3. Shell Programming and Scripting

Bash script for printing folder names and their sizes

Good day, everyone! I'm very new to bash scripting. Our teacher gave us a task to create a script that basically does the same job the 'du' command does, with the difference that 'du' command gives an output in the form of <size> <folder name>and what we need is <folder name> <size>As for... (1 Reply)
Discussion started by: UncleIS
1 Replies

4. Shell Programming and Scripting

Script to move one folder to multiple folder...

Hi All, I have to requirement to write a shell script to move file from one folder (A) to another five folder (B,C,D,E,F) and destination folder should be blank. In not blank just skip. This script will run as a scheduler every 2 minutes. It will check number of files in folder A and move 1 to... (9 Replies)
Discussion started by: peekuabc
9 Replies

5. UNIX for Dummies Questions & Answers

Do I need to extract the entire tar file to confirm the tar folder is fine?

I would like to confirm my file.tar is been tar-ed correctly before I remove them. But I have very limited disc space to untar it. Can I just do the listing instead of actual extract it? Can I say confirm folder integrity if the listing is sucessful without problem? tar tvf file1.tar ... (1 Reply)
Discussion started by: vivien_chu
1 Replies

6. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

7. Shell Programming and Scripting

editing names of files in multiple folder

I have 1000's of directories which is named as numbers. Each directory contains multiple files. Each of these directories have a file named "att". I need to rename all the att files by adding the directory name followed by "_" then att for each of the directories. Directories 120 att... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

8. Shell Programming and Scripting

Script to move files with similar names to folder

I have in directory /media/AUDIO/WAVE many .mp3 files with names like: my filename_01of02.mp3 my filename_02of02.mp3 Your File_01of06.mp3 Your File_02of06.mp3 etc.... In the same directory, /media/AUDIO/WAVE, I have many folders with names like 9780743579490 9780743579491 etc.. Inside... (7 Replies)
Discussion started by: glev2005
7 Replies

9. Shell Programming and Scripting

lvm/tar/rsync backup script feedback/criticism

I have written a shell script to perform backups using tar, rsync and optionally utilise lvm snapshots. The script is not finished but is in a working state and comments/descriptions are poor. I would greatly appreciate any criticism and suggestions of the script to help improve my own learning... (0 Replies)
Discussion started by: jelloir
0 Replies

10. UNIX for Dummies Questions & Answers

Copying multiple folders to local machine (don't know folder names)

Hi. I'm trying to copy multiple folders from the remote machine to the local machine. I wrote a batch file to run an ftp window. The problem I am having is that the only command to copy files is mget *, and this copies only files, not folders. For example, ftp ts555 cd ts555/test ' test... (5 Replies)
Discussion started by: leenyburger
5 Replies
Login or Register to Ask a Question