Sponsored Content
Top Forums Shell Programming and Scripting Script to tar/rsync/rm multiple folder names Post 302970937 by bakunin on Wednesday 13th of April 2016 04:00:00 PM
Old 04-13-2016
Quote:
Originally Posted by robertkwild
getting there

Code:
#!/bin/bash
cd /to_be_archived/
for DIR in * ; do
fSaveDir="${DIR##*/}"
tar -cf "${fSaveDir}".tar "${fSaveDir}"
rsync -a "${fSaveDir}".tar /archived_projects/
rm -f "${fSaveDir}".tar
rm -rf "${fSaveDir}"
done

Excellent!

Just a few points you might want to observe as you go along:

First, it is common style to indent code: every conditional statement and every looping statement triggers one (more) level of indentation. The reason is that it lets stand out the loop body and the conditionally executed statements. Software (any software - even 5-line-scripts) is written to be maintained easily and this helps getting faster what code does. Suppose you write your script, don't look at it for some months and then want to change something: you won't have its "inner organisation" as present as you have it now.

Another thing is: like you organize a longer text into paragraphs to make it easier to read you can put empty lines into the code to group parts of the commands. I would have written your script this way:

Code:
#!/bin/bash

cd /to_be_archived/
for DIR in * ; do
     fSaveDir="${DIR##*/}"

     tar -cf "${fSaveDir}".tar "${fSaveDir}"
     rsync -a "${fSaveDir}".tar /archived_projects/
     rm -f "${fSaveDir}".tar
     rm -rf "${fSaveDir}"
done

Second: "tar" is a command which takes subcommands. "c" and "f" are such subcommands, not options. They are therefore NOT introduced with a dash. Yes, this is inconsequent and, yes, most tar-versions tolerate the dashes anyway because the misuse is so common, but still: correct is this:

Code:
tar cf /target/file.tar /directory/to/process

and not this:

Code:
tar -cf /target/file.tar /directory/to/process

Finally: your script relies on a certain environment being set. enter env at the commandline and you will see many variables being set to certain values. All these variables are set also inside your script when it runs. You rely, for instance, on a certain value of "PATH", because tar is usually /usr/bin/tar. This implicitly set environment may not always be there, especially when you put this script into cron to have it executed automatically from time to time. Do yourself a favour and set the environment you need explicitly:

Code:
#!/bin/bash

PATH="/usr/bin:/usr/local/bin:<or whatever else you may need>"
export PATH
<other variables you may need inside your script>

cd /to_be_archived/
for DIR in * ; do
     fSaveDir="${DIR##*/}"

     tar -cf "${fSaveDir}".tar "${fSaveDir}"
     rsync -a "${fSaveDir}".tar /archived_projects/
     rm -f "${fSaveDir}".tar
     rm -rf "${fSaveDir}"
done

Quote:
Originally Posted by robertkwild
how did you know this command

fSaveDir="${DIR##*/}"?
It is called "parameter expansion" or "variable expansion" and you will find it (like most other things i touched upon) in the man page of your shell. Enter man bash at the command prompt or do a google search.

This specific expansion is "##" which means: take the pattern following after that and remove the left part of the variables content matching the pattern: because "*/" matches "everything up to the last "/" - which is everything save for the directories name. There is also a method for cutting from the rightmost part of the content, see below:

Try the following at the commandline:

Code:
x="a/b/c/d/e"            # our content to play with
echo "${x##*/}"         # gives you "e" - the longest possible match for "*/" is cut off leftside
echo "${x#*/}"         # gives you "b/c/d/e", the shortest possible match
echo "${x%/*}"         # gives you "a/b/c/d", the shortest possible match for "/*" is cut off rightside
echo "${x%%/*}"        # will give you "a", the longest possible match for "/*"

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Copying multiple folders to local machine (don't know folder names)

Hi. I'm trying to copy multiple folders from the remote machine to the local machine. I wrote a batch file to run an ftp window. The problem I am having is that the only command to copy files is mget *, and this copies only files, not folders. For example, ftp ts555 cd ts555/test ' test... (5 Replies)
Discussion started by: leenyburger
5 Replies

2. Shell Programming and Scripting

lvm/tar/rsync backup script feedback/criticism

I have written a shell script to perform backups using tar, rsync and optionally utilise lvm snapshots. The script is not finished but is in a working state and comments/descriptions are poor. I would greatly appreciate any criticism and suggestions of the script to help improve my own learning... (0 Replies)
Discussion started by: jelloir
0 Replies

3. Shell Programming and Scripting

Script to move files with similar names to folder

I have in directory /media/AUDIO/WAVE many .mp3 files with names like: my filename_01of02.mp3 my filename_02of02.mp3 Your File_01of06.mp3 Your File_02of06.mp3 etc.... In the same directory, /media/AUDIO/WAVE, I have many folders with names like 9780743579490 9780743579491 etc.. Inside... (7 Replies)
Discussion started by: glev2005
7 Replies

4. Shell Programming and Scripting

editing names of files in multiple folder

I have 1000's of directories which is named as numbers. Each directory contains multiple files. Each of these directories have a file named "att". I need to rename all the att files by adding the directory name followed by "_" then att for each of the directories. Directories 120 att... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

5. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

6. UNIX for Dummies Questions & Answers

Do I need to extract the entire tar file to confirm the tar folder is fine?

I would like to confirm my file.tar is been tar-ed correctly before I remove them. But I have very limited disc space to untar it. Can I just do the listing instead of actual extract it? Can I say confirm folder integrity if the listing is sucessful without problem? tar tvf file1.tar ... (1 Reply)
Discussion started by: vivien_chu
1 Replies

7. Shell Programming and Scripting

Script to move one folder to multiple folder...

Hi All, I have to requirement to write a shell script to move file from one folder (A) to another five folder (B,C,D,E,F) and destination folder should be blank. In not blank just skip. This script will run as a scheduler every 2 minutes. It will check number of files in folder A and move 1 to... (9 Replies)
Discussion started by: peekuabc
9 Replies

8. Shell Programming and Scripting

Bash script for printing folder names and their sizes

Good day, everyone! I'm very new to bash scripting. Our teacher gave us a task to create a script that basically does the same job the 'du' command does, with the difference that 'du' command gives an output in the form of <size> <folder name>and what we need is <folder name> <size>As for... (1 Reply)
Discussion started by: UncleIS
1 Replies

9. Homework & Coursework Questions

Bash script for printing folder names and their sizes

1. The problem statement, all variables and given/known data: The task is to create a script that would reproduce the output of 'du' command, but in a different way: what 'du' does is: <size> <folder name>and what is needed is <folder name> <size>We need to show only 10 folders which are the... (3 Replies)
Discussion started by: UncleIS
3 Replies

10. Shell Programming and Scripting

Checking Multiple File existance in a UNIX folder(Note: File names are all different)

HI Guys, I have some 8 files with different name and extensions. I need to check if they are present in a specific folder or not and also want that script to show me which all are not present. I can write if condition for each file but from a developer perspective , i feel that is not a good... (3 Replies)
Discussion started by: shankarpanda003
3 Replies
All times are GMT -4. The time now is 01:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy