Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Backup solution using rsync

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Backup solution using rsync
# 1  
Old 02-12-2014
Backup solution using rsync
Hello All,

I am looking at a fast way to script some backups. I am looking at using rsync to do the leg work. I am having a hard time conceiving a script though.

I have a tree with subfolders within subfolders. I was looking at the /xd option to parse the tree.

Code:
 
 Directory of k:\

02/11/2014  09:12 AM    <DIR>          Folder1
-----------------------------------------------------------
02/12/2014  02:36 PM                 0 tree.txt
               3 File(s)  1,759,431,185 bytes

 Directory of k:\Folder1

02/11/2014  09:12 AM    <DIR>          .
02/11/2014  09:12 AM    <DIR>          ..
09/27/2013  05:39 PM    <DIR>          BS210
11/12/2013  07:51 PM    <DIR>          BS212
11/07/2013  03:40 PM    <DIR>          BS213
12/18/2013  03:08 PM    <DIR>          BS214
01/10/2014  02:24 PM    <DIR>          BS215
01/24/2014  09:41 AM    <DIR>          BS216
02/04/2014  12:31 PM    <DIR>          BS217
02/11/2014  09:13 AM    <DIR>          BS218
12/18/2013  04:07 PM    <DIR>          Ep.204 W Fix
               2 File(s)    813,871,852 bytes


 Directory of k:\Buying and Selling 2\BS212

11/12/2013  07:51 PM    <DIR>          .
11/12/2013  07:51 PM    <DIR>          ..
10/28/2013  02:30 PM    <DIR>          CR
11/12/2013  07:51 PM    <DIR>          HGTV
10/28/2013  02:17 PM    <DIR>          W
               2 File(s)    662,412,651 bytes

 Directory of k:\Buying and Selling 2\BS212\CR

10/28/2013  02:30 PM    <DIR>          .
10/28/2013  02:30 PM    <DIR>          ..
10/28/2013  02:29 PM    <DIR>          5.1
10/28/2013  02:46 PM    <DIR>          Stereo
               0 File(s)              0 bytes

 Directory of k:\Buying and Selling 2\BS212\CR\5.1

10/28/2013  02:29 PM    <DIR>          .
10/28/2013  02:29 PM    <DIR>          ..
10/25/2013  04:01 PM       391,257,934 BS2012 CR Full Mix LtRt_L.wav
10/25/2013  04:01 PM       391,257,934 BS2012 CR Full Mix LtRt_R.wav
10/25/2013  04:08 PM       391,257,934 BS2012 CR MED LtRt_L.wav
10/25/2013  04:08 PM       391,257,934 BS2012 CR MED LtRt_R.wav
10/25/2013  03:13 PM       260,839,874 BS2012_ CR_English Mix.C.wav
10/25/2013  03:13 PM       260,839,874 BS2012_ CR_English Mix.L.wav
10/25/2013  03:13 PM       260,839,876 BS2012_ CR_English Mix.Lfe.wav
10/25/2013  03:13 PM       260,839,876 BS2012_ CR_English Mix.Ls.wav
10/25/2013  03:13 PM       260,839,874 BS2012_ CR_English Mix.R.wav
10/25/2013  03:13 PM       260,839,876 BS2012_ CR_English Mix.Rs.wav
              10 File(s)  3,130,070,986 bytes

 Directory of k:\Buying and Selling 2\BS212\CR\Stereo

10/28/2013  02:46 PM    <DIR>          .
10/28/2013  02:46 PM    <DIR>          ..
10/25/2013  04:01 PM       391,257,934 BS2012 CR Full Mix LtRt_L.wav
10/25/2013  04:01 PM       391,257,934 BS2012 CR Full Mix LtRt_R.wav
10/25/2013  04:08 PM       391,257,934 BS2012 CR MED LtRt_L.wav
10/25/2013  04:08 PM       391,257,934 BS2012 CR MED LtRt_R.wav
10/25/2013  03:13 PM       260,839,990 BS2012_ CR Dialogue Mix.M.wav
10/25/2013  03:13 PM       260,839,880 BS2012_ CR Stereo Music Mix.L.wav
10/25/2013  03:13 PM       260,839,880 BS2012_ CR Stereo Music Mix.R.wav
10/25/2013  03:13 PM       260,839,878 BS2012_ CR Stereo SFX Mix.L.wav
10/25/2013  03:13 PM       260,839,878 BS2012_ CR Stereo SFX Mix.R.wav
10/25/2013  03:13 PM       260,839,984 BS2012_ CR Vo Mix.M.wav
              10 File(s)  3,130,071,226 bytes

 Directory of k:\Buying and Selling 2\BS212\HGTV

11/12/2013  07:51 PM    <DIR>          .
11/12/2013  07:51 PM    <DIR>          ..
10/25/2013  03:44 PM       397,446,771 BS2012 HGTV Full Mix LtRt_L.wav
10/25/2013  03:44 PM       397,446,771 BS2012 HGTV Full Mix LtRt_R.wav
10/25/2013  02:50 PM       264,965,886 BS2012_HGTV_Dialogue Mix.M.wav
10/25/2013  02:50 PM       264,965,776 BS2012_HGTV_Stereo Music Mix.L.wav
10/25/2013  02:50 PM       264,965,776 BS2012_HGTV_Stereo Music Mix.R.wav
10/25/2013  02:50 PM       264,965,774 BS2012_HGTV_Stereo SFX Mix.L.wav
10/25/2013  02:50 PM       264,965,774 BS2012_HGTV_Stereo SFX Mix.R.wav
10/25/2013  02:50 PM       264,965,880 BS2012_HGTV_Vo Mix.M.wav
               8 File(s)  2,384,688,408 bytes

 Directory of k:\Buying and Selling 2\BS212\W

10/28/2013  02:17 PM    <DIR>          .
10/28/2013  02:17 PM    <DIR>          ..
10/28/2013  02:17 PM    <DIR>          5.1
10/28/2013  11:54 AM    <DIR>          BS2012 W DOLBY
10/28/2013  02:25 PM    <DIR>          Stereo
               0 File(s)              0 bytes

 Directory of k:\Buying and Selling 2\BS212\W\5.1

10/28/2013  02:17 PM    <DIR>          .
10/28/2013  02:17 PM    <DIR>          ..
10/25/2013  03:25 PM       406,948,500 BS2012 W Full Mix LtRt_L.wav
10/25/2013  03:25 PM       406,948,500 BS2012 W Full Mix LtRt_R.wav
10/25/2013  03:33 PM       406,948,500 BS2012 W ME LtRt_L.wav
10/25/2013  03:33 PM       406,948,500 BS2012 W ME LtRt_R.wav
10/25/2013  02:18 PM       271,300,254 BS2012_W_English Mix.C.wav
10/25/2013  02:18 PM       271,300,254 BS2012_W_English Mix.L.wav
10/25/2013  02:18 PM       271,300,256 BS2012_W_English Mix.Lfe.wav
10/25/2013  02:18 PM       271,300,256 BS2012_W_English Mix.Ls.wav
10/25/2013  02:18 PM       271,300,256 BS2012_W_English Mix.R.wav
10/25/2013  02:18 PM       271,300,256 BS2012_W_English Mix.Rs.wav
              10 File(s)  3,255,595,532 bytes

 Directory of k:\Buying and Selling 2\BS212\W\BS2012 W DOLBY

10/28/2013  11:54 AM    <DIR>          .
10/28/2013  11:54 AM    <DIR>          ..
10/25/2013  04:18 PM       406,947,995 BS2012 W DolbyE.L.wav
10/25/2013  04:18 PM       406,947,995 BS2012 W DolbyE.R.wav
10/25/2013  04:18 PM             3,121 BS2012 W DolbyE.xml
               3 File(s)    813,899,111 bytes

 Directory of k:\Buying and Selling 2\BS212\W\Stereo

10/28/2013  02:25 PM    <DIR>          .
10/28/2013  02:25 PM    <DIR>          ..
10/25/2013  03:25 PM       406,948,500 BS2012 W Full Mix LtRt_L.wav
10/25/2013  03:25 PM       406,948,500 BS2012 W Full Mix LtRt_R.wav
10/25/2013  02:18 PM       271,300,370 BS2012_W_Dialogue Mix.M.wav
10/25/2013  02:18 PM       271,300,260 BS2012_W_Stereo Music Mix.L.wav
10/25/2013  02:18 PM       271,300,260 BS2012_W_Stereo Music Mix.R.wav
10/25/2013  02:18 PM       271,300,258 BS2012_W_Stereo SFX Mix.L.wav
10/25/2013  02:18 PM       271,300,258 BS2012_W_Stereo SFX Mix.R.wav
10/25/2013  02:18 PM       271,300,364 BS2012_W_Vo Mix.M.wav
               8 File(s)  2,441,698,770 bytes

 Directory of k:\Buying and Selling 2\BS213

11/07/2013  03:40 PM    <DIR>          .
11/07/2013  03:40 PM    <DIR>          ..
11/11/2013  06:07 PM    <DIR>          CR
11/11/2013  02:57 PM    <DIR>          HGTV
11/11/2013  02:38 PM    <DIR>          W
               0 File(s)              0 bytes


            1643 File(s) 505,047,510,138 bytes
             792 Dir(s)  494,982,270,976 bytes free

Is there a good way to limit the recursive depth of rsync so that I can have a zip file for the contents of the folders inside the root directory folders? I want to keep things separate. I know it's really me just being finicky, the other method I did this was with winrar and archiving to specific folders using the contextual right click to zip in a specified target. I suppose robocopy could be used instead of rsync but it doesn't include compression.

I was hoping there was a better way to do this as I have time at work to open my mind and try new things. I just need someone to point me in the general direction. I suppose /xf could also be used for certain patterns. I just don't want everything gathered and dumped in one place.

Thanks so much!
# 2  
Old 02-12-2014
are you on windows?
# 3  
Old 02-12-2014
Hey blackrageous

I am on windows I have tried cygwin to use rsync and have come to the conclusion it is not the best way to go. Using an exclusion file and
Code:
--exclude-from

I can parse the tree so that I exclude two of the three subfolders. The collected files are not zipped though. Afterwards I could zip the resultant directory but I was looking for something a little more 'door to door'. I am now looking at the zip command but arriving at the same conclusion that it will just lump these files together. I don't know how I would script the rsync to be proceeded by a zip command. I guess that's not all that bad.
Do you have any suggestions? I'd love to learn more about scripting really...
# 4  
Old 02-12-2014
tar and gzip (or bzip2)

tar provides exclude options (it creates a single file that will be the same size as all the data you backup approximately, then you compress it

here an example of tar and compress in one command (alternately you can pipe to gzip or bzip2).

Code:
tar -cvzf mystuff.tar.gz <stuff-to-backup>

I offer this as a starting point, you should see the manpages for tar and gzip (and how you would extract files back).
# 5  
Old 02-12-2014
Pipe
Hey thanks so much for your input. I have thought about pipes but it is really hard for me to conceive. I know the output of one command can be piped to another but that is about it. The manual for piping is really dense. Can you explain it or share a link with some insight? Thanks so much!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rsync better use for backup

Hello, I have a list of working directory in a remote computer acesssible through ssh and the same directory structure in my home directory of my laptop. I sometimes work on both my laptop and my this other computer. I usually use Rsync this way to synchronize files rsync... (5 Replies)
Discussion started by: ajayram
5 Replies

2. Shell Programming and Scripting

rsync backup mode(--backup) Are there any options to remove backup folders on successful deployment?

Hi Everyone, we are running rsync with --backup mode, Are there any rsync options to remove backup folders on successful deployment? Thanks in adv. (0 Replies)
Discussion started by: MVEERA
0 Replies

3. Shell Programming and Scripting

Rsync backup

How do i use Rsync yo pickup only new or modified files from source? I am using rsync -ravzpotu --delete-excluded but sometimes it goes thru all files again (5 Replies)
Discussion started by: sprool
5 Replies

4. Linux

Backup solution

Hi all, i am not sure whether i selected the right topic or not, so excuse me, well i am mainly a Windows admin, but i do *NIX administration from time to time, for now i need to use an open source solution for backup windows environment mainly, i spent last days playing with bacula and backupPC,... (0 Replies)
Discussion started by: XP_2600
0 Replies

5. AIX

Offsite Backup Solution

I'm looking for a program or some way to backup my server to another location onto another server. Does anyone have any ideas? What utility, server O/S(I would like to use Linux RH or CentOS)? Whats the best way to do this? I am fairly new with Unix. Thanks in advance (6 Replies)
Discussion started by: ITAdmin08
6 Replies

6. AIX

Backup solution

Hello, I'm looking for a backup solution for my system. I have 3 AIX virtual partitions running on a IBM p5 server. Each partition has it's data on a DS4700 storage server. Also, I have a RedHat running on an IBM p720. This server has the tape hardware. I would like to know if I can backup from... (4 Replies)
Discussion started by: enzote
4 Replies

7. UNIX for Dummies Questions & Answers

Problems using rsync in Leopard with SSH... seeking solution/alternative

Precursor: ...it may sound like I'm at least somewhat comfortable with Unix, but I'm far from it. Layman's terms would be greatly appreciated :) Okay, so here's the situation. I created a website using iWeb, and I'm not using a .Mac account (you can sync website in one click if you do). Being... (4 Replies)
Discussion started by: compulsiveguile
4 Replies

8. Solaris

Sun and backup solution

My company is in the process of building a new datacenter and I am responsible for setting up the backup environment (everything from racking to implementation). I guess it is pretty basic but I don't have much experience with the initial setup. What we have so far is a SunV890 (backup server), Sun... (1 Reply)
Discussion started by: Jshwon
1 Replies

9. UNIX for Advanced & Expert Users

enterprise backup solution

We are using sun_sparc solaris 2.5.1 with oracle database 8.0.5 We are considering to buy for buying backup software for this purpose . Our systems integrator says veritas could not be installed due to some technical reasons (they are veritas authorised dealer) . And is suggesting Netvault... (3 Replies)
Discussion started by: Hitesh Shah
3 Replies
Login or Register to Ask a Question