Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

rsync incremental question

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users rsync incremental question
# 1  
Old 05-18-2007
rsync incremental question
Hi, I have problem with incremental copy of files from source to destination.
I would like to copy different sources to different destinations. I made a backup script which look like that:

Code:
curd=$(date +%w)
day=$(date +%A)

mkdir -p /home/backup/DATA/$day

case $curd in
1|2|3|4|5)
            rsync -rP --ignore-errors --force /home/backup/Data-Finance /home/backup/DATA/$day;
            rsync -rP --ignore-errors --force /home/backup/data /home/backup/DATA/$day;
            rsync -rP --ignore-errors --force /home/backup/data-tb /home/backup/DATA/$day;;
6)
            rsync -rWP --delete --ignore-errors --force /home/backup/Data-Finance /home/backup/DATA/$day;
            rsync -rWP --delete --ignore-errors --force /home/backup/data /home/backup/DATA/$day;
            rsync -rWP --delete --ignore-errors --force /home/backup/data-tb /home/backup/DATA/$day;;
*) exit

The problem is that I have source almost 400 GB.I want to back up only those files which was modify only on the current day without those are older.

When rsync start for first time it starts coping all files from src to dest.
Is there any option which to resolve this?

Thanks in advance!
Last edited by Yogesh Sawant; 12-10-2010 at 05:27 AM.. Reason: added code tags
# 2  
Old 05-18-2007
Hi zhivko.neychev (BG ? Smilie )
I don't see such option in rsync man, however, you may sort the files with "find" and "-ctime" option, and then use "--exclude=PATTERN" of rsync to move only them ? Just a wild suggestion.
# 3  
Old 05-18-2007
Thank you sisgate, I'll try it and I'll post the result. It's look logical.
Yes, I am from Bulgaria. Do you speak bulgarian Smilie?
Last edited by zhivko.neychev; 05-18-2007 at 12:11 PM..
# 4  
Old 05-18-2007
It is a great Idea but I have some problems:
1. I use -mtime 0 to became day-modified files.
2. I use rsync --include`find * mtime 0` /dest
the problem is that find back files such as: "My files" but bash understand only "My\ Files" - this is the problem. I can't use PATTERN.
# 5  
Old 05-21-2007
Yeah, i see... Can't you do something like (roughly) :
dig through the source files, determine which ones have space, and substitute the space with underscore " _ " for example ? I mean, prepare the source files for better pattern matching by avoiding the space. I'm not sure how feasible is that, it's up to you.
Offtopic : I speak bulgarian, it's my native language Smilie Have fun in unix.com Smilie
# 6  
Old 05-21-2007
You can use a pattern with find - find /path -name 'pattern'.log
where pattern contains metacharacters like ? and *

-mtime -1 finds files less than 24 hours old. If you have bash, are you on a GNU-based system (linux)? If so, try unison instead. It has more features.
# 7  
Old 05-27-2007
I resolved my problem yesterday. Thanks everybody who tried to help me.
This is my final script and it works fine for me:
Code:
#!/bin/bash

mount /dev/sda1 /home/backup/DATA/
cd /home/backup/src/

case $(date +%w) in

1|2|3|4|5)find * -mtime 0 > ../upd_list;rsync --progress --files-from=../upd_list /home/backup/src /home/backup/DATA/$(date +%A);;
        6)rsync -rWlP --delete --ignore-errors --force --exclude-from=../exclude /home/backup/src/Data-Finance /home/backup/src/data /home/backup/src/data-tb /home/backup/DATA/$(date +%A);;
        *)exit
esac

Last edited by Yogesh Sawant; 12-10-2010 at 05:27 AM.. Reason: added code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rsync Error: rsync: link_stat failed: No such file or directory (2)

I wish to copy all the files & folder under /web/Transfer_Files/data/ on mymac1 (Linux) to remote server mybank.intra.com (Solaris 10) /tmp/ location I am using Ansible tool synchronize module which triggers the unix rsync command as below:rsync --delay-updates -F --compress --archive --rsh=ssh... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. UNIX for Dummies Questions & Answers

Script Question (find, rsync)

Hello all, I'm using rsync to maintain two copies of a music library on my main workstation and a server running MPD. I'm wondering about using the script below as a basis for cron to update my MPD db if new files have been added or existing files modified: find -type f -iname 'FILENAME' -mmin... (9 Replies)
Discussion started by: pmicka
9 Replies

3. UNIX for Dummies Questions & Answers

rsync usage question

Hello folks; I'm using rsync on my Ubuntu servers to sync a remote folders on a remote machine to a local folders on a local machine. one thing i couldn't get to work is how to exclude folder. I know i'm suppose to use "exclude" argument but every time i do, i still see the excluded directory... (0 Replies)
Discussion started by: Katkota
0 Replies

4. Shell Programming and Scripting

rsync question...

hi all, i didnt know where to place the question on this site but seen as though it relates to a script i have done so here... my issue is this... i have 2 servers. On server 1 (master) I have a structure such as /home/test/folder1,/home/test/folder2 and /home/test/folder3. On server... (0 Replies)
Discussion started by: muay_tb
0 Replies

5. Shell Programming and Scripting

rsync question (regarding --delete)

Hi there Does anybody know of a way that i can, instead of issuing a --delete when syncing one directory to another, I can instead somehow receive a list of what would be deleted, but not actually delete it ? basically, people are occasionally putting files into one of synced folders, but... (2 Replies)
Discussion started by: rethink
2 Replies

6. UNIX for Dummies Questions & Answers

Simple Rsync question

I have a folder on a server that I update from different machines. So I think Rsync sounds like a good solution but I'm not sure how to make it a bidirectional check for updating, because if I work on the file on one of my client machines it could be newer than the server. (2 Replies)
Discussion started by: elbombillo
2 Replies

7. UNIX for Dummies Questions & Answers

Rsync question....

I have folders that sync over the network with an external software. I would like to know if rsync can be used to only sync permissions on files on the files. Remember I just want the permissions on the source to be copied over to the destination's already synced files. (2 Replies)
Discussion started by: elbombillo
2 Replies

8. UNIX for Advanced & Expert Users

rsync question

Hey there guys I'm trying to rsync up a subdirectory only. For example /Volumes/media/WebDocuments/2006/* but not /Volumes/media/WebDocuments/2007/* rsync -va /Volumes/media/WebDocuments/2006/ \ user@host:/backups/2006/ If I do it that way I get only the contents... (3 Replies)
Discussion started by: sidmuchrock
3 Replies

9. UNIX for Dummies Questions & Answers

rsync question

Hi All I am trying to rsync the directories between two servers. The source directory is huge (8 GB) and I have the need to take an offsite backup. The directory I am trying to rsync has 774 permissions and are owned by a user/ group. To startup I backed up the source directory and restored... (1 Reply)
Discussion started by: skotapal
1 Replies

10. IP Networking

rsync question?

Does anyone know if rsync will work without ssh or rsh? I am trying to use rsync to synchronize files on two servers, one a production server and the other a standbyserver. The problem is that I have't been given the ok to install openssh on our productions servers, so I am trying to make my job... (4 Replies)
Discussion started by: cstovall
4 Replies
Login or Register to Ask a Question