Sponsored Content
Top Forums Shell Programming and Scripting Using rsync --link-dest pointing to a remote server Post 302824807 by Orionizer on Friday 21st of June 2013 02:52:16 PM
Old 06-21-2013
Using rsync --link-dest pointing to a remote server

Ok, I'm trying to figure out how to reference a remote file using the rsync --link-dest parameter.

Here is the code I have so far:

Code:
#
# FILESERVER INCREMENTAL BACKUP SCRIPT
#

# Remove previous log file

rm /usr/local/bin/rsync-incremental.log

# Set daily variables

DAY0=`date -I`
DAY1=`date -I -d "1 day ago"`
DAY6=`date -I -d "2 days ago"`
DAY7=`date -I -d "3 days ago"`
DAY8=`date -I -d "4 days ago"`

# Set rsync variables

SRC="/usr/smbshares"
TRG="root@synology1:/volume1/Fileserver/$DAY0"
LNK="root@synology1:/volume1/Fileserver/$DAY1"
OPT="-avhz -e ssh --log-file=/usr/local/bin/rsync-incremental.log --delete --link-dest=$LNK"

# Remove old backups

ssh root@synology1 "if [[ -d /volume1/Fileserver/$DAY6 ]]; then nohup rm -rf "/volume1/Fileserver/$DAY6"; fi"

ssh root@synology1 "if [[ -d /volume1/Fileserver/$DAY7 ]]; then nohup rm -rf "/volume1/Fileserver/$DAY7"; fi"

ssh root@synology1 "if [[ -d /volume1/Fileserver/$DAY8 ]]; then nohup rm -rf "/volume1/Fileserver/$DAY8"; fi"

# Perform the rsync

rsync $OPT $SRC $TRG

Everything is working now except the --link-dest parameter. I get the following error at the beginning of execution:

--link-dest arg does not exist: root@synology1:/volume1/Fileserver/2013-06-20

What's funny is the rsync has no issue using the same syntax for the target folder of the command...

Anyone know how to reference the remote file for this?

Thanks!

---------- Post updated at 01:52 PM ---------- Previous update was at 01:31 PM ----------

If anyone is wondering, I'm trying to accomplish backing up a SLES Samba server to a remote Synology NAS. I want to keep 5 days worth of backups on the NAS, but only the original or changed files. Not backing up ALL files all five days.

If a backup folder exists that is 6, 7 or 8 days old, I want to remove it to keep files from accumulating on the NAS over time.

I know the $DAY6, etc. variables are wrong as I changed them for testing purposes.

So that's what I'm trying to accomplish. If you can think of any better way to accomplish this, I'd be glad to give that a shot as well, but from what I've read it seems rsync is pretty reliable and quick.

As it's working right now, I am getting the 5 days worth of backups, but it's copying every file every time as the --link-dest isn't working.
 

10 More Discussions You Might Find Interesting

1. Solaris

creating link for a file of remote server

Hi, I use to access a file on remote server. Can I create a link for this file present on remote server on my local server, if yes then please let me know. Thanx in advance. (7 Replies)
Discussion started by: sanjay1979
7 Replies

2. Shell Programming and Scripting

How to write a shell script for rsync to remote server?

Hello, Can you help me to write a shell script for rsync a folder from my server to remote server ? i do this in ssh by this command : rsync -avz -e ssh /copy/me/ login@serverip:/where/to i want have a shell script that do this command. and you know that this command ask remote... (0 Replies)
Discussion started by: Desperados
0 Replies

3. UNIX for Dummies Questions & Answers

rsync with --link-dest doesn't create hard links

I have been experimenting with rsync as a scriptable backup option, reading various example and tips pages, including Time Machine for every Unix out there - IMHO That page seems to describe the exact behavior I want: The ability to make a "full backup" snapshot regularly, but with rsync... (0 Replies)
Discussion started by: fitzwilliam
0 Replies

4. UNIX for Dummies Questions & Answers

To know the server which the production is pointing to?

Hi, How to know which server(Application or webserver) the production link or url is pointing to? Is there any command to get the server IP address? Thanks in advance. (3 Replies)
Discussion started by: venkatesht
3 Replies

5. AIX

How to create a sym link pointing a changing name.

Hello A new file is created every day with the date appended to the end of a name. We are using Autosys to run jobs which watches for the file and runs jobs. But Autosys does not have the capability to figure out the current date. I tried creating a symlink like this ln -s... (1 Reply)
Discussion started by: vra5107
1 Replies

6. UNIX for Advanced & Expert Users

Using remote rsync, but copy locally?

I'm looking to use rsync to compare remote files and to copy the diff to a local directory, rather than transfer over the net. The net connection is not fast enough to transfer these files (~1.8TB) and I'd like to sneakernet them instead. Possible? (4 Replies)
Discussion started by: dfbills
4 Replies

7. Solaris

rsync - remote connection error

Hi , We have installed rsync in two Solaris boxes, when we try to sync files from one machine to another.. it is giving the following error. ld.so.1: rsync: fatal: libiconv.so.2: open failed: No such file or directory rsync: connection unexpectedly closed (0 bytes received so far) rsync... (1 Reply)
Discussion started by: MVEERA
1 Replies

8. UNIX for Dummies Questions & Answers

Help with rsync using link-dest

Hi, I'm trying to keep backups of my home directory (i.e. back it up every day or so) using rsync with link-dest. This is how I do it, rsync -avh --delete --link-dest=/media/disk/oldbackup/ /home/user/ /media/disk/newbackup rm -f /media/disk/oldbackup ln -s newbackup... (1 Reply)
Discussion started by: lost.identity
1 Replies

9. Shell Programming and Scripting

Sudo connect to a remote server and execute scripts in remote server

Hello Every one!! I am trying to write a shell script which will connect to a remote server and execute scripts which are at a certain path in the remote server. Before this I am using a sudo command to change the user. The place where I am stuck is, I am able to connect to the... (6 Replies)
Discussion started by: masubram
6 Replies

10. Shell Programming and Scripting

Script connect to remote server, not find files and exit only from remote server, but not from scrip

I have a script, which connecting to remote server and first checks, if the files are there by timestamp. If not I want the script exit without error. Below is a code TARFILE=${NAME}.tar TARGZFILE=${NAME}.tar.gz ssh ${DESTSERVNAME} 'cd /export/home/iciprod/download/let/monthly;... (3 Replies)
Discussion started by: digioleg54
3 Replies
RDUP-BACKUPS(7) 						       rdup							   RDUP-BACKUPS(7)

NAME
rdup-backups - introduction into making backups with rdup INTRODUCTION
rdup is a simple program that prints out a list of files and directories that are changed changed on a filesystem. It is more sophisticated than for instance find, because rdup will find files that are removed or directories that are renamed. A long time ago rdup included a bunch of shell and Perl scripts that implemented a backup policy. These could be used in a pipeline to per- form a backup. Currently rdup consists out of three basic utilities: rdup With rdup you create the file list on which later programs in the pipeline can work. The default output format also includes the files' content. rdup can be seen as a tar replacement in this respect, but rdup also allows for all kinds of transformations of the content (encryption, compression, reversal), see the -P switch in rdup(1) for more information. rdup-tr With rdup-tr you can transform the files rdup delivers to you. You can create tar, cpio or pax files. You can encrypt pathnames. rdup-tr is filter that reads from standard input and writes to standard output. See rdup-tr(1) for more information. With rdup and rdup-tr you can create an encrypted archive which is put in a directory structure that is also encrypted. rdup-up With rdup-up you can update an existing directory structure with the updates as described by rdup. rdup-up reads rdup input and will create the files, symbolic links, hard links and directories (and sockets, pipes and devices) in the file system. See rdup-up(1) for more information. So the general backup pipeline for rdup will look something like this: create filelist | transform | update filesystem ( rdup | rdup-tr | rdup-up ) Note 1: The same sequence is used for restoring. In both cases you want to move files from location A to B. The only difference is that the transformation is reversed when you restore. Note 2: The use of rdup-tr is optional. BACKUPS AND RESTORES
For rdup there is no difference between backups and restores. If you think about this for a minute you understand why. Making a backup means copying a list of files somewhere else. Restoring files is copying a list of files back to the place they came from. Same difference. So rdup can be used for both, if you did any transformation with rdup during the backup you just need to reverse those operations during the restore. BACKUPS
It is always best to backup to another medium, be it a different local harddisk or a NFS/CIFS mounted filesystem. You can also use ssh to store file on a remote server, ala rsync (although not as network efficient). If you backup to a local disk you can just as well use rsync or plain old tar, but if you store your files at somebody else's disk you will need encryption. This is where you go beyond rsync and rdup comes in. Rsync cannot do per-file encryption, sure you can encrypt the network traffic with ssh, but at the remote side your files are kept in plain view. If you implement remote backups, the easy route is to allow root access on the backup medium. If the backup runs without root access the created files will not have their original ownership. For NFS this can be achieved by using no_root_squash, for ssh you could enable PermitRootLogin. Note that this may be a security risk. SNAPSHOT BACKUPS
We need a little help here in the form of the rdup-simple script. Keep in mind that the following scripts can also be run remotely with the help of ssh. The following script implements the algorithm of rdup-simple. #!/bin/bash # some tmp files are saved in ~/.rdup. This directory must exist DIR=/home # what to backup BACKUP=/vol/backup TODAY=$(date +%Y%m/%d) LIST=~/.rdup/list-$HOSTNAME STAMP=~/.rdup/timestamp-$HOSTNAME # for remote backup, this has to run on the remote host! BUGBUG RET=$? case $RET in 2|*) echo Error >&2 exit 1 ;; 1) # full dump, remove file-list and time-stamp file rm $LIST $STAMP ;; 0) # inc dump # do nothing here ;; esac # this is the place where you want to modify the command line # right now, nothing is translated we just use 'cat' rdup -N $STAMP -Pcat $LIST $DIR | rdup-up $BACKUP/$HOSTNAME/$TODAY # or do a remote backup #rdup -N $STAMP -Pcat $LIST $DIR | ssh root@remotehost # rdup-up $BACKUP/$HOSTNAME/$TODAY LOCAL BACKUPS
With rdup-simple you can easily create backups. Backing up my home directory to a backup directory: rdup-simple ~ /vol/backup/$HOSTNAME This will create a backup in /vol/backup/$HOSTNAME/200705/15. So each day will have its own directory. Multiple sources are allowed, so: rdup-simple ~ /etc/ /var/lib /vol/backup/$HOSTNAME Will backup your home directory, /etc and /var/lib to the backup location. Also if you need to compress your backup, simple add a '-z' switch: rdup-simple -z ~ /etc/ /var/lib /vol/backup/$HOSTNAME REMOTE BACKUPS
For a remote backup to work, both the sending machine and the receiving machine must have rdup installed. The currently implemented proto- col is ssh. Dumping my homedir to the remote server: rdup-simple ~ ssh://miekg@remote/vol/backup/$HOSTNAME The syntax is almost identical, only the destination starts with the magic string 'ssh://'. Compression and encryption are just as easily enabled as with a local backup, just add '-z' and/or a '-k keyfile' argument: rdup-simple -z -k 'secret-file' ~ ssh://miekg@remote/vol/backup/$HOSTNAME Remember though, that because of these advanced features (compression, encryption, etc, ...) the network transfer can never be as efficient as rsync. ALSO SEE
rdup(1), rdup-tr(1), rdup-up(1) and http://www.miek.nl/projects/rdup/ 1.1.x 15 Dec 2008 RDUP-BACKUPS(7)
All times are GMT -4. The time now is 09:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy