Sponsored Content
Top Forums UNIX for Advanced & Expert Users rsync: taking advantage of files in different directory other than destination Post 302192166 by wildkaarde on Tuesday 6th of May 2008 07:35:36 AM
Old 05-06-2008
Rsync - with a twist of lemon...

Let's low tech this for a minute and test this by doing the steps manually. I am *assuming* the dump file name is the same everyday, or it has simple enough naming convention that you can make a script to rename a file.

1) You already have rsync'd and you have a folder named 04-05-08 on the backup server. It's now time to get the 05-05-08 version, and magically you know the DB Server is done creating today's dumpfile....
2) On the backup server, cp -rp ./04-05-08 ./05-05-08
-- you now have a new folder to rsync into, and it has yesterday's version of the dump file itself, with yesterday's name and we've preserved the timestamps and permissions on the files.
3) If necessary, rename the ./05-05-08/dumpfile to match today's dump name.
3) start your rsync between the 05-05-08 folders.

To quote the rsync man page:
If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences. Your file already exists, so it will run update instead of a copy.

With any luck, the compare will take less time than an actual copy normally does. No matter what, 800MB is a lot to accomplish.


Important Question: Does your pgsql dump actually need to create a NEW 800MB file every day on the DB server, or can it just update the existing dumpfile?

If you can update the same "dump" instance, then I'd just start an rsync of the master dumpfile, and on the backup server just schedule a cp of the the currently rsync'd instance over to the daily folder. Schedule the rsync for every 3 hours, and you have a decent current backup, plus yesterday's copy on the shelf.

Please remember: If you are out of disk space, or your dumpfile failed to be created, it's really hard to finish a backup, so script with that in mind, or confirm something is monitoring your diskspace....

A backup is a terrible thing to lose.Smilie
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[BASH] rsync - error on destination

Hi everyone, and thanks to all for your assistance. I have a problem with the rsync command. I want to make a backup of a "source" directory in a "destination" directory. I want to specify: "absolute path of destination" (identified by a ~): ~/Destination or a "relative path of... (0 Replies)
Discussion started by: PaganoM
0 Replies

2. Shell Programming and Scripting

recursive searching for files in directory that matches a particular name - taking care of links

Hi, I am writing a shell script that finds all files named <myFile> in a directory <dir> or any of its subdirectories, recursively. I also need to take care of symbolic links that may form cycles, to avoid infinite loops. I started writing the code but got stuck. I thought using recursion... (7 Replies)
Discussion started by: vickylife
7 Replies

3. UNIX for Advanced & Expert Users

How to rsync or tar directory trees, with hidden directory, but without files?

I want to backup all the directory tress, including hidden directories, without copying any files. find . -type d gives the perfect list. When I tried tar, it won't work for me because it tars all the files. find . -type d | xargs tar -cvf a.tar So i tried rsync. On my own test box, the... (4 Replies)
Discussion started by: fld2007
4 Replies

4. Shell Programming and Scripting

Move all files from source to destination directory based on the filename

Move all files starting with a specific name to different directory. This shell script program should have three parameters File Name Source Directory Destination Directory User should be able to enter ‘AB_CD*' in file name parameter. In this case all the files starting with AB_CD will... (1 Reply)
Discussion started by: chetancrsp18
1 Replies

5. Shell Programming and Scripting

Rsync not deleting files on destination

I've got a rsync script that backups file from disk1 to disk3. This works great however if there are extra files on disk3 they do not get deleted by the sync. I'm syncing all folders from F-J rsync --progress -v --delete --delete-excluded -av --recursive /home/disk1/Files/*... (0 Replies)
Discussion started by: gmccarthy
0 Replies

6. UNIX for Advanced & Expert Users

Rsync error while running from destination to source

hi All, i have 2 server setup now for Rsync, i configured Rsync on both of the server and it worked well when i did run from source to destination. and while running back from destination to source it produced this error: bash-3.2$ ksh rsync_bravo_db.ksh usa0300uz1247.apps.mc.xerox.com... (0 Replies)
Discussion started by: lovelysethii
0 Replies

7. UNIX for Dummies Questions & Answers

Rsync not deleting destination files/folders

Hi All, I want delete all files from destination folder. I am trying below commands rsync -avFF --delete-after /home/vizion/source/ /home/vizion/destination/ I need to be delete all file/fodders from destination Any body have idea ? Please correct me. (3 Replies)
Discussion started by: Chenchireddy
3 Replies

8. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

9. UNIX for Advanced & Expert Users

Rsync - delete extra files in Destination without synchronising directories

I have a script that synchronises a directory to a DR server, but to improve the time, I actually use rsync to transfer files * in one batch and also * in another batch - both batches run from the same script and run in the background. My problem is that there isn't much space on the... (1 Reply)
Discussion started by: Catullus
1 Replies
sequel(1)						      General Commands Manual							 sequel(1)

NAME
sequel - The Database Toolkit for Ruby SYNOPSIS
sequel <uri|path> [options] DESCRIPTION
Sequel is a lightweight database access toolkit for Ruby. OPTIONS
-C, --copy-databases Copy one database to another -d, --dump-migration Print database migration to STDOUT -D, --dump-migration-same-db Print database migration to STDOUT without type translation -e, --env ENV Use environment config for database -l, --log logfile Log SQL statements to log file -L, --load-dir DIR Loads all *.rb under specified directory -m, --migrate-directory DIR Run the migrations in directory -M, --migrate-version VER Migrate the database to version given -N, --no-test-connection Do not test the connection -t, --trace Output the full backtrace if an exception is raised -?, -h, --help Show this message -v, --version Show version SEE ALSO
This program is shipped as part of the lisequel-ruby1.8 library package, you can check its corresponding documentation can be found in the libsquel-ruby-doc package. AUTHOR
This manual page was written by Deepak Tripathi <apenguinlinux@gmail.com>, based on the command-line output of this program, for the Debian GNU/Linux system (but may be freely used by others). May 25, 2010 sequel(1)
All times are GMT -4. The time now is 11:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy