How to copy very large directory trees Post: 302729953

Sponsored Content

Top Forums Shell Programming and Scripting How to copy very large directory trees Post 302729953 by siegfried on Monday 12th of November 2012 02:51:06 AM

11-12-2012

Registered User

How does --partial work?

How do I get the restart feature?
I tried the little script at linux - Resume rsync over SSH after broken connection? - Super User
and it seemed to work fine on a small directory.

Do I need to install the rsync deamon to have the automatic restart feature?
I think not -- but I'm not sure.

Does the --partial flag mean that if we previously rebooted in the middle of a transfer and we are executing this again, it will continue where it left off in the source directory? After reading the documentation, it does not look to me like "--partial" refers to partially copied directories but rather partially copied files.

Is there a way to tell rsync to only copy the files in the source directory that have not been correctly copied to the destination yet (because we were previously interrupted with a reboot)?

Thanks!
Siegfried

siegfried

View Public Profile for siegfried

Find all posts by siegfried

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy large dump file

Hi Experts.. Could anyone please let me know the easier way to copy large dump of files from one server to another. I am trying to copy a set of dump files on two different servers placed in different geographic locations.. Though there are other factors such as latency, etc., slowing up the...

2. Shell Programming and Scripting

Best way to diff two huge directory trees

Hi I have a job that will be running nightly incremental backsup of a large directory tree. I did the initial backup, now I want to write a script to verify that all the files were transferred correctly. I did something like this which works in principle on small trees: diff -r -q...

3. UNIX for Advanced & Expert Users

How to rsync or tar directory trees, with hidden directory, but without files?

I want to backup all the directory tress, including hidden directories, without copying any files. find . -type d gives the perfect list. When I tried tar, it won't work for me because it tars all the files. find . -type d | xargs tar -cvf a.tar So i tried rsync. On my own test box, the...

4. Shell Programming and Scripting

how to copy the directory but not copy certain file

Hi experts cp bin root src /mnt but not copy bin/bigfile any help? ( I post this thread in the "redhat" forum wrongly, I don't know how to withdraw that question in that wrong forum) Thanks

5. UNIX for Dummies Questions & Answers

Is it better/possible to pause the rsyncing of a very large directory?

Possibly a dumb question, but I'm deciding how I'm going to do this. I'm currently rsyncing a 25TB directory (with several layers of sub directories most of which have video files ranging from 500 megs to 4-5 gigs), from one NAS to another using rsync -av. By the time I need to act ~15TB should...

6. Slackware

What is the medium usually used to backup large trees?

Hi: What's asked.

7. Shell Programming and Scripting

Copy the files in directory and sub folders as it is to another directory.

How to copy files from one directory to another directory with the subfolders copied. If i have folder1/sub1/sub2/* it needs to copy files to folder2/sub1/sub2/*. I do not want to create sub folders in folder2. Can copy command create them automatically? I tried cp -a and cp -R but did...

8. Shell Programming and Scripting

ksh - Checking directory trees containing wild cards

Hi Can somebody please show me how to check from within a KSH script if a directory exists on that same host when parts of the directory tree are unknown? If these wildcard dirs were the only dirs at that level then ... RETCODE=$(ls -l /u01/app/oracle/local/*/* | grep target_dir) ... will...

LEARN ABOUT DEBIAN

dtc-xen_migrate

dtc-xen_migrate(8)					      System Manager's Manual						dtc-xen_migrate(8)

NAME

       dtc-xen_migrate - migrate a VPS to another Xen server

SYNOPSIS

       LOCAL_VPS_ID DESTINATION_HOSTNAME [ DEST_VPS_ID ]

DESCRIPTION

       dtc-xen_migrate	This  shell  script  is a part of the dtc-xen package that is to be used by the dtc panel to manage a Xen VPS server. Note
       that you need rsync in both the source and the destination dom0.

       This script is used to move the content of a Xen virtual machine (VM) managed with dtc-xen to another Xen server of the same type.  The	VM
       destination will first be prepared, then the local VM will be shut down, then it's content will be transfered over using rsync.

       Note that this is NOT a live migration, and down time of approximatively the time needed for the rsync will occure.

PARAMETERS

       LOCAL_VPS_ID The DTC-Xen id of the VM, from 00 to 99.

       DESTINATION_HOSTNAME Destination Xen server hostname to migrate to.

       DEST_VPS_ID Destination DTC-Xen id of the VM, from 00 to 99.

EXAMPLE

       dtc-xen_migrate 19 alpha.example.com 21

       This will migrate the content of xen19 on the local machine to the destination server alpha.example.com using xen21 as VM name.

SEE ALSO

       dtc_reinstall_os(8), dtc_kill_vps_disk(8)

																dtc-xen_migrate(8)