Sponsored Content
Top Forums Shell Programming and Scripting How to copy very large directory trees Post 302723251 by siegfried on Monday 29th of October 2012 06:51:34 PM
Old 10-29-2012
How to copy very large directory trees

I have constant trouble with XCOPY/s for multi-gigabyte transfers.
I need a utility like XCOPY/S that remembers where it left off if I reboot. Is there such a utility? How about a free utility (free as in free beer)?

How about an md5sum sanity check too?

I posted the above query in another forum. Answers to the above question are welcome here too. My question for this forum is: How feasible is it it write a script in groovy, python, cygwin bash, Activestate perl, powershell, etc... to implement a linux "cp -R" or windows "xcopy/s" program that can continue where it left off after a reboot?

I cannot think of clever way do do this easily -- can you?

I could write it in C++ -- but I'm hoping for something more quick and elegant.


Thanks
Siegfried
 

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy large dump file

Hi Experts.. Could anyone please let me know the easier way to copy large dump of files from one server to another. I am trying to copy a set of dump files on two different servers placed in different geographic locations.. Though there are other factors such as latency, etc., slowing up the... (4 Replies)
Discussion started by: ganga.dharan
4 Replies

2. Shell Programming and Scripting

Best way to diff two huge directory trees

Hi I have a job that will be running nightly incremental backsup of a large directory tree. I did the initial backup, now I want to write a script to verify that all the files were transferred correctly. I did something like this which works in principle on small trees: diff -r -q... (6 Replies)
Discussion started by: same1290
6 Replies

3. UNIX for Advanced & Expert Users

How to rsync or tar directory trees, with hidden directory, but without files?

I want to backup all the directory tress, including hidden directories, without copying any files. find . -type d gives the perfect list. When I tried tar, it won't work for me because it tars all the files. find . -type d | xargs tar -cvf a.tar So i tried rsync. On my own test box, the... (4 Replies)
Discussion started by: fld2007
4 Replies

4. Shell Programming and Scripting

how to copy the directory but not copy certain file

Hi experts cp bin root src /mnt but not copy bin/bigfile any help? ( I post this thread in the "redhat" forum wrongly, I don't know how to withdraw that question in that wrong forum) Thanks (6 Replies)
Discussion started by: yanglei_fage
6 Replies

5. UNIX for Dummies Questions & Answers

Is it better/possible to pause the rsyncing of a very large directory?

Possibly a dumb question, but I'm deciding how I'm going to do this. I'm currently rsyncing a 25TB directory (with several layers of sub directories most of which have video files ranging from 500 megs to 4-5 gigs), from one NAS to another using rsync -av. By the time I need to act ~15TB should... (3 Replies)
Discussion started by: DeCoTwc
3 Replies

6. Slackware

What is the medium usually used to backup large trees?

Hi: What's asked. (2 Replies)
Discussion started by: stf92
2 Replies

7. Shell Programming and Scripting

Copy the files in directory and sub folders as it is to another directory.

How to copy files from one directory to another directory with the subfolders copied. If i have folder1/sub1/sub2/* it needs to copy files to folder2/sub1/sub2/*. I do not want to create sub folders in folder2. Can copy command create them automatically? I tried cp -a and cp -R but did... (4 Replies)
Discussion started by: santosh2626
4 Replies

8. Shell Programming and Scripting

ksh - Checking directory trees containing wild cards

Hi Can somebody please show me how to check from within a KSH script if a directory exists on that same host when parts of the directory tree are unknown? If these wildcard dirs were the only dirs at that level then ... RETCODE=$(ls -l /u01/app/oracle/local/*/* | grep target_dir) ... will... (4 Replies)
Discussion started by: user052009
4 Replies
reboot(1M)                                                                                                                              reboot(1M)

NAME
reboot - restart the operating system SYNOPSIS
/usr/sbin/reboot [-dlnq] [boot_arguments] The reboot utility restarts the kernel. The kernel is loaded into memory by the PROM monitor, which transfers control to the loaded kernel. Although reboot can be run by the super-user at any time, shutdown(1M) is normally used first to warn all users logged in of the impending loss of service. See shutdown(1M) for details. The reboot utility performs a sync(1M) operation on the disks, and then a multi-user reboot is initiated. See init(1M) for details. On systems, reboot may also update the boot archive as needed to ensure a successful reboot. The reboot utility normally logs the reboot to the system log daemon, syslogd(1M), and places a shutdown record in the login accounting file /var/adm/wtmpx. These actions are inhibited if the -n or -q options are present. Normally, the system reboots itself at power-up or after crashes. The following options are supported: -d Force a system crash dump before rebooting. See dumpadm(1M) for information on configuring system crash dumps. -l Suppress sending a message to the system log daemon, syslogd(1M) about who executed reboot. -n Avoid calling sync(2) and do not log the reboot to syslogd(1M) or to /var/adm/wtmpx. The kernel still attempts to sync filesystems prior to reboot, except if the -d option is also present. If -d is used with -n, the kernel does not attempt to sync filesystems. -q Quick. Reboot quickly and ungracefully, without shutting down running processes first. The following operands are supported: boot_arguments An optional boot_arguments specifies arguments to the uadmin(2) function that are passed to the boot program and kernel upon restart. The form and list of arguments is described in the boot(1M) and kernel(1M) man pages.. If the arguments are specified, whitespace between them is replaced by single spaces unless the whitespace is quoted for the shell. If the boot_arguments begin with a hyphen, they must be preceded by the -- delimiter (two hyphens) to denote the end of the reboot argument list. Example 1: Passing the -r and -v Arguments to boot In the following example, the delimiter -- (two hyphens) must be used to separate the options of reboot from the arguments of boot(1M). example# reboot -dl -- -rv Example 2: Rebooting Using a Specific Disk and Kernel The following example reboots using a specific disk and kernel. example# reboot disk1 kernel.test/unix /var/adm/wtmpx login accounting file See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ mdb(1), boot(1M), dumpadm(1M), fsck(1M), halt(1M), init(1M), kernel(1M), shutdown(1M), sync(1M), syslogd(1M), sync(2), uadmin(2), reboot(3C), attributes(5) The reboot utility does not execute the scripts in /etc/rcnum.d or execute shutdown actions in inittab(4). To ensure a complete shutdown of system services, use shutdown(1M) or init(1M) to reboot a Solaris system. 11 Apr 2005 reboot(1M)
All times are GMT -4. The time now is 02:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy