01-26-2010
I would use a tool like rsync or unison that are optimized for this kind of use and making backups.
Try this:
rsync -avz /data4/dbx /backup/dbx
Last edited by jostber; 01-26-2010 at 06:47 PM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi
I need to split a huge file into multiple smaller files using split command.
After that i need to process each file in the back ground with sql loader .Sql loader is a utlity to load CSV files into oracle .
Check the status of each of these sqlloaders and then after sucessfull... (6 Replies)
Discussion started by: xiamin
6 Replies
2. UNIX for Advanced & Expert Users
hey i have to implement copy command in parallel in c language.
i dont know how to create a new directory in destination.
if anything u know related to this help me (1 Reply)
Discussion started by: rajsekhar28
1 Replies
3. Shell Programming and Scripting
Hi,
I have a function abc(). i am calling the function 9 times. it is working fine and completed the script execution 10 hours.
input files:
CUSTOMER_INFO_1111_12345.csv
CUSTOMER_INFO_1222_12345.csv
CUSTOMER_INFO_1322_12345.csv
CUSTOMER_INFO_1333_12345.csv
CUSTOMER_INFO_1151_12345.csv... (4 Replies)
Discussion started by: onesuri
4 Replies
4. Shell Programming and Scripting
Hi.
I'm trying to speed up an rsync command by running it in parallel. There's no real option for this other than if the files are in multiple directories (which they're not). And even then there's no way of knowing if rsync has succeeded as the process is running in the background .. and... (4 Replies)
Discussion started by: Big_Jeffrey
4 Replies
5. Shell Programming and Scripting
Following is the requirement
In FileA I have the content as follows.
1,2,3
111,222,333
1000,2000,3000
In FileB I have the content as follows.
4,5,6
444,555,666
4000,5000,6000
I need to read FileA and FileB parallely and create the FileC as follows.
1,2,3,4,5,6... (1 Reply)
Discussion started by: kmanivan82
1 Replies
6. Shell Programming and Scripting
Hi everyone.
I have a list of files like:
file001
file002
file003
.
.
.
.
file385
file386
file387
There are more files than above, but I hope you understand what I'm trying to do here.
Is there a way I can create a loop to compare:
file001 with file385
file002 with file386 (9 Replies)
Discussion started by: craigsky
9 Replies
7. Shell Programming and Scripting
The below will move all the files in the directory dir to the destination using parallel and create a log, however will not keep them in the directory. I have tried mkdir -p but that does not seem to work or at least I can not seem to get it (as it deletes others files when I use it). What is the... (2 Replies)
Discussion started by: cmccabe
2 Replies
8. Shell Programming and Scripting
Hello folks,
Can you please help me to solve the below concern.
I have a source server with 2 ports and have to copy the files from both the port to destination server simultaneously in my shell script.
How can I achieve that?
Source : x.x.x.x port -22
X.x.x.x port -2222
... (7 Replies)
Discussion started by: sadique.manzar
7 Replies
9. Shell Programming and Scripting
I have two directories called English and Hindi. Each directory contains the same number of files with the only difference being that in the case of the English Directory the tag is
.english
and in the Hindi one the tag is
.Hindi
The file may contain either a single text or more than one text... (7 Replies)
Discussion started by: gimley
7 Replies
10. UNIX for Beginners Questions & Answers
Hi all,
I am trying to paste thousands of files together into a matrix. Each file has only 1 column and all the files have the same number of rows (~27k rows). I tried
paste * > output as well as some other for loops
but the output only contains the columns from the 1st and last files. The... (8 Replies)
Discussion started by: notimenocall
8 Replies
LEARN ABOUT DEBIAN
parallel-rsync
PARALLEL-RSYNC(1) PARALLEL-RSYNC(1)
NAME
parallel-rsync - deploy files to listed hosts
SYNOPSIS
parallel-rsync [OPTIONS] -h hosts.txt local remote
DESCRIPTION
pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of
homogenously-configured hosts.
parallel-rsync deploy files files to all hosts you listed.
OPTIONS
-r --recursive
recusively copy directories (OPTIONAL)
-a ----archive
use rsync -a (archive mode) (OPTIONAL)
-z --compress
use rsync compression (OPTIONAL)
-h --hosts
hosts file (each line "host[:port] [user]")
-l --user
username (OPTIONAL)
-p --par
max number of parallel threads (OPTIONAL)
-o --outdir
output directory for stdout files (OPTIONAL)
-e --errdir
output directory for stderr files (OPTIONAL)
-t --timeout
timeout (secs) (-1 = no timeout) per host (OPTIONAL)
-O --options
SSH options (OPTIONAL)
-v --verbose
turn on warning and diagnostic messages (OPTIONAL)
EXAMPLE
# parallel-rsync -r -h hosts.txt -l irb2 foo /home/irb2/foo
ENVIRONMENT
All four programs take similar sets of options. All of these options can be set using the following environment variables:
o PSSH_HOSTS
o PSSH_USER
o PSSH_PAR
o PSSH_OUTDIR
o PSSH_VERBOSE
o PSSH_OPTIONS
SEE ALSO
parallel-ssh(1), parallel-scp(1), parallel-slurp(1), parallel-nuke(1), ssh(1), rsync(1)
AUTHOR
Brent N. Chun <bnc@theether.org>
COPYING
Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun
NOTES
1. bnc@theether.org
mailto:bnc@theether.org
03/30/2009 PARALLEL-RSYNC(1)