In a directory I have several folders and I want to send just the sapdata1 to sapdata14 folders and their contents but not sapdataXX/.snapshot
the script is:
The first rsync seems to work, I can see all the folders at the destination.
The find works and finds just over 600 files, however the xargs command only seems to send 40 files and then stops, I thought the -P would spawn 40 processes at a time until all the files were sent . I'm probably missing something obvious but I just can't figure it out.
I should also mention that I'm sending about 15Tb worth of large files. Hence the send in parallel script.
Hi
I need to move multiple (say 10 files) from one location to another location. My selection would be like this...
ls -ltr *.arc | head ---> Need to move top 10 files with single command without iterating in loop. I know we can move files like this with find command but not sure if I can... (4 Replies)
I believe what is happening is rm is executing in the script on every directory and on failure of the first it stops although returns status 0.
find $HOME -name /directory/filename | xargs -l rm
This is the code I use but file remains. I am using sun solaris system which has way limited... (4 Replies)
Hi,
I have many files like below in my currect dir
test1.me
test2.me
test3.me
I wanted them to rename without .me extention
I tried below
find . -name "*.me" | xargs -i mv {} `echo {} | sed 's/\.me//'`
It thorws error that
cant mv ./test1.me to ./test1.me as they are... (6 Replies)
I need to apply mp3gain (album mode) to all mp3 files in a given directory. Each album is in its own directory under /media/data/music/albums for example:
/media/data/music/albums/foo
/media/data/music/albums/bar
/media/data/music/albums/more
What needs to happen is:
cd... (4 Replies)
Hi, I'd like to ask if anybody can help improve my code to move 1 million+ files from a directory to another:
find /source/dir -name file* -type f | xargs -I '{}' mv {} /destination/dir
I learned this line of code from this forum as well and it works fine. However, file movement is kinda... (6 Replies)
Hello Experts,
I need to copy files from one location to another using xargs.
Tried something like this (In Ubuntu & Solaris ).
mkdir -p 1234;
find /home/emd/Desktop/n007/M007/ -type f -name "A2014*" | xargs -0 cp -r {} /home/emd/Desktop/1234
But every time i run this, a weird error... (6 Replies)
Hello,
i have an dynamical apache_cache that I need to clean all days (older tant one day) with an unix command :
find /usr/IBM/HTTPServer/apache_cache/ -name '*' -mtime +1 -print0|xargs -0 rm -r --
but it didn't work.
Could you explain me why.
So I will put below all my script :... (13 Replies)
I want to run parallelise rsync with xargs. The plan is to
take separate directories and run rsync in parallel.
find . -mindepth 1 -maxdepth 1 print0 | xargs -n1 -P0 rsync -avz
Hoxever I am stuck on how to use rsync in this way
---------- Post updated at 04:44 AM ---------- Previous... (3 Replies)
Please, i have a question about rsync command:
Here is the command that i have used in my script:
rsync -ratlz --rsh="/usr/bin/sshpass ssh -o StrictHostKeyChecking=no" -aAXHltzh --progress --numeric-ids --devices --rsync-path="rsync" $1 $hostname:$1
using this command, i can... (0 Replies)
I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when
| xargs sh is added I get the error. I tried adding | xargs... (4 Replies)
Discussion started by: cmccabe
4 Replies
LEARN ABOUT DEBIAN
parallel-rsync
PARALLEL-RSYNC(1)PARALLEL-RSYNC(1)NAME
parallel-rsync - deploy files to listed hosts
SYNOPSIS
parallel-rsync [OPTIONS] -h hosts.txt local remote
DESCRIPTION
pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of
homogenously-configured hosts.
parallel-rsync deploy files files to all hosts you listed.
OPTIONS -r --recursive
recusively copy directories (OPTIONAL)
-a ----archive
use rsync -a (archive mode) (OPTIONAL)
-z --compress
use rsync compression (OPTIONAL)
-h --hosts
hosts file (each line "host[:port] [user]")
-l --user
username (OPTIONAL)
-p --par
max number of parallel threads (OPTIONAL)
-o --outdir
output directory for stdout files (OPTIONAL)
-e --errdir
output directory for stderr files (OPTIONAL)
-t --timeout
timeout (secs) (-1 = no timeout) per host (OPTIONAL)
-O --options
SSH options (OPTIONAL)
-v --verbose
turn on warning and diagnostic messages (OPTIONAL)
EXAMPLE
# parallel-rsync -r -h hosts.txt -l irb2 foo /home/irb2/foo
ENVIRONMENT
All four programs take similar sets of options. All of these options can be set using the following environment variables:
o PSSH_HOSTS
o PSSH_USER
o PSSH_PAR
o PSSH_OUTDIR
o PSSH_VERBOSE
o PSSH_OPTIONS
SEE ALSO parallel-ssh(1), parallel-scp(1), parallel-slurp(1), parallel-nuke(1), ssh(1), rsync(1)AUTHOR
Brent N. Chun <bnc@theether.org>
COPYING
Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun
NOTES
1. bnc@theether.org
mailto:bnc@theether.org
03/30/2009 PARALLEL-RSYNC(1)