06-16-2017
Your hard drive does not work in parallel. There is no point or benefit in running cp, mv, rsync, rm, rmdir, etc in parallel. They will still have to wait for each other. Trying to run them in parallel will actually slow it down and cause fragmentation. Don't do it.
Running tar in parallel - as in your other thread - only worked because bzip2 is so slow that the disk was waiting for the CPU, not vice versa. That's an unusual situation and not at all true of basic file utilities.
Last edited by Corona688; 06-16-2017 at 06:23 PM..
This User Gave Thanks to Corona688 For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hi:
I have some files like this
folder1/recording1.mp3
folder1/docs/budget.doc
folder2/others/misc.mp3
folder3/others/notes.doc
all this folders and files are under the mp3 folder.
I would like to move just the mp3s to another folder but retain the subdir structure i have.
So if... (4 Replies)
Discussion started by: jason7
4 Replies
2. Shell Programming and Scripting
This is the file structure:
DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg
DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg
DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg
DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg
DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies
3. Shell Programming and Scripting
Hey, this might be a really basic question, but I'm new to Unix scripting.
I'm trying to find a command to replicate a file structure from one location to another & move the actual files contained in the base directories, i.e. I have this structure -
home/temp/test/dir1/ ... (3 Replies)
Discussion started by: SOCLA_CELT
3 Replies
4. Shell Programming and Scripting
Hi all,
Hello everyone, my first post here :). I tried to search the forum but I didn't find exactly what I was looking for...
I need to zip/tar files across entire filesystem which are more 3+ years old but leave folder structure intact. If the script locates tar/zip files they are more... (1 Reply)
Discussion started by: sashruby
1 Replies
5. Shell Programming and Scripting
Hello Team,
We would like to backup a lot of files inside of a structure of directories, four, five or more levels in some Ubuntu, Mac and Solaris systems.
For instance:
/home/chuck/sales/virgin/rent-quote.pdf
/home/chuck/sales/marriott/vacation-quote.pdf... (2 Replies)
Discussion started by: csierra
2 Replies
6. Shell Programming and Scripting
Hi,
I need to write a script the has to copy the files from folders and subfolders to the same folder structure located in another location.
Ex:
mainfolder1
file1,file2,file3 subfolder1(file1,etc) subfolder2(file1,etc) to another folder location of same folder structure.
rsync is not... (7 Replies)
Discussion started by: Raji Perumal
7 Replies
7. Shell Programming and Scripting
I want to move a folder with spaces from one folder to another.
I have two folders like this,
1).RT_032-222 -4444-01/ 2). RT_032-555 -7777-01/
I want to move files from 2 to 1 through shell script.Here I want to assign this like a user defined variable like as Source branch... (2 Replies)
Discussion started by: kannansoft1985
2 Replies
8. Shell Programming and Scripting
Hello,
I need to move all the files inside /XYZ (has multi-depth sub directories) that are older than 14 days to/ABC directory but with retaining the SAME directory structure.
for example:
/XYZ/1/2/3/A/b.txt should be moved as /ABC/1/2/3/A/b.txt
I know about find /XYZ -type f -mtime +14... (3 Replies)
Discussion started by: prvnrk
3 Replies
9. Shell Programming and Scripting
We are receiving few zipped files in one location say : apple/oranges/incoming
All .zip files are placed here in incoming folder.
So few of the files are password encrypted.
There are only 10 zipped files, so we are planning to create a script which will pick that zip file from incoming... (1 Reply)
Discussion started by: Sidhant
1 Replies
10. Shell Programming and Scripting
Hi Team,
I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder.
Eg:
parent folder --> /Interface/data/test/IN
Sub folder -->/Interface/data/test/IN/Invoice20180607233338
Subfolder will be always with timestamp... (6 Replies)
Discussion started by: srivarun15
6 Replies
LEARN ABOUT DEBIAN
parallel-scp
PARALLEL-SCP(1) PARALLEL-SCP(1)
NAME
parallel-scp - parallel versions of scp
SYNOPSIS
parallel-scp [OPTIONS] -h hosts.txt local remote
DESCRIPTION
pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of
homogenously-configured hosts.
parallel-scp copy files in parallel to a set of machines.
OPTIONS
-r --recursive
recusively copy directories (OPTIONAL)
-h --hosts
hosts file (each line "host[:port] [user]")
-l --user
username (OPTIONAL)
-p --par
max number of parallel threads (OPTIONAL)
-o --outdir
output directory for stdout files (OPTIONAL)
-e --errdir
output directory for stderr files (OPTIONAL)
-t --timeout
timeout (secs) (-1 = no timeout) per host (OPTIONAL)
-O --options
SSH options (OPTIONAL)
-v --verbose
turn on warning and diagnostic messages (OPTIONAL)
EXAMPLE
The following example runs hostname on three machines (IPs or hostnames) specified in the file ips.txt using login irb2 and saves the
output in /tmp/foo.
# parallel-scp -h ips.txt -l irb2 /etc/hosts /tmp/hosts
Success on 128.112.152.122:22
Success on 18.31.0.190:22
Success on 128.232.103.201:22
ENVIRONMENT
All four programs take similar sets of options. All of these options can be set using the following environment variables:
o PSSH_HOSTS
o PSSH_USER
o PSSH_PAR
o PSSH_OUTDIR
o PSSH_VERBOSE
o PSSH_OPTIONS
SEE ALSO
parallel-ssh(1), parallel-slurp(1), parallel-nuke(1), parallel-rsync(1), ssh(1), scp(1),
AUTHOR
Brent N. Chun <bnc@theether.org>
COPYING
Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun
NOTES
1. bnc@theether.org
mailto:bnc@theether.org
03/30/2009 PARALLEL-SCP(1)