Sponsored Content
Top Forums Shell Programming and Scripting Parallel move keeping folder structure along with files in it Post 302999313 by Corona688 on Friday 16th of June 2017 04:59:16 PM
Old 06-16-2017
Your hard drive does not work in parallel. There is no point or benefit in running cp, mv, rsync, rm, rmdir, etc in parallel. They will still have to wait for each other. Trying to run them in parallel will actually slow it down and cause fragmentation. Don't do it.

Running tar in parallel - as in your other thread - only worked because bzip2 is so slow that the disk was waiting for the CPU, not vice versa. That's an unusual situation and not at all true of basic file utilities.

Last edited by Corona688; 06-16-2017 at 06:23 PM..
This User Gave Thanks to Corona688 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

move files and retain subdir structure

hi: I have some files like this folder1/recording1.mp3 folder1/docs/budget.doc folder2/others/misc.mp3 folder3/others/notes.doc all this folders and files are under the mp3 folder. I would like to move just the mp3s to another folder but retain the subdir structure i have. So if... (4 Replies)
Discussion started by: jason7
4 Replies

2. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

3. Shell Programming and Scripting

Move files & folder structure

Hey, this might be a really basic question, but I'm new to Unix scripting. I'm trying to find a command to replicate a file structure from one location to another & move the actual files contained in the base directories, i.e. I have this structure - home/temp/test/dir1/ ... (3 Replies)
Discussion started by: SOCLA_CELT
3 Replies

4. Shell Programming and Scripting

Need to Zip files three years old or longer but leave folder structure intact

Hi all, Hello everyone, my first post here :). I tried to search the forum but I didn't find exactly what I was looking for... I need to zip/tar files across entire filesystem which are more 3+ years old but leave folder structure intact. If the script locates tar/zip files they are more... (1 Reply)
Discussion started by: sashruby
1 Replies

5. Shell Programming and Scripting

Archiving files keeping the same structure directory

Hello Team, We would like to backup a lot of files inside of a structure of directories, four, five or more levels in some Ubuntu, Mac and Solaris systems. For instance: /home/chuck/sales/virgin/rent-quote.pdf /home/chuck/sales/marriott/vacation-quote.pdf... (2 Replies)
Discussion started by: csierra
2 Replies

6. Shell Programming and Scripting

Copying files from various folders to similar folder structure in another location

Hi, I need to write a script the has to copy the files from folders and subfolders to the same folder structure located in another location. Ex: mainfolder1 file1,file2,file3 subfolder1(file1,etc) subfolder2(file1,etc) to another folder location of same folder structure. rsync is not... (7 Replies)
Discussion started by: Raji Perumal
7 Replies

7. Shell Programming and Scripting

Move files from Space Folder to other folder

I want to move a folder with spaces from one folder to another. I have two folders like this, 1).RT_032-222 -4444-01/ 2). RT_032-555 -7777-01/ I want to move files from 2 to 1 through shell script.Here I want to assign this like a user defined variable like as Source branch... (2 Replies)
Discussion started by: kannansoft1985
2 Replies

8. Shell Programming and Scripting

How to move the files older than x days with similar directory structure?

Hello, I need to move all the files inside /XYZ (has multi-depth sub directories) that are older than 14 days to/ABC directory but with retaining the SAME directory structure. for example: /XYZ/1/2/3/A/b.txt should be moved as /ABC/1/2/3/A/b.txt I know about find /XYZ -type f -mtime +14... (3 Replies)
Discussion started by: prvnrk
3 Replies

9. Shell Programming and Scripting

Script to move files and Creation of folder structure

We are receiving few zipped files in one location say : apple/oranges/incoming All .zip files are placed here in incoming folder. So few of the files are password encrypted. There are only 10 zipped files, so we are planning to create a script which will pick that zip file from incoming... (1 Reply)
Discussion started by: Sidhant
1 Replies

10. Shell Programming and Scripting

Request for Shell script to move files from Subfolder to Parent folder and delete sub folder

Hi Team, I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder. Eg: parent folder --> /Interface/data/test/IN Sub folder -->/Interface/data/test/IN/Invoice20180607233338 Subfolder will be always with timestamp... (6 Replies)
Discussion started by: srivarun15
6 Replies
PARALLEL-SCP(1) 														   PARALLEL-SCP(1)

NAME
parallel-scp - parallel versions of scp SYNOPSIS
parallel-scp [OPTIONS] -h hosts.txt local remote DESCRIPTION
pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of homogenously-configured hosts. parallel-scp copy files in parallel to a set of machines. OPTIONS
-r --recursive recusively copy directories (OPTIONAL) -h --hosts hosts file (each line "host[:port] [user]") -l --user username (OPTIONAL) -p --par max number of parallel threads (OPTIONAL) -o --outdir output directory for stdout files (OPTIONAL) -e --errdir output directory for stderr files (OPTIONAL) -t --timeout timeout (secs) (-1 = no timeout) per host (OPTIONAL) -O --options SSH options (OPTIONAL) -v --verbose turn on warning and diagnostic messages (OPTIONAL) EXAMPLE
The following example runs hostname on three machines (IPs or hostnames) specified in the file ips.txt using login irb2 and saves the output in /tmp/foo. # parallel-scp -h ips.txt -l irb2 /etc/hosts /tmp/hosts Success on 128.112.152.122:22 Success on 18.31.0.190:22 Success on 128.232.103.201:22 ENVIRONMENT
All four programs take similar sets of options. All of these options can be set using the following environment variables: o PSSH_HOSTS o PSSH_USER o PSSH_PAR o PSSH_OUTDIR o PSSH_VERBOSE o PSSH_OPTIONS SEE ALSO
parallel-ssh(1), parallel-slurp(1), parallel-nuke(1), parallel-rsync(1), ssh(1), scp(1), AUTHOR
Brent N. Chun <bnc@theether.org> COPYING
Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun NOTES
1. bnc@theether.org mailto:bnc@theether.org 03/30/2009 PARALLEL-SCP(1)
All times are GMT -4. The time now is 03:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy