Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Parallel move keeping folder structure along with files in it

Shell Programming and Scripting


Tags
mv, parallel, solved

Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 1 Week Ago
cmccabe cmccabe is offline
Registered User
 
Join Date: Nov 2013
Last Activity: 26 June 2017, 3:21 PM EDT
Location: Chicago
Posts: 1,144
Thanks: 686
Thanked 14 Times in 13 Posts
Parallel move keeping folder structure along with files in it

The below will move all the files in the directory dir to the destination using parallel and create a log, however will not keep them in the directory. I have tried mkdir -p but that does not seem to work or at least I can not seem to get it (as it deletes others files when I use it). What is the correct way to move the 3 files into the same directory? Thank you Linux.


Code:
dir="/home/cmccabe/Downloads/snp"
cd "$dir"
parallel -j10 --progress mv -v {} /home/cmccabe/Desktop/indel/ ::: * > /home/cmccabe/medex.logs/archive.log

Folder structure:

Code:
FolderName   --- dir
|
file1
file2
file3

Desired after move:

Code:
FolderName
|
3 files in it

Sponsored Links
    #2  
Old Unix and Linux 1 Week Ago
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 23 June 2017, 6:44 PM EDT
Location: Saskatchewan
Posts: 22,254
Thanks: 1,105
Thanked 4,199 Times in 3,884 Posts
Your hard drive does not work in parallel. There is no point or benefit in running cp, mv, rsync, rm, rmdir, etc in parallel. They will still have to wait for each other. Trying to run them in parallel will actually slow it down and cause fragmentation. Don't do it.

Running tar in parallel - as in your other thread - only worked because bzip2 is so slow that the disk was waiting for the CPU, not vice versa. That's an unusual situation and not at all true of basic file utilities.

Last edited by Corona688; 1 Week Ago at 05:23 PM..
The Following User Says Thank You to Corona688 For This Useful Post:
cmccabe (1 Week Ago)
Sponsored Links
    #3  
Old Unix and Linux 1 Week Ago
cmccabe cmccabe is offline
Registered User
 
Join Date: Nov 2013
Last Activity: 26 June 2017, 3:21 PM EDT
Location: Chicago
Posts: 1,144
Thanks: 686
Thanked 14 Times in 13 Posts
Thank you very much Linux.
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Move files from Space Folder to other folder kannansoft1985 Shell Programming and Scripting 2 08-22-2014 11:59 AM
Archiving files keeping the same structure directory csierra Shell Programming and Scripting 2 08-19-2013 11:52 PM
Move files & folder structure SOCLA_CELT Shell Programming and Scripting 3 08-26-2010 06:47 PM
File Management: How do I move all JPGS in a folder structure to a single folder? guptaxpn Shell Programming and Scripting 4 06-11-2009 02:41 AM
move files and retain subdir structure jason7 Shell Programming and Scripting 4 03-25-2009 01:10 PM



All times are GMT -4. The time now is 06:38 AM.