Moving files to their respective subdirectories


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Moving files to their respective subdirectories
# 1  
Old 03-23-2010
Moving files to their respective subdirectories

Hello,

I have files like this

img.txt
Code:
./Rearrange_zoca/fitted_data/WX226_05b_Ncere.jpg
./Rearrange_zoca/fitted_data/w322_03B_60_70.jpg
./Rearrange_zoca/fitted_data/wrx226_12A_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_01A_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_02A_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_04B_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_05B_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_06A_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_07A_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_08B_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_11B_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_15B_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_20B_50_60.jpg
./Rearrange_zoca/fitted_data/wx124_21B_50_60.jpg
./Rearrange_zoca/fitted_data/wx226_04B_50_60.jpg
./Rearrange_zoca/fitted_data/wx226_04B_Ncere.jpg
./Rearrange_zoca/fitted_data/wx226_05B_50_60.jpg
./Rearrange_zoca/fitted_data/wx226_09A_50_60.jpg
./Rearrange_zoca/fitted_data/wx226_09A_Ncere.jpg
./Rearrange_zoca/fitted_data/wx322-04A_50_60.jpg
./Rearrange_zoca/fitted_data/wx322_01A_50_60.jpg
./Rearrange_zoca/fitted_data/wx322_01A_Ncere.jpg
./Rearrange_zoca/fitted_data/wx322_02A_50_60.jpg
./Rearrange_zoca/fitted_data/wx322_02A_Ncere.jpg
./Rearrange_zoca/fitted_data/wx322_04A_Ncere.jpg
./Rearrange_zoca/fitted_data/wx322_05B_50_60.jpg

they need to be moved to their specific directories which look like

petdir.txt
Code:
./1/zoca_226-04B_JPR/zoca_226-04B_JPR_PET
./1/zoca_226-05B_DMP/zoca_226-05B_DMP_PET
./1/zoca_226-09A_J-L/zoca_226-09A_J-L_PET
./1/zoca_226-12A_ATH/zoca_226-12A_ATH_PET
./1/zoca_322-01A_WSL/zoca_322-01A_WSL_PET
./1/zoca_322-02A_ECB/zoca_322-02A_ECB_PET
./1/zoca_322-03B_KSJ/zoca_322-03B_KJ_PET
./1/zoca_322-04A_J-K/zoca_322-04A_JK_PET
./1/zoca_322-05B-L-S/zoca_322-05B-L-S_PET
./2/zoca_124-01A_KMT/zoca_124-01A_KMT_PET
./2/zoca_124-02A_KJM/zoca_124-02A_KJM_PET
./2/zoca_124-04B_DRC/zoca_124-04B_DRC_PET
./2/zoca_124-05B_LAB/zoca_124-05B_LAB_PET
./2/zoca_124-06A_PLD/zoca_124-06A_PLD_PET
./2/zoca_124-07A_CCR/zoca_124-07A_CCR_PET
./2/zoca_124-08B_DYW/zoca_124-08B_DYW_PET
./2/zoca_124-11B_JKR/zoca_124-11B_JKR_PET
./2/zoca_124-15B_LAM/zoca_124-15B_LAM_PET
./2/zoca_124-20B_C-T/zoca_124-20B_C-T_PET
./2/zoca_124-21B_KWB/zoca_124-21B_KWB_PET

the patterns that need to be matched in these two files are

Code:
grep -o -i  '[0-9]\{3\}[-_][0-9]\{2\}[Ab]' img.txt

awk -F"zoca_" '{print substr($2,1,7)}' petdir.txt

Thanks in advance
# 2  
Old 03-23-2010
The following Perl script prints the appropriate "mv" commands given the two files as input -

Code:
$ 
$ 
$ cat -n img.txt
     1 ./Rearrange_zoca/fitted_data/WX226_05b_Ncere.jpg
     2 ./Rearrange_zoca/fitted_data/w322_03B_60_70.jpg
     3 ./Rearrange_zoca/fitted_data/wrx226_12A_50_60.jpg
     4 ./Rearrange_zoca/fitted_data/wx124_01A_50_60.jpg
     5 ./Rearrange_zoca/fitted_data/wx124_02A_50_60.jpg
     6 ./Rearrange_zoca/fitted_data/wx124_04B_50_60.jpg
     7 ./Rearrange_zoca/fitted_data/wx124_05B_50_60.jpg
     8 ./Rearrange_zoca/fitted_data/wx124_06A_50_60.jpg
     9 ./Rearrange_zoca/fitted_data/wx124_07A_50_60.jpg
    10 ./Rearrange_zoca/fitted_data/wx124_08B_50_60.jpg
    11 ./Rearrange_zoca/fitted_data/wx124_11B_50_60.jpg
    12 ./Rearrange_zoca/fitted_data/wx124_15B_50_60.jpg
    13 ./Rearrange_zoca/fitted_data/wx124_20B_50_60.jpg
    14 ./Rearrange_zoca/fitted_data/wx124_21B_50_60.jpg
    15 ./Rearrange_zoca/fitted_data/wx226_04B_50_60.jpg
    16 ./Rearrange_zoca/fitted_data/wx226_04B_Ncere.jpg
    17 ./Rearrange_zoca/fitted_data/wx226_05B_50_60.jpg
    18 ./Rearrange_zoca/fitted_data/wx226_09A_50_60.jpg
    19 ./Rearrange_zoca/fitted_data/wx226_09A_Ncere.jpg
    20 ./Rearrange_zoca/fitted_data/wx322-04A_50_60.jpg
    21 ./Rearrange_zoca/fitted_data/wx322_01A_50_60.jpg
    22 ./Rearrange_zoca/fitted_data/wx322_01A_Ncere.jpg
    23 ./Rearrange_zoca/fitted_data/wx322_02A_50_60.jpg
    24 ./Rearrange_zoca/fitted_data/wx322_02A_Ncere.jpg
    25 ./Rearrange_zoca/fitted_data/wx322_04A_Ncere.jpg
    26 ./Rearrange_zoca/fitted_data/wx322_05B_50_60.jpg
$ 
$ cat -n petdir.txt
     1 ./1/zoca_226-04B_JPR/zoca_226-04B_JPR_PET
     2 ./1/zoca_226-05B_DMP/zoca_226-05B_DMP_PET
     3 ./1/zoca_226-09A_J-L/zoca_226-09A_J-L_PET
     4 ./1/zoca_226-12A_ATH/zoca_226-12A_ATH_PET
     5 ./1/zoca_322-01A_WSL/zoca_322-01A_WSL_PET
     6 ./1/zoca_322-02A_ECB/zoca_322-02A_ECB_PET
     7 ./1/zoca_322-03B_KSJ/zoca_322-03B_KJ_PET
     8 ./1/zoca_322-04A_J-K/zoca_322-04A_JK_PET
     9 ./1/zoca_322-05B-L-S/zoca_322-05B-L-S_PET
    10 ./2/zoca_124-01A_KMT/zoca_124-01A_KMT_PET
    11 ./2/zoca_124-02A_KJM/zoca_124-02A_KJM_PET
    12 ./2/zoca_124-04B_DRC/zoca_124-04B_DRC_PET
    13 ./2/zoca_124-05B_LAB/zoca_124-05B_LAB_PET
    14 ./2/zoca_124-06A_PLD/zoca_124-06A_PLD_PET
    15 ./2/zoca_124-07A_CCR/zoca_124-07A_CCR_PET
    16 ./2/zoca_124-08B_DYW/zoca_124-08B_DYW_PET
    17 ./2/zoca_124-11B_JKR/zoca_124-11B_JKR_PET
    18 ./2/zoca_124-15B_LAM/zoca_124-15B_LAM_PET
    19 ./2/zoca_124-20B_C-T/zoca_124-20B_C-T_PET
    20 ./2/zoca_124-21B_KWB/zoca_124-21B_KWB_PET
$ 
$ perl -lne 'chomp;
           if ($ARGV eq "img.txt") {if (/.*(\d{3}[_-]\d\d[ab]).*/i) {$y=uc($1); $y=~s/-/_/; $x{$_}=$y}}
           elsif (/.*zoca_(.{7}).*/i) {$y=uc($1); $y=~s/-/_/; foreach $k (keys %x) {if ($x{$k} eq $y) {$x{$k} = $_}}}
           END {foreach $k (keys %x) {printf("mv %s %s\n",$k,$x{$k})}}' img.txt petdir.txt
mv ./Rearrange_zoca/fitted_data/wx322_02A_Ncere.jpg ./1/zoca_322-02A_ECB/zoca_322-02A_ECB_PET
mv ./Rearrange_zoca/fitted_data/wx226_05B_50_60.jpg ./1/zoca_226-05B_DMP/zoca_226-05B_DMP_PET
mv ./Rearrange_zoca/fitted_data/wx124_11B_50_60.jpg ./2/zoca_124-11B_JKR/zoca_124-11B_JKR_PET
mv ./Rearrange_zoca/fitted_data/wx124_02A_50_60.jpg ./2/zoca_124-02A_KJM/zoca_124-02A_KJM_PET
mv ./Rearrange_zoca/fitted_data/wx322-04A_50_60.jpg ./1/zoca_322-04A_J-K/zoca_322-04A_JK_PET
mv ./Rearrange_zoca/fitted_data/wx226_04B_50_60.jpg ./1/zoca_226-04B_JPR/zoca_226-04B_JPR_PET
mv ./Rearrange_zoca/fitted_data/wx124_21B_50_60.jpg ./2/zoca_124-21B_KWB/zoca_124-21B_KWB_PET
mv ./Rearrange_zoca/fitted_data/wx322_01A_Ncere.jpg ./1/zoca_322-01A_WSL/zoca_322-01A_WSL_PET
mv ./Rearrange_zoca/fitted_data/wx124_07A_50_60.jpg ./2/zoca_124-07A_CCR/zoca_124-07A_CCR_PET
mv ./Rearrange_zoca/fitted_data/wx322_05B_50_60.jpg ./1/zoca_322-05B-L-S/zoca_322-05B-L-S_PET
mv ./Rearrange_zoca/fitted_data/wx124_01A_50_60.jpg ./2/zoca_124-01A_KMT/zoca_124-01A_KMT_PET
mv ./Rearrange_zoca/fitted_data/wx124_08B_50_60.jpg ./2/zoca_124-08B_DYW/zoca_124-08B_DYW_PET
mv ./Rearrange_zoca/fitted_data/wx226_09A_50_60.jpg ./1/zoca_226-09A_J-L/zoca_226-09A_J-L_PET
mv ./Rearrange_zoca/fitted_data/wx322_04A_Ncere.jpg ./1/zoca_322-04A_J-K/zoca_322-04A_JK_PET
mv ./Rearrange_zoca/fitted_data/WX226_05b_Ncere.jpg ./1/zoca_226-05B_DMP/zoca_226-05B_DMP_PET
mv ./Rearrange_zoca/fitted_data/wx226_09A_Ncere.jpg ./1/zoca_226-09A_J-L/zoca_226-09A_J-L_PET
mv ./Rearrange_zoca/fitted_data/wx124_04B_50_60.jpg ./2/zoca_124-04B_DRC/zoca_124-04B_DRC_PET
mv ./Rearrange_zoca/fitted_data/wx322_01A_50_60.jpg ./1/zoca_322-01A_WSL/zoca_322-01A_WSL_PET
mv ./Rearrange_zoca/fitted_data/wrx226_12A_50_60.jpg ./1/zoca_226-12A_ATH/zoca_226-12A_ATH_PET
mv ./Rearrange_zoca/fitted_data/wx124_15B_50_60.jpg ./2/zoca_124-15B_LAM/zoca_124-15B_LAM_PET
mv ./Rearrange_zoca/fitted_data/wx226_04B_Ncere.jpg ./1/zoca_226-04B_JPR/zoca_226-04B_JPR_PET
mv ./Rearrange_zoca/fitted_data/w322_03B_60_70.jpg ./1/zoca_322-03B_KSJ/zoca_322-03B_KJ_PET
mv ./Rearrange_zoca/fitted_data/wx322_02A_50_60.jpg ./1/zoca_322-02A_ECB/zoca_322-02A_ECB_PET
mv ./Rearrange_zoca/fitted_data/wx124_20B_50_60.jpg ./2/zoca_124-20B_C-T/zoca_124-20B_C-T_PET
mv ./Rearrange_zoca/fitted_data/wx124_06A_50_60.jpg ./2/zoca_124-06A_PLD/zoca_124-06A_PLD_PET
mv ./Rearrange_zoca/fitted_data/wx124_05B_50_60.jpg ./2/zoca_124-05B_LAB/zoca_124-05B_LAB_PET
$ 
$

Hence, all we have to do is feed the "mv" string as an argument to Perl's system() function. The following Perl script should do the trick.

Code:
##
perl -lne 'chomp;
           if ($ARGV eq "img.txt") {if (/.*(\d{3}[_-]\d\d[ab]).*/i) {$y=uc($1); $y=~s/-/_/; $x{$_}=$y}}
           elsif (/.*zoca_(.{7}).*/i) {$y=uc($1); $y=~s/-/_/; foreach $k (keys %x) {if ($x{$k} eq $y) {$x{$k} = $_}}}
           END {foreach $k (keys %x) {$c=sprintf("mv %s %s\n",$k,$x{$k}); system($c)}}' img.txt petdir.txt

I haven't tested it explicitly since I do not have the appropriate directory structure and files in my system.

HTH,
tyler_durden
# 3  
Old 03-24-2010
Thank you Tyler,

could you please explain your code a little bit. its doing the right thing but i am kind of confused

This is how I did but it involved a lot of steps and few manual edits as well.

list the directory tree
Code:
ls -R >ilist.txt

find the specific directory
Code:
cat ilist.txt |grep "_PET:$" |sed 's/:$//' > petdir.txt

create the new directory where the files are supposed to be moved
Code:
awk '{print "mkdir "$0"/Fitted_Data/"}' petdir.txt |sh

awk '{print $0"/Fitted_Data/"}' petdir.txt >petfit.txt
extract the key words
Code:
awk -F"zoca_" '{print substr($2,1,7)}' petdir.txt
grep -o -i  '[0-9]\{3\}[-_][0-9]\{2\}[Ab]' imgfiles.txt

sort files accordingly in excel and then move
Code:
awk '{print "mv "$3" "$1}' match_sort.txt |sh

i could have used sort -k 1,1 here
# 4  
Old 03-24-2010
Quote:
Originally Posted by avatar_007
...could you please explain your code a little bit. its doing the right thing but i am kind of confused...
Well, it would've been much easier for me if you had asked *specific questions* about the code. It's very difficult for me to gauge your level of understanding of different concepts like -
(a) the options passed to the perl interpreter, e.g. "l", "n", "e"
(b) the chomp operator of Perl
(c) the concept of hashes in Perl, and in general
(d) level of understanding of regular expressions
(e) BEGIN and END blocks in a Perl one-liner
(f) the "uc" function of Perl

So, I will try to explain my code at a very high level. If you want to delve into specifics, you may want to look up all the concepts (a) through (f) in a good book on Perl, or at the perl.org website, or by talking to your local Perl guru.

In one sentence - I built a hash that had filenames from "img.txt" as the keys and their destination directories from "petdir.txt" as their corresponding values.

Here's the code spread out and documented -

Code:
     1  ##
     2  perl -lne 'chomp;                               # remove the newline at the end of the current line being processed
     3             if ($ARGV eq "img.txt") {            # if the current line is of file "img.txt" then
     4               if (/.*(\d{3}[_-]\d\d[ab]).*/i) {  # extract the "NNN[_-]NN[ab]" pattern and store it in $1
     5                 $y=uc($1);                       # change the case of $1 to upper-case and assign it to $y
     6                 $y=~s/-/_/;                      # replace the hyphen "-" by underscore "_" in $y
     7                 $x{$_}=$y                        # add (key,value) = (current_line, $y) in the hash %x
     8               }                                  # end of pattern match
     9             }                                    # end of processing "img.txt"
    10             elsif (/.*zoca_(.{7}).*/i) {         # at this point, we are *ALWAYS* processing "petdir.txt"; 
    11                                                  # so extract 7 characters after "zoca_" and store it in $1
    12               $y=uc($1);                         # change the case of $1 to upper-case and assign it to $y
    13               $y=~s/-/_/;                        # replace the hyphen "-" by underscore "_" in $y
    14               foreach $k (keys %x) {             # now we have to check if the current value of $y exists as a *key value*
    15                                                  # for any key in the hash %x; if it does, then set the *key value* to
    16                                                  # current line
    17                 if ($x{$k} eq $y) {              # if $y is the same as a *key value* for any key in %x, then
    18                   $x{$k} = $_                    # set that *key value* to current line (of "petdir.txt", that is)
    19                 }                                # done comparing *key value* to $y
    20               }                                  # done looping through the hash %x
    21             }                                    # done processing "petdir.txt"
    22             END {                                # we have the complete hash now. each key is a filename from "img.txt"
    23                                                  # and its corresponding value is the destination from "petdir.txt"
    24                                                  # and this correspondence was set due to the matching regexes
    25               foreach $k (keys %x) {             # now just loop through the hash and display "mv <key> <value>"
    26                 printf("mv %s %s\n",$k,$x{$k})
    27               }
    28             }' img.txt petdir.txt

Hope that clears up any questions that you might have.

tyler_durden
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy files in respective directories

Hi Guys, I need to copy the files to respective directories based on name of the file. My script is something like below con=$1 for file in `cat $con` do file_tmp=$(ls -t1 $path| grep -i $file | head -n 1) echo $file_tmp if then cp $path$file_tmp $DIR/ap if then... (16 Replies)
Discussion started by: Master_Mind
16 Replies

2. Shell Programming and Scripting

Append string to all the files inside a directory excluding subdirectories and .zip files

Hii, Could someone help me to append string to the starting of all the filenames inside a directory but it should exclude .zip files and subdirectories. Eg. file1: test1.log file2: test2.log file3 test.zip After running the script file1: string_test1.log file2: string_test2.log file3:... (4 Replies)
Discussion started by: Ravi Kishore
4 Replies

3. Shell Programming and Scripting

Unzip all the files with subdirectories present and append a part of string from the main .zip files

Hi frnds, My requirement is I have a zip file with name say eg: test_ABC_UH_ccde2a_awdeaea_20150422.zip within that there are subdirectories on each directory we again have .zip files and in that we have files like mama20150422.gz and so on. Iam in need of a bash script so that it unzips... (0 Replies)
Discussion started by: Ravi Kishore
0 Replies

4. Shell Programming and Scripting

Bash script deleting my files, and editing files in subdirectories question

#!/bin/bash # name=$1 type=$2 number=1 for file in ./** do if then filenumber=00$number elif then filenumber=0$number fi tempname="$name""$filenumber"."$type" if (4 Replies)
Discussion started by: TheGreatGizmo
4 Replies

5. UNIX for Dummies Questions & Answers

rename files based on their respective directory name

I have a number of files in directories labeled like this: /Data/tr_gray/tr_DTI/dti_FA.nii.gz (the brackets here represent a range of number that the files are labeled with) I need to rename each dti_FA.nii.gz file according to the name of the folder it resides in. For example, the file ... (3 Replies)
Discussion started by: tk0034
3 Replies

6. Shell Programming and Scripting

Script to extract all deb files in a fldr to respective folderstructures

Hi, since half an hour I currently try to write a script that will: 1) Extract every debian file in a directory 2) Extracts it to a folder that matches the debians filename 3) Extracts it so the structure looks like that: DEBIANFILENAME | -------------- DEBIAN ... (9 Replies)
Discussion started by: pasc
9 Replies

7. Shell Programming and Scripting

Find all .htaccess files and make a backup copy in respective directories

Hey guys, I need to know how to locate all .htaccess files on the server and make a backup of them in the folder they reside before I run a script to modify all of them. So basically taking dir1/.htaccess and copying it as dir1/.htaccess_bk dir2/.htaccess copying as dir2/.htaccess_bk... (5 Replies)
Discussion started by: boxx
5 Replies

8. Shell Programming and Scripting

Help with moving files with same names from subdirectories?

I was wondering if someone could help me with this: I have multiple directories and subdirectories with files in them. I need to move all files from all directories and subdirectories into one root directory. However, when i do find /home/user -mindepth 1 -iname "*" -type f -exec mv {} . \; ... (3 Replies)
Discussion started by: r4v3n
3 Replies

9. UNIX for Dummies Questions & Answers

list the files but exclude the files in subdirectories

If I execute the command "ls -l /export/home/abcde/dev/proj/code/* | awk -F' ' '{print $9}' | cut -d'/' -f6-8" it will list all the files in /export/home/abcde/dev/proj/code/ directory as well as the files in subdirectories also proj/code/test.sh proj/code/test1.c proj/code/unix... (8 Replies)
Discussion started by: shyjuezy
8 Replies

10. Shell Programming and Scripting

copy certain files from subdirectories

Hi all, I'd very grateful for some help with the following. I have a variable number of subdirectories each of which contain a uniquely names results file of the form Results*.dat. I would like to search through all subdirectories and copy out these results file to another directory so that... (3 Replies)
Discussion started by: iomaire
3 Replies
Login or Register to Ask a Question