[Solved] Search, Extract and Rename Multiple Files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers [Solved] Search, Extract and Rename Multiple Files
# 1  
Old 02-26-2014
Question [Solved] Search, Extract and Rename Multiple Files

Hi,
I want perl script for the below requirement.

We have lot of files like below name in the directory
Code:
750464921-RE-file2.csv
750452173-RE-file1.csv
750385426-RE-file3.csv
750373470-RE-file4.csv

And also we have another file as "Group.csv" in the same directory as per the below format
Code:
Message ID,Group Name,Group ID
750464921,HB271_NDM,750049864
750325599,Homebase,700006685
750325600,Homebase,700006685
750325713,Homebase,700006685
750325714,Homebase,700006685
750385426,Homebase,700006685
750341026,Nectar NEC,750013247
750341070,Product review instore,750000029
750373470,Nectar NEC,750013247
750452173,Homebase,700006685

Script has to get the message id from one the above files (750464921-RE-file2.csv) and search it in the "Group.csv" file if it matches then get the respective Group Name (2nd Column) and add the group name with hypen(-) in the prefix of the file name (HB271_NDM-750464921-RE-file2.csv) like that script has to do for all the files which starts with 750*.csv

Thanks,
Saravanan

Last edited by Don Cragun; 02-26-2014 at 06:39 AM.. Reason: Add CODE tags.
# 2  
Old 02-26-2014
Hello Armsaran,

Following may help you.

Code:
cat check_files_ls_OUPUT_together.ksh
ls -ltr > OUTPUT_ls
while read line
do
value=`echo $line | awk '{print $9}' | awk -F"\-" '{print $1}'`
#echo $value
while read line1
do
value1=`echo $line1 | awk -F"," '{print $1}'`
value2=`echo $line1 | awk -F"," '{print $2}'`
 
if [[ "$value" == "$value1" ]]
then
echo $value2"-"$value1"-RE-file2.csv"
continue 2
fi
done < "check_file_names_OUTPUT"
 
done < "OUTPUT_ls"

Ouptut will be as follows.


Code:
HB271_NDM-750464921-RE-file2.csv
Homebase-750452173-RE-file2.csv
Homebase-750385426-RE-file2.csv
Nectar NEC-750373470-RE-file2.csv

NOTE: Where OUTPUT_ls is the output of command ls -ltr
and filename check_file_names_OUTPUT is the file with your given input and which present in same directory.


EDIT: In place of echo you can use mv command to rename the filenames for same.


Thanks,
R. Singh

Last edited by RavinderSingh13; 02-26-2014 at 06:44 AM.. Reason: Adding solution to rename the files in same
This User Gave Thanks to RavinderSingh13 For This Post:
# 3  
Old 02-26-2014
See this might help you..

This only for test purpose
Code:

$ cat test 
750464921-RE-file2.csv
750452173-RE-file1.csv
750385426-RE-file3.csv
750373470-RE-file4.csv

This is also only for test purpose
Code:
$ cat test | xargs touch

So these are your files
Code:
$ ls -1
750373470-RE-file4.csv
750385426-RE-file3.csv
750452173-RE-file1.csv
750464921-RE-file2.csv
test

This is your group file
Code:
$ nano group.csv

Contents of group file
Code:
$ cat group.csv 
Message ID,Group Name,Group ID
750464921,HB271_NDM,750049864
750325599,Homebase,700006685
750325600,Homebase,700006685
750325713,Homebase,700006685
750325714,Homebase,700006685
750385426,Homebase,700006685
750341026,Nectar NEC,750013247
750341070,Product review instore,750000029
750373470,Nectar NEC,750013247
750452173,Homebase,700006685

Just to print matchup
Code:
$ ls *-*-*.csv -1 | awk -F'[-,]' 'FNR==NR{A[$1]=$0;next}($1 in A){print $2"-"A[$1]}' - group.csv
HB271_NDM-750464921-RE-file2.csv
Homebase-750385426-RE-file3.csv
Nectar NEC-750373470-RE-file4.csv
Homebase-750452173-RE-file1.csv

copy with newnames
Code:
$ ls *-*-*.csv -1 | awk -F'[-,]' 'FNR==NR{A[$1]=$0;next}($1 in A){gsub(/[[:space:]]/,"\\ ",$2);print "cp "A[$1],$2"-"A[$1]}' - group.csv | sh

Final output
Code:
$ ls -1
750373470-RE-file4.csv
750385426-RE-file3.csv
750452173-RE-file1.csv
750464921-RE-file2.csv
group.csv
HB271_NDM-750464921-RE-file2.csv
Homebase-750385426-RE-file3.csv
Homebase-750452173-RE-file1.csv
Nectar NEC-750373470-RE-file4.csv
test

---------- Post updated at 05:47 PM ---------- Previous update was at 05:44 PM ----------

If happy with result replace cp with mv
# 4  
Old 02-26-2014
Thanks Akshay.

Its working good.
Also i want to add only the first 10 character if its more than 10 from the group name.

Could you please provide the update script as per the above requirement?
# 5  
Old 02-26-2014
Glad to know that its working
use this one
Code:
ls *-*-*.csv -1 | awk -F'[-,]' 'FNR==NR{A[$1]=$0;next}($1 in A){gsub(/[[:space:]]/,"\\ ",$2);print "cp "A[$1],substr($2,1,10)"-"A[$1]}' - group.csv | sh

# 6  
Old 02-26-2014
If any of your Group.csv 2nd field values contain a tab character or if any of your original filenames contain a space or tab character, the following might work better:
Code:
ls *.csv | awk -F, '
NR == 1 { next }
FNR == NR { c[$1] = $2; next }
$1 in c { printf("mv \"%s\" \"%s-%s\"\n", $0, c[$1], $0) 
}' Group.csv FS='-' -

This code just prints mv commands to be executed. If the output produces the mv commands you want to be run, change the last line in the script to:
Code:
}' Group.csv FS='-' - | sh

to actually move the files.

Note that none of the suggestions presented so far (including the script above) will work correctly if any of the filenames being used as sources or targets for the move contain any dollar-sign ($), double-quote ("), or newline characters.

If you want to run any of these scripts that use awk on a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of the default /usr/bin/awk.

Update for new requirements:
To use at most 10 characters in the added prefix, change the line:
Code:
FNR == NR { c[$1] = $2; next }

to:
Code:
FNR == NR { c[$1] = substr($2, 1, 10); next }


Last edited by Don Cragun; 02-26-2014 at 08:57 AM.. Reason: Add changes for new requirements.
# 7  
Old 02-26-2014
Thanks Akshay.
It looks good but i have the below issue
Group.csv
Code:
750428595,Product Review Email (online),Product review online,750000028
750400039,Copy of Gardening Category Clickers,Test_Hanh,750058890
750416394,Power Tools and Grass Category Clickers_19_03_2013,Lawn_Manual_19032013,750062723

Output files as per the script
Code:
Product R-750428595-RE-file190.csv
Copy of -750400039-RE-file277.csv
Power Too-750416394-RE-file258.csv

Output files should be like below
Code:
Product Re-750428595-RE-file190.csv
Copy of Ga-750400039-RE-file277.csv
Power Too -750416394-RE-file258.csv

Thanks Don.
Code is not restricted for the first 10 chars
Output file as per your code
Code:
HB280RS_CDEF_CorporateEvent-750383990-RE-file26.csv
HB273_CDEF_November_corporate_event-750343656-RE-file89.csv
Copy of Gardening Category Clickers-750400039-RE-file277.csv


Last edited by Franklin52; 02-26-2014 at 10:45 AM.. Reason: Please use code tags
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

2. UNIX for Dummies Questions & Answers

Rename Multiple Files

Hey guys, I am the definition of a newbie. I am in the process of trying to rip all my dvds onto a new HTPC I setup. While doing this, I am also trying to organize a bunch of other files I already have to proper naming conventions. So far I have just been naming each file separately (I am on a... (4 Replies)
Discussion started by: Ralze34
4 Replies

3. UNIX for Dummies Questions & Answers

[SOLVED] Rename multiple files

Hi. I have a large number of files with names like: t_ 0.20000E-02.dat There is actually a space after the underscore. These files are numbered numerically, i.e. t_ 0.20000E-02.dat, t_ 0.21000E-02.dat, t_ 0.22000E-02.dat and so on. What I would like to do is rename such that the file with... (8 Replies)
Discussion started by: lost.identity
8 Replies

4. Shell Programming and Scripting

Rename the multiple files

Hi I need to reanme the multiple file using unix script I have multiple file like: sample_YYYYMMDD.xls test new_YYYYMMDD.xls simple_YYYYMMDD.xls I need to rename this file sample.xls testnew.xls SIMPLE.xls thanks (8 Replies)
Discussion started by: murari83.ds
8 Replies

5. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

6. UNIX for Dummies Questions & Answers

help with multiple files rename...

Hi everyone, I'm very green in Linux. Please help me to solve my problem. I have thousands of files and I want to change their names. They have naming convection: prefix_date_date+1_suffix.nc prefix: ext-GLORY date_date+1: 20020101_20020102 and two types of suffix: gridV_R20020130 and... (3 Replies)
Discussion started by: makikicindy
3 Replies

7. UNIX for Dummies Questions & Answers

How to rename multiple files

Hi all, I have some files like: pickup.0000043200.t001.t001.data pickup.0000043200.t001.t002.data pickup.0000043200.t002.t001.data pickup.0000043200.t002.t002.data pickup.0000043200.t003.t001.data pickup.0000043200.t003.t002.data I need to rename these files to ... (4 Replies)
Discussion started by: a_dor8
4 Replies

8. UNIX for Dummies Questions & Answers

rename multiple files from search output

Variations of multiple renames seems to come up a lot but i can't find the answer to this situation. Tidying up a directory where people rename files to .working, .bob, .attempt1 & so on. what i am trying to do is: list the file type, & rename from ".whatever" to .fixed. As the ".whatever" is... (5 Replies)
Discussion started by: woodstock
5 Replies

9. Shell Programming and Scripting

now to rename multiple files

I have several hundred files in one directory which I need to move to another directory with the new extension, for example: /bb/data/rptmgr* are in the source directory need to be moved to /bb/data55/rptmgr*.new Is there an efficient way to do it? Thanks -A (4 Replies)
Discussion started by: aoussenko
4 Replies

10. Shell Programming and Scripting

rename multiple files

Hi, can anyone have a ksh script to rename multiple files (ie to remove .Z extension of the files) can someone correct this? for i in *.Z do var1 = substr($i, 1,at(".Z",$i)-1) mv $i $var1 done Thanks.. Antony (13 Replies)
Discussion started by: antointoronto
13 Replies
Login or Register to Ask a Question