Issue with search and replacing multiple items in multiple files


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Issue with search and replacing multiple items in multiple files
# 1  
Old 09-23-2019
Issue with search and replacing multiple items in multiple files

Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values.

Code:
for date in {1..31}
do
for val in `cat val.txt`
do
csv="file/location/csv"
fid=$(awk -F"," '$2 == "'$val'" { print $3 }' $csv)
awk '$1=='${val}'{$1='${fid}'}{print}' data-${date}.txt > data-${date}-new.txt
done
done

For instance:
Code:
val.txt
01
02
03
04

csv
Code:
abc,01,01
abc,50,02
abc,02,04
abc,03,05
abc,04,06

data-1.txt
Code:
01  001  1.062
01  003  0.216
01  005  1.89
01  007  0.828
02  001  1.44
02  003  -3.492
02  005  -4.536
03  147  -0.144
03  149  0.216
04  001  -0.144
04  003  -1.62

The result I was hoping for was this in all of the files:

data-1-new.txt
Code:
01  001  1.062
01  003  0.216
01  005  1.89
01  007  0.828
04  001  1.44
04  003  -3.492
04  005  -4.536
05  147  -0.144
05  149  0.216
06  001  -0.144
06  003  -1.62

# 2  
Old 09-23-2019
Why that complicated? Try
Code:
awk '
FNR == 1        {FC++
                 FN = FILENAME
                 sub (/.txt/, "-new&", FN)
                }
FC == 1         {VAL[$1]
                 next
                }
FC == 2         {CSV[$2] = $3
                 next
                }
$1 in VAL       {$1 = CSV[$1]
                }
                {print >  FN
                }

' FS=, val.txt file/location/csv FS=" " data-*.txt

It will read the val and csv files and put their values into respective arrays, and then pick any data*.text file around and convert it into a "new" one. If you want to restrict the file selection to 1 .. 31 but skip non-exististing ones, try using shell patterns. You may need to adapt the field separator FS.
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed parser behaving strange on replacing multiple words in multiple files

I have 4000 files like $cat clus_grp_seq10_g.phy 18 1002 anig_OJJ65951_1 ATGGTTTCGCAGCGTGATAGAGAATTGTTTAGGGATGATATTCGCTCGCGAGGAACGAAGCTCAATGCTGCCGAGCGCGAGAGTCTGCTAAGGCCATATCTGCCAGATCCGTCTGACCTTCCACGCAGGCCACTTCAGCGGCGCAAGAAGGTTCCTCG aver_OOF92921_1 ... (1 Reply)
Discussion started by: sammy777888
1 Replies

2. Shell Programming and Scripting

Replacing old TNS entries with New one in multiple files

I have requirement to replace old TNS entries with New one in multiple files. one file may contain more then one occurrence of tns. Example: Below is the one of occurrence in a current file(s). i am interested to replace only red part. <connection-pool name="Google_APP_CP"... (4 Replies)
Discussion started by: KDDubai333
4 Replies

3. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

4. Shell Programming and Scripting

Search & Replace: Multiple Strings / Multiple Files

I have a list of files all over a file system e.g. /home/1/foo/bar.x /www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g. if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc. There... (2 Replies)
Discussion started by: spacegoose
2 Replies

5. Shell Programming and Scripting

Search multiple patterns in multiple files

Hi, I have to write one script that has to search a list of numbers in certain zipped files. For eg. one file file1.txt contains the numbers. File1.txt contains 5,00,000 numbers and I have to search each number in zipped files(The number of zipped files are around 1000 each file is 5 MB) I have... (10 Replies)
Discussion started by: vsachan
10 Replies

6. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Hi guys, say I have a few files in a directory (58 text files or somthing) each one contains mulitple strings that I wish to replace with other strings so in these 58 files I'm looking for say the following strings: JAM (replace with BUTTER) BREAD (replace with CRACKER) SCOOP (replace... (19 Replies)
Discussion started by: rich@ardz
19 Replies

7. Shell Programming and Scripting

Replacing text from multiple files at multiple location

Hi, I have many files scattered in all different folders. I want to replace the text within all the files using a single command ( awk, sed...) Is it possible? example find all the files in which there is text "memory" and replace it with "branded_memories". the files can be at the... (2 Replies)
Discussion started by: rudoraj
2 Replies

8. Shell Programming and Scripting

Replacing string in multiple files

Hi, I need to replace the string 'abcd' with 'xyz' in a file sample.xml This sample.xml is also present in the subdirectories of the current directory. Eg, If I am in /user/home/ the sample.xml if present in /user/home/ /user/home/folder1/ /user/home/folder2/... (3 Replies)
Discussion started by: arulanandsp
3 Replies

9. Shell Programming and Scripting

Multiple search string in multiple files using awk

Hi, filenames: contains name of list of files to search in. placelist contains the names of places to be searched in all files in "filenames" for i in $(<filenames) do egrep -f placelist $i if ] then echo $i fi done >> outputfile Output i am getting: (0 Replies)
Discussion started by: pinnacle
0 Replies

10. Shell Programming and Scripting

Multiple search in multiple files

Hello, I would like to perform a search of several different strings in different files. I have the ouput of a unix command X which is for instance: aaa bbb ccc and I would like to look for each of these three strings into several files: file1 file2 etc. I have come up with a solution... (2 Replies)
Discussion started by: maxvirrozeito
2 Replies
Login or Register to Ask a Question