awk to replace values in one file using a second reference file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers awk to replace values in one file using a second reference file
# 1  
Old 04-21-2016
awk to replace values in one file using a second reference file

Hi,

I'd be grateful for your help with the following:

I have a file with a single column (file1). Let's say the values are:

Code:
a
b
c
5
d

I have a second, reference file (ref_file), which is colon-delimited, and is effectively a key. Let's say the values in it are:

Code:
a:1
b:2
c:3
d:4
etc.

I want to use an awk command to scan through file 1, and replace the values with the elements in the reference file, so that the output will be:

Code:
1
2
3
5
4

(I want values that are already in the correct format in file 1 to be left alone - the '5' in this case).

The script I have tried is:

Code:
awk -F: 'FNR==NR{a[$1]=$2;next} {for (i in a)sub(i, a[i]);print}' ref_file file1

This doesn't work for some reason, and I don't know why. When I tried the script on a much shortened version of file1, the error message that I get is: awk: can't open file file1.

Any help/suggestions would be much appreciated.
Many thanks.

Last edited by jim mcnamara; 04-21-2016 at 12:02 PM.. Reason: code tags
# 2  
Old 04-21-2016
Code:
awk -F: 'NR==FNR{a[$1]=$2;next}$1 in a{$1=a[$1]}1' ref_file file1

This User Gave Thanks to Yoda For This Post:
# 3  
Old 04-21-2016
@OP, you script works fine here.. What is your OS and version? Perhaps you made a typo somewhere , maybe the quotes?

---
Slightly different alternative approach, which makes the script a bit more robust by eliminating spaces if present in file1:
Code:
awk 'FNR==NR{A[$1]=$2; next} $1 in A{$1=A[$1]}1' FS=: ref_file FS=" " file1

# 4  
Old 04-21-2016
Thanks for the quick reply. I've tried your version of the script. While it's now reading the files and actually executing the command, the output is completely unchanged from the original file 1...

---------- Post updated at 10:12 AM ---------- Previous update was at 10:09 AM ----------

Scrutinizer, your version of the code appears to have worked.

Thank you both very much.
# 5  
Old 04-21-2016
Can it be that your input files are in DOS format (contain carriage returns)?

Try:
Code:
tr -d '\r' file > file.new

first and on both files and try again...

---
Otherwise there may be excess spaces in file1. My script should take care of this...
This User Gave Thanks to Scrutinizer For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace values on file

Gents, Please i need your help. Using the file2.txt i will like to replace values in file3.txt. Example in file 2 column 1 is the value to find in file3.txt and replace with value in colunm2 (file2.txt). Example file2.txt 21 1209 22 1210file3.txt SCI TB Timestamp Local : 8/30/17... (2 Replies)
Discussion started by: jiam912
2 Replies

2. Shell Programming and Scripting

Keeping record of file 2 based on a reference file 1 in awk

I have 2 input files (tab separated): file1: make_A 1990 foo bar make_B 2010 this that make_C 2004 these those file2: make_X 1970 1995 ref_1:43 ref_2:65 make_A 1970 1995 ref_1:4 ref_2:21 ref_3:18 make_A 1980 2002 ref_1:7 ref_2:7 ref_3:0 ... (2 Replies)
Discussion started by: beca123456
2 Replies

3. Shell Programming and Scripting

Replace values using other file

Gents, Please can you help me. I need to update file1 using file2 values file1 S 44519.00 49349.00 1V1 0.0 0 0.0 0.0 0.0 0.0289091513 S 44513.00 48581.00 1V1 0.0 0 0.0 0.0 0.0 0.0289094319 S 44511.00 48605.00 1V1 0.0 0 0.0... (1 Reply)
Discussion started by: jiam912
1 Replies

4. Shell Programming and Scripting

Replace a value using a reference data from other file

Gents, Can you please help me to solve this case In my input file I have a values in column 49 which always need to be one, but sometimes the system create a value 2, in this case I need to go to search in the original file and replace the values in the row where the value 2 is and in the... (6 Replies)
Discussion started by: jiam912
6 Replies

5. Shell Programming and Scripting

awk file to read values from Db2 table replacing hard coded values

Hi, I want to replace a chain of if-else statement in an old AWK file with values from Db2 table or CSV file. The part of code is below... if (start_new_rec=="true"){ exclude_user="false"; user=toupper($6); match(user, "XXXXX."); if (RSTART ==2 ) { ... (9 Replies)
Discussion started by: asandy1234
9 Replies

6. Shell Programming and Scripting

Replace values in a file with values from another file

Hi, I have 2 input files: File 1: echo Name > create_Name.txt echo Group /dir/group, Name >> create_Name.txt File 2: Name AAA BBB CCC group A B C dir A1 B1 C1 ................................ Need to replace the contents of File 1 with column 2, 3 & 4 values of File 2... (8 Replies)
Discussion started by: gctex
8 Replies

7. Shell Programming and Scripting

Replace from reference file awk

Basically want to replace any field in input file from the refernce file ... for example. clar_2400:3113 in input file will be replaced by clar_2400:3113 Input file field seperator is "," Field which is not found in reference will stay as it is ... Input File ... (3 Replies)
Discussion started by: greycells
3 Replies

8. Shell Programming and Scripting

Help with replace column one content based on reference file

Input file 1 testing 10 20 1 A testing 20 40 1 3 testing 23 232 2 1 testing 10 243 2 . . Reference file 1 final 3 used . . Output file (1 Reply)
Discussion started by: perl_beginner
1 Replies

9. Shell Programming and Scripting

Help with replace column one content based on reference file

Input file 1 testing 10 20 1 A testing 20 40 1 3 testing 23 232 2 1 testing 10 243 2 . . Reference file 1 final 3 used . . Output file (2 Replies)
Discussion started by: perl_beginner
2 Replies

10. Shell Programming and Scripting

Replace character based on reference file problem asking

I got two files right now, input file (target file), reference file 1 (query file) reference file 1 (long list of data) KOLOPWMOPOPO ADASDASD ADSASDASDAD . . target file (one long liner content) ADASDASDTYUKOKOLOPWMOPOPOOPLUAADSASDASDADPOPOUYADADASDASD desired output file content ... (1 Reply)
Discussion started by: patrick87
1 Replies
Login or Register to Ask a Question