awk to update file with partial matching line in another file and append text


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers awk to update file with partial matching line in another file and append text
# 1  
Old 04-13-2019
awk to update file with partial matching line in another file and append text

In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is much larger (several hundreds of lines) in both f1 and f2. When the line in f2 is pasted to $3 in f1, the value in $1 is appended to it at the end of the line with a /test/id/$1_raw.file_fixed.txt. Most of this is static text, except the value from $1 is after the third /. Thank you Smilie.

f1
Code:
xyxy_0268 0000-yyyy
xyxy_0270 1111-xxxx
R_0000_00_02_00_45_32_xxxx_x0-0000-100-x0.0_xxxx_xxxx_xxxx_xxxx_xxxx_xxxx

f2
Code:
xyxy_0268 0000-yyyy /path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449 /path/to/the/xxx/data/00-0000_xxxx-03_v1/00-0000_xxxx-03_v1_20190322115521953
xyxy_0270 1111-xxxx /path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953

desired
Code:
xyxy_0268 0000-yyyy /path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449/test/id/xyxy_0268_raw.file_fixed.txt
xyxy_0270 1111-xxxx /path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953/test/id/xyxy_0270_raw.file_fixed.txt
R_0000_00_02_00_45_32_xxxx_x0-0000-100-x0.0_xxxx_xxxx_xxxx_xxxx_xxxx_xxxx

awk
Code:
awk 'NR==FNR {id[$2]; next} $2 in id' f1 f2 | awk '1;$NF=/$[id]/{ print "/test/id/$[id]_raw.file_fixed.txt"}' > out

# 2  
Old 04-13-2019
I'm not sure that I am following what you are trying to do.

We can easily produce the output you say you want from your two sample input files with just:
Code:
awk 'FNR == NR{print; next} NF == 1' f2 f1

or even:
Code:
awk 'FNR == NR; NF == 1' f2 f1

This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 04-14-2019
Hi,

Try:
Code:
awk 'NR==FNR {id[$2]=$3; next} $2 in id{$3=id[$2] "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1


But you wrote:

Quote:
Originally Posted by cmccabe
[..]I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. [..]
So in that case you may need to try something like:
Code:
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$3; next} $2 in id{$3=id[$2] "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1

This User Gave Thanks to Scrutinizer For This Post:
# 4  
Old 04-16-2019
I made an typo in f2
Should just be one column of multiple strings.
Code:
/path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449 
/path/to/the/xxx/data/00-0000_xxxx-03_v1/00-0000_xxxx-03_v1_20190322115521953
/path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953

So I thought i adjusted the script below correctly to capture the partial match. That is the $2 will be in that long string. Thank you Smilie.

Awk
Code:
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$2]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$i]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1


Last edited by cmccabe; 04-17-2019 at 12:23 PM.. Reason: fixed format, updated awk
# 5  
Old 04-19-2019
Quote:
Originally Posted by cmccabe
I made an typo in f2
Should just be one column of multiple strings.
Code:
/path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449 
/path/to/the/xxx/data/00-0000_xxxx-03_v1/00-0000_xxxx-03_v1_20190322115521953
/path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953

So I thought i adjusted the script below correctly to capture the partial match. That is the $2 will be in that long string. Thank you Smilie.

Awk
Code:
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$2]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$i]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1

It still isn't clear to me what you are trying to do, but I assume that the above code is giving errors for trying to divide by an element of an unnamed array and then trying to divide the result of that by a non-numeric string. But, that problem may be hidden by the fact that no value in $2 in f1 (i.e., 0000-yyyy, 1111-xxxx, or the empty string from the 3rd line in f1) ever appears as a field in f2. And, therefore, the condition $2 in id is never true.

And, since there is only one field in every line in f2, the only array subscripts in the id[] array are complete lines from f2. Why run a loop from 1 through NF when NF is always 1?

Since you say that the corrected f2 only contains one string, the first part of the code in both awk scripts above:
Code:
NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next}

could more simply be written as:
Code:
NR==FNR {id[$0]=$0; next}

or as:
Code:
NR==FNR {id[$1]=$1

which would both produce exactly the same id[] arrays.

Since there are no matching lines, I still can't make any sense of the description of what you are trying to combine from matching lines in the two files???
These 2 Users Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to update file with sum of matching fields in another file

In the awk below I am trying to add a penalty to a score to each matching $1 in file2 based on the sum of $3+$4 (variable TL) from file1. Then the $4 value in file1 is divided by TL and multiplied by 100 (this valvue is variable S). Finally, $2 in file2 - S gives the updated $2 result in file2.... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Append Text File from another File in line 6

Hi, Anyone can help on how to append a file1.txt into file2.txt after line 3 using sed command. file1.txt 1. testa 2. testb 3. testc 4. testd 5. test5 6. test6 file2.txt break here this is a test break end here output 1. testa 2. testb (1 Reply)
Discussion started by: fspalero
1 Replies

4. Shell Programming and Scripting

Needed shell script to append desired text to each line in a file

Hi, I had generated a report in my tool as followsoutput.txt 43.35 9 i needed the script to generate a new file like below i want to append the text to each of these lines of my filenewoutputfile.txt should be Total Amount : 43.35 Record Count:9 Regards, Vasa Saikumar. ... (2 Replies)
Discussion started by: hemanthsaikumar
2 Replies

5. Shell Programming and Scripting

Update specific field in a line of text file

I have a text file like this: subject1:LecturerA:10 subject2:LecturerA:40 if I was given string in column 1 and 2 (which are subject 1 and LecturerA) , i need to update 3rd field of that line containing that given string , which is, number 10 need to be updated to 100 ,for example. The... (6 Replies)
Discussion started by: bmtoan
6 Replies

6. UNIX for Dummies Questions & Answers

Append a line to single column text file

I would like to add a line to the end of a single column text file. How do I go about doing that? Input: BEGIN 1 2 3 Output: BEGIN 1 2 3 END Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. Shell Programming and Scripting

AWK - Print partial line/partial field

Hello, this is probably a simple request but I've been toying with it for a while. I have a large list of devices and commands that were run with a script, now I have lines such as: a-router-hostname-C#show ver I want to print everything up to (and excluding) the # and everything after it... (3 Replies)
Discussion started by: ippy98
3 Replies

8. Shell Programming and Scripting

append delimeter count for each line in text file

Hi guys, plz tell me how to achieve this how to delete the lines in a file using sed command (6 Replies)
Discussion started by: hari908
6 Replies

9. Shell Programming and Scripting

How to append text to the second line of a file

Say I have a text file like: 1 3 4 How would I use ksh to put the number '2' into the second line of that file? I'm using OpenBSD so the sed syntax might be a bit different (I have no idea how to use sed, though) (4 Replies)
Discussion started by: guitarscn
4 Replies

10. Shell Programming and Scripting

Append text at end of the first line in a file

Hi I need to append some text @ end of the first line in a file. like myfile.txt list = a,b,c list.a=some.. I give the arg "d" . now it append at end of first line list=a,b,c,d list.a=some... Please help me out this (7 Replies)
Discussion started by: catgovind
7 Replies
Login or Register to Ask a Question