Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

awk to update file with partial matching line in another file and append text

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers awk to update file with partial matching line in another file and append text
# 1  
Old 04-13-2019
awk to update file with partial matching line in another file and append text
In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is much larger (several hundreds of lines) in both f1 and f2. When the line in f2 is pasted to $3 in f1, the value in $1 is appended to it at the end of the line with a /test/id/$1_raw.file_fixed.txt. Most of this is static text, except the value from $1 is after the third /. Thank you Smilie.

f1
Code:
xyxy_0268 0000-yyyy
xyxy_0270 1111-xxxx
R_0000_00_02_00_45_32_xxxx_x0-0000-100-x0.0_xxxx_xxxx_xxxx_xxxx_xxxx_xxxx

f2
Code:
xyxy_0268 0000-yyyy /path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449 /path/to/the/xxx/data/00-0000_xxxx-03_v1/00-0000_xxxx-03_v1_20190322115521953
xyxy_0270 1111-xxxx /path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953

desired
Code:
xyxy_0268 0000-yyyy /path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449/test/id/xyxy_0268_raw.file_fixed.txt
xyxy_0270 1111-xxxx /path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953/test/id/xyxy_0270_raw.file_fixed.txt
R_0000_00_02_00_45_32_xxxx_x0-0000-100-x0.0_xxxx_xxxx_xxxx_xxxx_xxxx_xxxx

awk
Code:
awk 'NR==FNR {id[$2]; next} $2 in id' f1 f2 | awk '1;$NF=/$[id]/{ print "/test/id/$[id]_raw.file_fixed.txt"}' > out

# 2  
Old 04-13-2019
I'm not sure that I am following what you are trying to do.

We can easily produce the output you say you want from your two sample input files with just:
Code:
awk 'FNR == NR{print; next} NF == 1' f2 f1

or even:
Code:
awk 'FNR == NR; NF == 1' f2 f1

This User Gave Thanks to Don Cragun For This Post:
cmccabe
# 3  
Old 04-14-2019
Hi,

Try:
Code:
awk 'NR==FNR {id[$2]=$3; next} $2 in id{$3=id[$2] "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1


But you wrote:

Quote:
Originally Posted by cmccabe
[..]I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. [..]
So in that case you may need to try something like:
Code:
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$3; next} $2 in id{$3=id[$2] "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1

This User Gave Thanks to Scrutinizer For This Post:
cmccabe
# 4  
Old 04-16-2019
I made an typo in f2
Should just be one column of multiple strings.
Code:
/path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449 
/path/to/the/xxx/data/00-0000_xxxx-03_v1/00-0000_xxxx-03_v1_20190322115521953
/path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953

So I thought i adjusted the script below correctly to capture the partial match. That is the $2 will be in that long string. Thank you Smilie.

Awk
Code:
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$2]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$i]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1

Last edited by cmccabe; 04-17-2019 at 12:23 PM.. Reason: fixed format, updated awk
# 5  
Old 04-19-2019
Quote:
Originally Posted by cmccabe
I made an typo in f2
Should just be one column of multiple strings.
Code:
/path/to/the/xxx/data/0000-yyyy_v1_0000-yyyy_RNA_v1/190326-Control_v1_20190328071906449 
/path/to/the/xxx/data/00-0000_xxxx-03_v1/00-0000_xxxx-03_v1_20190322115521953
/path/to/the/xxx/data/1111-xxxx-03_v1/1111-xxxx-03_v1_20190322115521953

So I thought i adjusted the script below correctly to capture the partial match. That is the $2 will be in that long string. Thank you Smilie.

Awk
Code:
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$2]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1
awk 'NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next} $2 in id{$3=id/[$i]/ "/test/id/" $1 "_raw.file_fixed.txt"}1' f2 f1

It still isn't clear to me what you are trying to do, but I assume that the above code is giving errors for trying to divide by an element of an unnamed array and then trying to divide the result of that by a non-numeric string. But, that problem may be hidden by the fact that no value in $2 in f1 (i.e., 0000-yyyy, 1111-xxxx, or the empty string from the 3rd line in f1) ever appears as a field in f2. And, therefore, the condition $2 in id is never true.

And, since there is only one field in every line in f2, the only array subscripts in the id[] array are complete lines from f2. Why run a loop from 1 through NF when NF is always 1?

Since you say that the corrected f2 only contains one string, the first part of the code in both awk scripts above:
Code:
NR==FNR {for(i=1; i<=NF; i++) id[$i]=$1; next}

could more simply be written as:
Code:
NR==FNR {id[$0]=$0; next}

or as:
Code:
NR==FNR {id[$1]=$1

which would both produce exactly the same id[] arrays.

Since there are no matching lines, I still can't make any sense of the description of what you are trying to combine from matching lines in the two files???
These 2 Users Gave Thanks to Don Cragun For This Post:
Neo rbatte1
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to update file with sum of matching fields in another file

In the awk below I am trying to add a penalty to a score to each matching $1 in file2 based on the sum of $3+$4 (variable TL) from file1. Then the $4 value in file1 is divided by TL and multiplied by 100 (this valvue is variable S). Finally, $2 in file2 - S gives the updated $2 result in file2.... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Append Text File from another File in line 6

Hi, Anyone can help on how to append a file1.txt into file2.txt after line 3 using sed command. file1.txt 1. testa 2. testb 3. testc 4. testd 5. test5 6. test6 file2.txt break here this is a test break end here output 1. testa 2. testb (1 Reply)
Discussion started by: fspalero
1 Replies

4. Shell Programming and Scripting

Needed shell script to append desired text to each line in a file

Hi, I had generated a report in my tool as followsoutput.txt 43.35 9 i needed the script to generate a new file like below i want to append the text to each of these lines of my filenewoutputfile.txt should be Total Amount : 43.35 Record Count:9 Regards, Vasa Saikumar. ... (2 Replies)
Discussion started by: hemanthsaikumar
2 Replies

5. Shell Programming and Scripting

Update specific field in a line of text file

I have a text file like this: subject1:LecturerA:10 subject2:LecturerA:40 if I was given string in column 1 and 2 (which are subject 1 and LecturerA) , i need to update 3rd field of that line containing that given string , which is, number 10 need to be updated to 100 ,for example. The... (6 Replies)
Discussion started by: bmtoan
6 Replies

6. UNIX for Dummies Questions & Answers

Append a line to single column text file

I would like to add a line to the end of a single column text file. How do I go about doing that? Input: BEGIN 1 2 3 Output: BEGIN 1 2 3 END Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. Shell Programming and Scripting

AWK - Print partial line/partial field

Hello, this is probably a simple request but I've been toying with it for a while. I have a large list of devices and commands that were run with a script, now I have lines such as: a-router-hostname-C#show ver I want to print everything up to (and excluding) the # and everything after it... (3 Replies)
Discussion started by: ippy98
3 Replies

8. Shell Programming and Scripting

append delimeter count for each line in text file

Hi guys, plz tell me how to achieve this how to delete the lines in a file using sed command (6 Replies)
Discussion started by: hari908
6 Replies

9. Shell Programming and Scripting

How to append text to the second line of a file

Say I have a text file like: 1 3 4 How would I use ksh to put the number '2' into the second line of that file? I'm using OpenBSD so the sed syntax might be a bit different (I have no idea how to use sed, though) (4 Replies)
Discussion started by: guitarscn
4 Replies

10. Shell Programming and Scripting

Append text at end of the first line in a file

Hi I need to append some text @ end of the first line in a file. like myfile.txt list = a,b,c list.a=some.. I give the arg "d" . now it append at end of first line list=a,b,c,d list.a=some... Please help me out this (7 Replies)
Discussion started by: catgovind
7 Replies
Login or Register to Ask a Question