awk to add text to each line of matching id


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to add text to each line of matching id
Prev   Next
# 1  
Old 01-18-2019
awk to add text to each line of matching id

The awk below executes as expected if the id in $4 (like in f) is unique. However most of my data is like f1 where the same id can appear multiple times. I think that is the reason why the awk is not working as expected. I added a comment on the line that I can not change without causing the script to abort. Each line in f2 is searched and must contain the id, in this case COL1A2 but that id may not be a single entry. That is the id may appear 5 times, but each line with that is in f2 is searched. Using the $4 in f1 as the id and reading each $1, $2, and $3 value into a variable min and max.

The $4 is then split on the _ in f2 and read into array. The same id from f1 may appear in multiple lines of f2 however each will have unique $2 and $3 values. Each value in the split will match a $4 id in f1. The min and max must match the $1 of f2 and be between the $2 and $3 values in f2. An exact match is not needed rather just that the min or max variables falls within $2 and $3. If that is true then exon is printed in $5 of f2 if it is not true then intron is printed in $5. Most of this works as expected I just did not account for the possibity for multiple enteries and am nut sure how to adjust for it. Thank you Smilie

For example using the contents of the f1 where COL1A2 appears 3 times, each entry or line is searched in f2. Currently, I believe since COL1A2 is not unique not match is found in f2 as the min and max are not set per entry or line. Thank you Smilie.

awk w/ desired output
Code:
awk '
 BEGIN{
  SUBSEP=","
}
FNR==NR{
  max[$1,$NF]=$3
  min[$1,$NF]=$2
  next
}
{
 split($4,array,"_")   # How do I change/modify this so it only looks a each line with this id `COL1A2` in it?
}
(($1,array[1]) in max){
if(($2>min[array[5],array[1]] && $2<max[array[5],array[1]]) || ($3>max[array[5],array[1]] && $3<max[array[5],array[1]])){
  print array[5],array[1],min[array[5],array[1]],max[array[5],array[1]],"exon"
  next
}
}
{
  print $0,"intron"}' f f2 

chr7    94024333    94024423    COL1A2_cds_0_0_chr7_94024344_f  0   + intron
chr7    94027049    94027080    COL1A2_cds_1_0_chr7_94027060_f  0   + intron
chr7 COL1A2 94027591 94027701 exon

awk w/ current output
Code:
  .... }' f1 f2

chr7    94024333    94024423    COL1A2_cds_0_0_chr7_94024344_f  0   + intron
chr7    94027049    94027080    COL1A2_cds_1_0_chr7_94027060_f  0   + intron
chr7    94027683    94027718    COL1A2_cds_2_0_chr7_94027694_f  0   + intron

Code:
contents of f single COL1A2 entry

chr7    94027591    94027701    COL1A2

Code:
contents of f1  multiple COL1A2 entry, this is most of the actual data, very few are single entries though there are some

chr7    94027591    94027701    COL1A2  
chr7    94027799    94027811    COL1A2  
chr7    94030799    94030847    COL1A2

Code:
contents of f2 always the same format

chr7    94024333    94024423    COL1A2_cds_0_0_chr7_94024344_f  0   +
chr7    94027049    94027080    COL1A2_cds_1_0_chr7_94027060_f  0   +
chr7    94027683    94027718    COL1A2_cds_2_0_chr7_94027694_f  0   +

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk to update file with partial matching line in another file and append text

In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is much larger (several hundreds of lines) in both f1 and f2. When the line in f2 is pasted to $3 in... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

awk to add text to matching pattern in field

In the awk I am trying to add :p.=? to the end of each $9 that matches the pattern NM_. The below executes andis close but I can not seem to figure out why the :p.=? repeats in the split as in the green in the current output. I have added comments as well. Thank you :). file ... (4 Replies)
Discussion started by: cmccabe
4 Replies

3. Shell Programming and Scripting

awk to lookup stored variable in file and print matching line

The bash bash below extracts the oldest folder from a directory and stores it in filename That result will match a line in bold in input. In the matching line there is an_xxx digit in italics that (once the leading zero is removed) will match a line in link. That is the lint to print in output.... (2 Replies)
Discussion started by: cmccabe
2 Replies

4. Shell Programming and Scripting

awk to copy previous line matching a particular columns

Hello Help, 2356798 7689867 999 000 123678 20385907 9797 666 17978975 87468976 968978 98798 I am trying to have out put which actually look for the third column value of 9797 and then it insert line there after with first, second column value exactly as the previous line and replace the third... (3 Replies)
Discussion started by: Indra2011
3 Replies

5. Shell Programming and Scripting

Matching and printing line with awk

Hi there, I'm trying to use awk to print out the entire line that contains a match to a certain regex and then append some text,plus the match to the end of the line. So far I have: awk -F: '{print "RG:Z:" $2}' file Which prints out the match I want plus the additional text, but I'm stuck... (3 Replies)
Discussion started by: jim_lad
3 Replies

6. Shell Programming and Scripting

Replace and add line in file with line in another file based on matching string

Hi, I want to achieve something similar to what described in another post: The difference is I want to add the line if the pattern is not found. File 1: A123, valueA, valueB B234, valueA, valueB C345, valueA, valueB D456, valueA, valueB E567, valueA, valueB F678, valueA, valueB ... (11 Replies)
Discussion started by: jyu3
11 Replies

7. Shell Programming and Scripting

sed or awk delete character in the lines before and after the matching line

Sample file: This is line one, this is another line, this is the PRIMARY INDEX line l ; This is another line The command should find the line with “PRIMARY INDEX” and remove the last character from the line preceding it (in this case , comma) and remove the first character from the line... (5 Replies)
Discussion started by: KC_Rules
5 Replies

8. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Dear All, I would like to add values of a field, if the lines match in a certain field. Then I would like to divide the sum though the number of lines that have a matched field. This is the Input: Input: Test1 5 Test1 10 Test2 2 Test2 5 Test2 13 Test3 4 Output: Test1 7.5 Test1 7.5... (6 Replies)
Discussion started by: DerSeb
6 Replies

9. Shell Programming and Scripting

using command line arguments as columns for pattern matching using awk

Hi, I wish to use a column, as inputted by a user from command line, for pattern matching. awk file: { if($1 ~ /^8/) { print $0> "temp2.csv" } } something like this, but i want '$1' to be any column as selected by the user from command line. ... (1 Reply)
Discussion started by: invinclible0009
1 Replies

10. UNIX for Dummies Questions & Answers

how to delete line with matching text and line immediately after

hello experts, I have a file: File1 Sample Test1 This is a Test Sample Test2 Another Test Final Test3 A final Test I can use sed to delete the line with specific text ie: sed '/Test2/d' File1.txt > File2.txt How can I delete the line with the matching text and the line immediately... (6 Replies)
Discussion started by: orahi001
6 Replies
Login or Register to Ask a Question