awk to remove lines in file if specific field matches


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to remove lines in file if specific field matches
# 1  
Old 03-17-2016
awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you Smilie.

grep
Code:
grep -v -F -f targets.bed sort_list

Code:
grep -vFf sort_list targets

awk
Code:
awk -F, '
>   FILENAME == ARGV[1] {to_remove[$1]=1; next}
>   ! ($5 in to_remove) {print}
> ' sort_list targets

example:
sorted_list
Code:
AGRN
ABL
SCN1A

file2
Code:
chr1    955543  955763  chr1:955543-955763  AGRN-6|gc=75
chr1    957571  957852  chr1:957571-957852  AGRN-7|gc=61.2
chr1    970621  970740  chr1:970621-970740  BCR-8|gc=57.1
chr1    976035  976270  chr1:976035-976270  BCR-9|gc=74.5

desired output (AGRN removed as it is in file1)
Code:
chr1    970621  970740  chr1:970621-970740  BCR-8|gc=57.1
chr1    976035  976270  chr1:976035-976270  BCR-9|gc=74.5


Last edited by cmccabe; 03-17-2016 at 05:19 PM.. Reason: added additional grep
# 2  
Old 03-17-2016
Your second grep works although it may false trigger on substrings or other fields than $5.
For your awk, why do you set the field separator to , when there's not a single comma in your file? Try to split $5 on minus signs and use the first array element for your pattern.
This User Gave Thanks to RudiC For This Post:
# 3  
Old 03-17-2016
Thank you, I modified the awk. Thank you Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk with sed to combine lines and remove specific odd # pattern from line

In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There may be multiple lines in file but they are in the same format. The Filename_ID line is the last line... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

awk to remove lines where field count is greather than 1 in two fields

I am trying to remove all the lines and spaces where the count in $4 or $5 is greater than 1 (more than 1 letter). The file and the output are tab-delimited. Thank you :). file X 5811530 . G C NLGN4X 17 10544696 . GA G MYH3 9 96439004 . C ... (1 Reply)
Discussion started by: cmccabe
1 Replies

3. Shell Programming and Scripting

awk to output specific matches in file

Using the attached file, the below awk command results in the output below: I can not seem to produce the desired results and need some expert help. Thank you :). awk -F'' ' { id += $4 value += $5 occur++ } END{ printf "%-8s%8s%8s%8s\n", "Gene", "Targets", "Average Depth", "Average... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

Combine multiple lines in file based on specific field

Hi, I have an issue to combine multiple lines of a file. I have records as below. Fields are delimited by TAB. Each lines are ending with a new line char (\n) Input -------- ABC 123456 abcde 987 890456 7890 xyz ght gtuv ABC 5tyin 1234 789 ghty kuio ABC ghty jind 1234 678 ght ... (8 Replies)
Discussion started by: ratheesh2011
8 Replies

5. Shell Programming and Scripting

awk - remove row if specific field is empty/blank

I have this text.filecharles darwin sam delight george washington johnson culper darwin sam delight micheal jackson penny lite and would like to remove the row, if the first field is blank. so the result would be: result.filecharles darwin sam ... (4 Replies)
Discussion started by: charles33
4 Replies

6. Shell Programming and Scripting

NAWK to remove lines that matches a specific pattern

Hi, I have requirement that I need to split my input file into two files based on a search pattern "abc" For eg. my input file has below content abc defgh zyx I need file 1 with abc and file2 with defgh zyx I can use grep command to acheive this. But with grep I need... (8 Replies)
Discussion started by: sbhuvana20
8 Replies

7. Shell Programming and Scripting

awk to sum specific field when pattern matches

Trying to sum field #6 when field #2 matches string as follows: Input data: 2010-09-18-20.24.44.206117 UOWEXEC db2bp DB2XYZ hostname 1 2010-09-18-20.24.44.206117 UOWWAIT db2bp DB2XYZ hostname ... (3 Replies)
Discussion started by: ux4me
3 Replies

8. Shell Programming and Scripting

Displaying lines of a file where the second field matches a pattern

Howdy. I know this is most likely possible using sed or awk or grep, most likely a combination of them together, but how would one go about running a grep like command on a file where you only try to match your pattern to the second field in a line, space delimited? Example: You are... (3 Replies)
Discussion started by: LordJezoX
3 Replies

9. Shell Programming and Scripting

Sed or Awk to remove specific lines

I have searched the forum for this - forgive me if I missed a previous post. I have the following file: blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah alter table "informix".esc_acct add constraint (foreign key (fi_id) references "informix".fi ... (5 Replies)
Discussion started by: Shoeless_Mike
5 Replies

10. Shell Programming and Scripting

how to remove specific lines from a file

When restoring a file in my uninstall program I need to remove the lines I added to a file during the install. In between the file can be modified by the users. Assume file1 is as follow: xxx str2 xxxx ..... ...The Following lines containing str* have to be removed... xxx str1 xxxx xxx ... (17 Replies)
Discussion started by: bluemoon1
17 Replies
Login or Register to Ask a Question