awk to remove lines where field count is greather than 1 in two fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to remove lines where field count is greather than 1 in two fields
# 1  
Old 01-09-2017
awk to remove lines where field count is greather than 1 in two fields

I am trying to remove all the lines and spaces where the count in $4 or $5 is greater than 1 (more than 1 letter). The file and the output are tab-delimited. Thank you Smilie.

file
Code:
X    5811530    .    G    C    NLGN4X
17    10544696    .    GA    G    MYH3
9    96439004    .    C    CCCTGCCTCCACCACA    PHF2
11    47463483    .    AG    A    RAPSN
15    58988898    .    A    C    AAAA
6    32006214    .    CCTG    C    CYP21A2

desired output
Code:
X    5811530    .    G    C    NLGN4X
15    58988898    .    A    C    AAAA

awk
Code:
awk -F'\t' -v OFS='\t' '(NR>1) && ($4 > 1 ) && ($5 > 1) ' file


Last edited by cmccabe; 01-09-2017 at 03:11 PM.. Reason: added details
# 2  
Old 01-09-2017
Try:
Code:
awk -F'\t' 'length($4) == 1 && length($5) == 1' file

Note that:
  • the condition $4 > 1 looks for a numeric value at the start of the 4th field evaluating to a number greater than 1 no matter how many characters are in the 4th input field,
  • there is no need to set OFS if you only print copies of some of your input lines without changing the contents of any of the lines you print, and
  • there is no need to use the condition NR>1 when there is no header line to discard.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to print lines based on text in field and value in two additional fields

In the awk below I am trying to print the entire line, along with the header row, if $2 is SNV or MNV or INDEL. If that condition is met or is true, and $3 is less than or equal to 0.05, then in $7 the sub pattern :GMAF= is found and the value after the = sign is checked. If that value is less than... (0 Replies)
Discussion started by: cmccabe
0 Replies

2. Shell Programming and Scripting

awk joining multiple lines based on field count

Hi Folks, I have a file with fields as follows which has last field in multiple lines. I would like to combine a line which has three fields with single field line for as shown in expected output. Please help. INPUT hname01 windows appnamec1eda_p1, ... (5 Replies)
Discussion started by: shunya
5 Replies

3. Shell Programming and Scripting

awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :). grep grep -v -F -f targets.bed sort_list grep -vFf sort_list targets awk awk -F, ' > FILENAME == ARGV {to_remove=1; next} > ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies

4. Shell Programming and Scripting

Count of unique lines in field 4

When I use the below awk to count the unique lines in $4 for the input it seems to work. The answer is 3 because $4 is only unique 3 times in all the entries. However, when I use the same on actual data I get 56,536 and I know the answer should be 56,548. My question is there a better way to... (8 Replies)
Discussion started by: cmccabe
8 Replies

5. UNIX for Dummies Questions & Answers

Remove zeros from first field, but print all fields

Hello Everyone, I've got a comma-delimited file that looks like this: 0012,123 ,456 ,05/12/2014 0123,525 ,286 ,05/12/2014 0456,791 ,300 ,05/12/2014 1095,759 ,300 ,05/12/2014 1344,576 ,292 ,05/12/2014 1558,551 ,283 ,05/12/2014 002183719, , ... (9 Replies)
Discussion started by: Scottie1954
9 Replies

6. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk... (6 Replies)
Discussion started by: 100bees
6 Replies

7. Shell Programming and Scripting

awk - count character count of fields

Hello All, I got a requirement when I was working with a file. Say the file has unloads of data from a table in the form 1|121|asda|434|thesi|2012|05|24| 1|343|unit|09|best|2012|11|5| I was put into a scenario where I need the field count in all the lines in that file. It was simply... (6 Replies)
Discussion started by: PikK45
6 Replies

8. UNIX for Dummies Questions & Answers

Count Fields with Quoted Field

Hi, I used to count number of fields using following command head -1 <filename> | awk -F"," '{print NF}' Now the scenario is the delimiter(comma) occurs inside one of the data field. How to ignore the comma inside data and consider only delimiter and count number of fields. The fields are... (1 Reply)
Discussion started by: ethanr100
1 Replies

9. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the... (1 Reply)
Discussion started by: rocket_dog
1 Replies

10. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Dear All, I would like to add values of a field, if the lines match in a certain field. Then I would like to divide the sum though the number of lines that have a matched field. This is the Input: Input: Test1 5 Test1 10 Test2 2 Test2 5 Test2 13 Test3 4 Output: Test1 7.5 Test1 7.5... (6 Replies)
Discussion started by: DerSeb
6 Replies
Login or Register to Ask a Question