Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Filtering based on column values

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Filtering based on column values
# 15  
Old 05-27-2017
The original Perl command gives me the same result you want when run against your latest example.
Code:
perl -nle '/SF=(\d{3})/; print if $1 > 400 or $. == 1' real.file
#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT
chr19   9088447 9088447 T       C       2420.41 .       AC=5;AN=10;SF=441,477,517,643,662;VRT=1 GT:GQ:DP:AD:PL
chr19   9072886 9072886 G       C       245.11  .       AC=5;AN=10;SF=448,498,500,741,778;VRT=1 GT:GQ:PL:AD:DP
chr7    149484550       149484550       G       A       305.11  .       AC=5;AN=10;SF=606,704,723,736,763;VRT=1 GT:DP:AD:PL:GQ
chr7    149482784       149482784       C       T       822     .       AC=5;AN=10;SF=472,478,684,708,799;VRT=1 GT:GQ:DP:AD:PL

If you only want the number as you mentioned in post #12, then:

Code:
perl -nle '/SF=(\d+)/; print $1 if $1 > 400' real.file
441
448
606
472

# 16  
Old 05-27-2017
SF=are not only 3 digits
its random and it goes from 0-818
e.g
SF=0,10,35,55,300,455,654
SF=754,789,790,800
# 17  
Old 05-27-2017
Quote:
Originally Posted by daashti
SF=are not only 3 digits
its random and it goes from 0-818
e.g
SF=0,10,35,55,300,455,654
SF=754,789,790,800
Sure, but any value greater than 400 would be always 3 digits (or more) and if the maximum number is 818 a three digit is sufficient.
A variation, if the maximum number could be arbitrary.
Code:
perl -nle '/SF=(\d+)/; print if $1 > 400 or $. == 1' real.file

Question:

Are you trying to filter if ANY of the numbers in the string SF=0,10,35,55,300,455,654 is greater than 400?
Last edited by Aia; 05-27-2017 at 12:17 PM..
This User Gave Thanks to Aia For This Post:
daashti
# 18  
Old 05-27-2017
yes i want lines/rows with SF=values larger than 400
# 19  
Old 05-27-2017
So this brings us back to the question that should have been raised right in the beginning: What OS, shell, and awk version are you running?
Because, taking your last example from post#14, I see
Code:
awk '{match ($0, ";SF=[^,]*,"); VAL = substr($0, RSTART+4, RLENGTH-4)} FNR == 1 || VAL > 400' file 
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO	FORMAT
chr19	9088447	9088447	T	C	2420.41	.	AC=5;AN=10;SF=441,477,517,643,662;VRT=1	GT:GQ:DP:AD:PL
chr19	9072886	9072886	G	C	245.11	.	AC=5;AN=10;SF=448,498,500,741,778;VRT=1	GT:GQ:PL:AD:DP
chr7	149484550	149484550	G	A	305.11	.	AC=5;AN=10;SF=606,704,723,736,763;VRT=1	GT:DP:AD:PL:GQ
chr7	149482784	149482784	C	T	822	.	AC=5;AN=10;SF=472,478,684,708,799;VRT=1	GT:GQ:DP:AD:PL

- EXACTLY what you required!

(Aside - as did awk 'FNR == 1 || $6 > 400' FS="[,;=]" file yield...)
(Next aside: as did Aia's perl code do!)
Last edited by RudiC; 05-27-2017 at 12:48 PM.. Reason: typo
# 20  
Old 05-27-2017
Hi RudiC

thanks for input RudiC,

i used
Code:
awk '{match ($0, ";SF=[^,]*,"); VAL = substr($0, RSTART+4, RLENGTH-4)} FNR == 1 || VAL > 400' file

i got no output at all apart from header

My OS is Ubuntu 3.4.0+ #1 PREEMPT Thu Aug 1 17:06:05 CST 2013 x86_64 x86_64 x86_64 GNU/Linux
Awk version
GNU Awk 4.0.1
Shell
bash 4.3.11(1)-release
# 21  
Old 05-27-2017
Please post the output of
- awk ... print VAL
- od -ctx1 file
where file is a small but representative subset of your data.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies

2. Shell Programming and Scripting

Concatenate values in the first column based on the second column.

I have a file (myfile.txt) with contents like this: 1.txt apple is 3.txt apple is 5.txt apple is 2.txt apple is a 7.txt apple is a 8.txt apple is a fruit 4.txt orange not a fruit 6.txt zero isThe above file is already sorted using this command: sort -k2 myfile.txtMy objective is to get... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

3. UNIX for Dummies Questions & Answers

Repositioning based on column values

Dear all ... I have a file which I want to change the structure based on the values in some columns and I would be grateful if you can help... one of my files looks like ... they all have ten rows 1,0,0 10,0,0 2,0,0 3,0,0 4,1,1 4,1,1 4,1,1 5,0,0 6,0,0 7,0,0 8,0.5,2 9,0.33,3 9,0.33,3... (1 Reply)
Discussion started by: A-V
1 Replies

4. Shell Programming and Scripting

Sum column values based in common identifier in 1st column.

Hi, I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column) The input is for example, after sorted: K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies

5. Shell Programming and Scripting

Adding values of a column based on another column

Hello, I have a data such as this: ENSGALG00000000189 329 G A 4 2 0 ENSGALG00000000189 518 T C 5 1 0 ENSGALG00000000189 1104 G A 5 1 0 ENSGALG00000000187 3687 G T 5 1 0 ENSGALG00000000187 4533 A T 4 2 0 ENSGALG00000000233 5811 T C 4 2 0 ENSGALG00000000233 5998 C A 5 1 0 I want to... (3 Replies)
Discussion started by: Homa
3 Replies

6. Shell Programming and Scripting

Filtering lines for column elements based on corresponding counts in another column

Hi, I have a file like this ACC 2 2 21 aaa AC 443 3 22 aaa GCT 76 1 33 xxx TCG 34 2 33 aaa ACGT 33 1 22 ggg TTC 99 3 44 wee CCA 33 2 33 ggg AAC 1 3 55 ddd TTG 10 1 22 ddd TTGC 98 3 22 ddd GCT 23 1 21 sds GTC 23 4 32 sds ACGT 32 2 33 vvv CGT 11 2 33 eee CCC 87 2 44... (1 Reply)
Discussion started by: polsum
1 Replies

7. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

8. Shell Programming and Scripting

How to averaging column based on first column values

Hello I have file that consist of 2 columns of millions of entries timestamp and throughput I want to find the average (throughput ) for each equal timestamp before change it to proper format e.g : i want to average 2 coloumnd fot all 1308154800 values in column 1 and then print... (4 Replies)
Discussion started by: aadel
4 Replies

9. Shell Programming and Scripting

How to pick values from column based on key values by usin AWK

Dear Guyz:) I have 2 different input files like this. I would like to pick the values or letters from the inputfile2 based on inputfile1 keys (A,F,N,X,Z). I have done similar task by using awk but in that case the inputfiles are similar like in inputfile2 (all keys in 1st column and values in... (16 Replies)
Discussion started by: repinementer
16 Replies

10. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies
Login or Register to Ask a Question