Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Filtering based on column values

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Filtering based on column values
# 8  
Old 05-27-2017
Would this work?
Code:
perl -nle '/SF=(\d{3})/; print if $1 > 400 or $. == 1' daashti.example

# 9  
Old 05-27-2017
Try
Code:
awk '{match ($0, ";SF=[^,]*,"); VAL = substr($0, RSTART+4, RLENGTH-4)} FNR == 1 || VAL > 400' file

# 10  
Old 05-27-2017
Code:
awk '{match ($0, ";SF=[^,]*,"); VAL = substr($0, RSTART+4, RLENGTH-4)} FNR == 1 || VAL > 400' file

This didn't give the desired outcome.

and

Code:
perl -nle '/SF=(\d{3})/; print if $1 > 400 or $. == 1' daashti.example

gave me only the header
# 11  
Old 05-27-2017
Quote:
Originally Posted by daashti
Code:
awk '{match ($0, ";SF=[^,]*,"); VAL = substr($0, RSTART+4, RLENGTH-4)} FNR == 1 || VAL > 400' file

This didn't give the desired outcome.

and

Code:
perl -nle '/SF=(\d{3})/; print if $1 > 400 or $. == 1' daashti.example

gave me only the header
Hi daashti,
If you got only the header, using the Perl option, it means your input is not as advertised.
The command would had picked any pattern SF= followed by 3 digits and evaluated the digits as a number. If greater that 400 it would had output the whole line.

May I ask if you could post the result of running this command from the original input?
Code:
perl -nle '/(SF=\d+)/ and print $1' real.file

When I run that with the posted example, it returns the following:
Code:
SF=341
SF=348
SF=606
SF=372

# 12  
Old 05-27-2017
Thanks for your reply, real file is very similar.
I do need the a whole line/row when filtering not only SF=values alone i.e SF values

I tried this
Code:
perl -nle '/(SF=\d+) and print $1' real.file

and got this
Code:
Search pattern not terminated at -e line 1.

not very familiar with perl.
# 13  
Old 05-27-2017
Quote:
Originally Posted by daashti
Thanks for your reply, real file is very similar.
I do need the a whole line/row when filtering not only SF=values alone i.e SF values

I tried this
Code:
perl -nle '/(SF=\d+)/ and print $1' real.file

and got this
Code:
Search pattern not terminated at -e line 1.

not very familiar with perl.
Add the closing pattern search slash (highlighted)
# 14  
Old 05-27-2017
I will clean the mess and put the example in a better format.

Data
Code:
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO	FORMAT
chr1	65872	.	T	G	2480.51	.	AC=65;AN=92;SF=0,1;VRT=1	GT:GQ:DP:AD:PL
chr1	65893	.	G	A	433.77	.	AC=3;AN=4;SF=0,1;VRT=1	GT:GQ:PL:AD:DP
chr1	65898	.	T	C	2055.11	.	AC=7;AN=10;SF=0,1;VRT=1	GT:DP:AD:PL:GQ
chr1	69511	rs75062661	A	G	822333	.	AC=894;AN=934;SF=0,1;VRT=1	GT:GQ:DP:AD:PL
chr19	9088447	9088447	T	C	2420.41	.	AC=5;AN=10;SF=441,477,517,643,662;VRT=1	GT:GQ:DP:AD:PL
chr19	9072886	9072886	G	C	245.11	.	AC=5;AN=10;SF=448,498,500,741,778;VRT=1	GT:GQ:PL:AD:DP
chr7	149484550	149484550	G	A	305.11	.	AC=5;AN=10;SF=606,704,723,736,763;VRT=1	GT:DP:AD:PL:GQ
chr7	149482784	149482784	C	T	822	.	AC=5;AN=10;SF=472,478,684,708,799;VRT=1	GT:GQ:DP:AD:PL

After filtering
Code:
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO	FORMAT
chr19	9088447	9088447	T	C	2420.41	.	AC=5;AN=10;SF=441,477,517,643,662;VRT=1	GT:GQ:DP:AD:PL
chr19	9072886	9072886	G	C	245.11	.	AC=5;AN=10;SF=448,498,500,741,778;VRT=1	GT:GQ:PL:AD:DP
chr7	149484550	149484550	G	A	305.11	.	AC=5;AN=10;SF=606,704,723,736,763;VRT=1	GT:DP:AD:PL:GQ
chr7	149482784	149482784	C	T	822	.	AC=5;AN=10;SF=472,478,684,708,799;VRT=1	GT:GQ:DP:AD:PL
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies

2. Shell Programming and Scripting

Concatenate values in the first column based on the second column.

I have a file (myfile.txt) with contents like this: 1.txt apple is 3.txt apple is 5.txt apple is 2.txt apple is a 7.txt apple is a 8.txt apple is a fruit 4.txt orange not a fruit 6.txt zero isThe above file is already sorted using this command: sort -k2 myfile.txtMy objective is to get... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

3. UNIX for Dummies Questions & Answers

Repositioning based on column values

Dear all ... I have a file which I want to change the structure based on the values in some columns and I would be grateful if you can help... one of my files looks like ... they all have ten rows 1,0,0 10,0,0 2,0,0 3,0,0 4,1,1 4,1,1 4,1,1 5,0,0 6,0,0 7,0,0 8,0.5,2 9,0.33,3 9,0.33,3... (1 Reply)
Discussion started by: A-V
1 Replies

4. Shell Programming and Scripting

Sum column values based in common identifier in 1st column.

Hi, I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column) The input is for example, after sorted: K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies

5. Shell Programming and Scripting

Adding values of a column based on another column

Hello, I have a data such as this: ENSGALG00000000189 329 G A 4 2 0 ENSGALG00000000189 518 T C 5 1 0 ENSGALG00000000189 1104 G A 5 1 0 ENSGALG00000000187 3687 G T 5 1 0 ENSGALG00000000187 4533 A T 4 2 0 ENSGALG00000000233 5811 T C 4 2 0 ENSGALG00000000233 5998 C A 5 1 0 I want to... (3 Replies)
Discussion started by: Homa
3 Replies

6. Shell Programming and Scripting

Filtering lines for column elements based on corresponding counts in another column

Hi, I have a file like this ACC 2 2 21 aaa AC 443 3 22 aaa GCT 76 1 33 xxx TCG 34 2 33 aaa ACGT 33 1 22 ggg TTC 99 3 44 wee CCA 33 2 33 ggg AAC 1 3 55 ddd TTG 10 1 22 ddd TTGC 98 3 22 ddd GCT 23 1 21 sds GTC 23 4 32 sds ACGT 32 2 33 vvv CGT 11 2 33 eee CCC 87 2 44... (1 Reply)
Discussion started by: polsum
1 Replies

7. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

8. Shell Programming and Scripting

How to averaging column based on first column values

Hello I have file that consist of 2 columns of millions of entries timestamp and throughput I want to find the average (throughput ) for each equal timestamp before change it to proper format e.g : i want to average 2 coloumnd fot all 1308154800 values in column 1 and then print... (4 Replies)
Discussion started by: aadel
4 Replies

9. Shell Programming and Scripting

How to pick values from column based on key values by usin AWK

Dear Guyz:) I have 2 different input files like this. I would like to pick the values or letters from the inputfile2 based on inputfile1 keys (A,F,N,X,Z). I have done similar task by using awk but in that case the inputfiles are similar like in inputfile2 (all keys in 1st column and values in... (16 Replies)
Discussion started by: repinementer
16 Replies

10. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies
Login or Register to Ask a Question