Select lines in which column have value greater than some percent of total file lines


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Select lines in which column have value greater than some percent of total file lines
# 1  
Old 04-21-2011
Select lines in which column have value greater than some percent of total file lines

i have a file in following format
Code:
 1    32    3    
 4    6    4
 4    45    1    
 45    4    61    
 54    66    4    
 5    65    51
 56    65    1
 12  32  85

now here the total number of lines are 8(they vary each time)

Now i want to select only those lines in which the values in third column are having the value greater than 25% of the length .
i.e. the value in third column shud be greater than 2 here

so that selected lines will be

Code:
1    32    3    
4    6    4
45    4    61    
54    66    4    
5    65    51
12  32  85

i can use awk like
Code:
awk '{if(some condition) printf $1"\t"$2"\t"$3"\n"}'

but not getting how to get this condition exactly

Please help me out
Thanks in advance
# 2  
Old 04-21-2011
You did not specify which value represents the length so I just test > 2 as you said.

Code:
awk '$3 > 2 {printf("%-5s%-5s%-5s\n", $1, $2, $3)}' infile
1    32   3
4    6    4
45   4    61
54   66   4
5    65   51
12   32   85

# 3  
Old 04-21-2011
Actually the total number of lines present in the file will be the length
and we need to choose the column in some percents of it
# 4  
Old 04-21-2011
Code:
kent$ echo " 1    32    3    
dquote>  4    6    4
dquote>  4    45    1    
dquote>  45    4    61    
dquote>  54    66    4    
dquote>  5    65    51
dquote>  56    65    1
dquote>  12  32  85" |awk '{a[NR]=$3;b[NR]=$0} END{x=NR/4; for(i=1;i<=NR;i++)if (a[i]>x)print b[i]}'
 1    32    3    
 4    6    4
 45    4    61    
 54    66    4    
 5    65    51
 12  32  85

# 5  
Old 04-21-2011
no it wont work for every file
coz the length is nothing but number of lines in file
# 6  
Old 04-21-2011
Quote:
Originally Posted by vaibhavkorde
no it wont work for every file
coz the length is nothing but number of lines in file
have you tried my command?
i used NR, at the end, the NR indicates: how many lines in your file.

if you apply the command on a different file, it works as well, for example:
Code:
 echo "1    32    3    
4    6    4
45    4    61
54    66    4
5    65    51
12  32  85
1    32    3
4    6    4
45    4    61
54    66    4
5    65    51
12  32  85
1    32    3
4    6    4
45    4    61
54    66    4
5    65    51
12  32  85"|awk '{a[NR]=$3;b[NR]=$0} END{x=NR/4; for(i=1;i<=NR;i++)if (a[i]>x)print b[i]}'
45    4    61    
5    65    51
12  32  85
45    4    61    
5    65    51
12  32  85
45    4    61    
5    65    51
12  32  85

This User Gave Thanks to sk1418 For This Post:
# 7  
Old 04-21-2011
yeah it worked fine
thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to calculate total and percent off field in file

Trying to use awk to print the lines in file that have either REF or SNV in $3, add a header line, sort by $4 in numerical order. The below code does that already, but where I am stuck is on the last part where the total lines are counted and printed under Total_Targets, under Targets_less_than is... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

awk to select lines with maximum value of each record based on column value

Hello, I want to get the maximum value of each record separated by empty line based on the 3rd column of each row within each record? Input: A1 chr5D 634 7 82 707 A2 chr5D 637 6 82 713 A3 chr5D 637 5 82 713 A4 chr5D 626 1 82 704... (4 Replies)
Discussion started by: yifangt
4 Replies

3. Shell Programming and Scripting

awk - mixed for and if to select particular lines in a data file

Hi all, I am new to AWK and I am trying to solve a problem that is probably easy for an expert. Suppose I have the following data file input.txt: 20 35 43 20 23 54 20 62 21 20.5 43 12 20.5 33 11 20.5 89 87 21 33 20 21 22 21 21 56 87 I want to select from all lines having the... (4 Replies)
Discussion started by: naska
4 Replies

4. Shell Programming and Scripting

Select lines from a file based on a criteria

Hi I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies

5. UNIX for Dummies Questions & Answers

How to randomly select lines from a text file

I have a text file with 1000 lines, I want to randomly select 200 lines from it and print them as output. How do I go about doing that? Thanks! (7 Replies)
Discussion started by: evelibertine
7 Replies

6. Shell Programming and Scripting

quickest way to get the total number of lines in a file

i have a file that's about 2GB, i have to get the total number of lines in this file every 10 minutes. the interval is not an issue. i just need the proper, most efficient way to do this. any ideas? i got the following from another thread on this site, but: awk 'int(100*rand())%5<1'... (12 Replies)
Discussion started by: SkySmart
12 Replies

7. UNIX for Dummies Questions & Answers

Select only certain lines from file and mantain formatting

I want to take the below data, and have it output to file only the STMC#/(IP address) and the "there are X number of updates to install" lines for each machine. I know it's easy, but Im a beginner in BASH stuff, my solution would probably take way too many lines to do something easy.Thanks! ... (5 Replies)
Discussion started by: glev2005
5 Replies

8. Shell Programming and Scripting

total number of lines in a file

Hi , How about find the total number of lines in a file ? How can i do that with the "grep" command ? (9 Replies)
Discussion started by: Raynon
9 Replies

9. Shell Programming and Scripting

Find lines greater than 80 characters in a file

Hi, Can anyone please give me the grep command to find all the lines in a file that exceed 80 columns Thanks, gubbala (8 Replies)
Discussion started by: mrgubbala
8 Replies

10. Shell Programming and Scripting

Total of lines w/out header and footer incude for a file

I am trying to get a total number of tapes w/out headers or footers in a ERV file and append it to the file. For some reason I cannot get it to work. Any ideas? #!/bin/sh dat=`date +"%b%d_%Y"` + date +%b%d_%Y dat=Nov16_2006 tapemgr="/export/home/legato/tapemgr/rpts"... (1 Reply)
Discussion started by: gzs553
1 Replies
Login or Register to Ask a Question