Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk second largest, third largest value

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk second largest, third largest value
# 1  
Old 09-13-2011
awk second largest, third largest value
I have two text files like this:
file1.txt:

Code:
133 10  
133 22
133 13
133 56
133 78
133 98

file2.txt:

Code:
158 38
158 67 
158 94
158 17
158  23

I'm basically trying to have awk check the second largest value of the second column of each text file, and cat it to its own text file. There are 27 text files, my idea is to use perl to loop awk through each one, but i use backticks in every loop, hence it's quite slow.

Afterwards it needs to find the third largest value of the text files and cat it to its own text file as well. And then fourth largest, 5th largest etc.


THanks for the help guys, very much appreciated.

EDIT: Net result would look like this for second largest values

Code:
133  78
158  67
...
...

Last edited by radoulov; 09-13-2011 at 05:28 PM.. Reason: Code tags.
# 2  
Old 09-13-2011
you can simply do a numeric sort on second column of your file and then your lines are arranged in required order.
# 3  
Old 09-13-2011
Code:
#!/bin/sh

DEPTH=4
while [ $# -gt 0 ]
do
        N=0
        sort -r -n -k 2 < "$1" |
        while read LINE
        do
                echo "$LINE" >> "$N.txt"
                N=`expr $N + 1`
                [ "$N" -ge "$DEPTH" ] && break
        done

        shift
done

Code:
$ ./asort.sh file*.txt
$ cat 0.txt
133 98
158 94
$ cat 1.txt
133 78
158 67
$ cat 2.txt
133 56
158 38
$ cat 3.txt
133 22
158 23
$

# 4  
Old 09-13-2011
my problem has gotten a bit stickier after sorting

my text files actually look like:

Code:
133 800         
133 799
133 798
133 403
133 402
133 401

I'm looking to simply read the max value of 800, and then ignore values that are within 5 of it, and then read for the second largest value (403 in this case) and then cat these to their own files.

sorry for the mixup!! and thanks for the help so far!

Moderator's Comments:
Mod Comment Use code tags!
Last edited by radoulov; 09-13-2011 at 05:27 PM..
# 5  
Old 09-13-2011
assumed that you have sorted the file. based on your example data:
Code:
kent$  echo "133 800         
133 799
133 798
133 403
133 402
133 401"|awk 'NR==1{max=$2;next;}max-$2>5{print $0;exit;}'

133 403

if you want to put the line into a new file:

awk 'NR==1{max=$2;next;}max-$2>5{print $0 >> "2ndLargest.txt";exit;}' inputFile

This User Gave Thanks to sk1418 For This Post:
theawknewbie
# 6  
Old 09-13-2011
Thanks for that last reply! That really did it for 2nd max, but what can I do for a 3rd largest max with the same "max - $2>5" idea??
It's sort of like this (continued)

Code:
133 800 
         133 799 
133 798 
133 403 
133 402 
133 401
133 127
133 126
133 125

pseudocode: select real max;print $0;exit && select second max; print$0;exit && select thirdmax;print$0 exit >> newfile.txt

eventually to obtain: cat newfile.txt
Code:
133 800
133 403
133 127

I'm stuck here because the next function seems built in here to find a second max, can I store the second max in a variable and tell it to 'next' that record to find the 3rd maximum? Any tips would be well appreciated. Thanks!
Last edited by theawknewbie; 09-13-2011 at 10:27 PM..
# 7  
Old 09-14-2011
this gives your 2nd and 3rd largest values:
Code:
 echo "133 800         
dquote> 133 799
dquote> 133 798
dquote> 133 403
dquote> 133 402
dquote> 133 401"|awk 'NR==1{max=$2;next;}max-$2>5 && !second{print $0;second=$2;}$2<second{print $0;exit;}'

133 403
133 402

output to a file is omitted.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

In the below I am trying to use awk to match all the $13 values in input, which is tab-delimited, that are in $1 of gene which is just a single column of text. However only the line with the greatest $9 value in input needs to be printed. So in the example below all the MECP2 and LTBP1... (0 Replies)
Discussion started by: cmccabe
0 Replies

2. Shell Programming and Scripting

Largest number in array. Help!

I need to calculate the biggest number in array size n. Example: Users enter: 1 7 4 9 The biggest number is : 9 Simple but I'm really new on this on Shell/Bash! Anything will be helpful! Thanks! #!/bin/bash printf "\tEnter a list of numbers, with spaces: " read -a ARRAY BIG=$1... (5 Replies)
Discussion started by: Sundown
5 Replies

3. UNIX for Dummies Questions & Answers

Printf awk: argument to adjust automatically width to the largest value

Hi ! Here is an example of a tab-delimited file with variable width values in field 2 (space separated values). As the real file contains lot of lines, I can adjust manually the width to the largest value in field 2. Does it exist an argument to do it automatically? input.tab:... (4 Replies)
Discussion started by: lucasvs
4 Replies

4. Shell Programming and Scripting

Help with keep the largest record in file

Input file US Score 10 UK Ball 20 AS Score 50 AK Ball 10 PZ Ballon 50 PA Score 70 WT Data 10 . . Desired output file UK Ball 20 PZ Ballon 50 PA Score 70 WT Data 10 . . (1 Reply)
Discussion started by: perl_beginner
1 Replies

5. Shell Programming and Scripting

AWK (how) to get smallest/largest nr of ls -la

Hey, This is a long-shot however, I am stuck with the following problem: I have the output from ls -la, and I want to sort some of that data out by using AWK to filter it. ls -la | awk -f scriptname.awk Input: For example: drwxr-xr-x 3 user users 4096 2010-03-14 20:15 bin/... (5 Replies)
Discussion started by: abciscool
5 Replies

6. UNIX for Dummies Questions & Answers

largest file

My directory only contains files..no sub directories..I want to know about the largest file in directory..Largest in terms of size..And what if I want to know about first 3 largest files.. Is it possible to know largest file in terms of its filename?? (4 Replies)
Discussion started by: aadi_uni
4 Replies

7. Shell Programming and Scripting

Find largest files

Hello, i am on linux 2.6.13-1.1526_FC4smp , and i would be very greatfull if someone could help to find the largest files on that server. I know i can find files with find command or list them with ls , but is there a way that i could list let's say 10 biggest files on server ? :o (11 Replies)
Discussion started by: tonijel
11 Replies

8. Shell Programming and Scripting

largest field , awk , help

Hi All, My file is like this: $ cat max.txt abcd:1982:a efghij:1980:e klmn:1923:k opqrst:1982:o I have to find out the largest first field and the corresponding line. i.e Output required: efghij efghij:1980:e opqrst opqrst:1982:o HTH, jkl_jkl (6 Replies)
Discussion started by: jkl_jkl
6 Replies

9. Shell Programming and Scripting

largest value from within a file

hi, i am having a file that looks like this : myhome111 oper 11 myhome333 oper 19 ... how can i get the largest value from this file ..in this example i am consiering 19 as the largest value.. how can i do that.. as this file is having other numbers also within it like 111, 333... (4 Replies)
Discussion started by: kripssmart
4 Replies

10. Shell Programming and Scripting

find largest file

Hi, 1)I have XX directory and have lot of files ,I want to find largest file in that directory 2)how calculate the size of file in MB. Thanks, Mohan (15 Replies)
Discussion started by: mohan705
15 Replies
Login or Register to Ask a Question