Searching blocks of rows and returning maximum as output


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching blocks of rows and returning maximum as output
# 1  
Old 07-29-2008
Searching blocks of rows and returning maximum as output

Hello,

I am just getting starting with awk and wondering if anyone could help with the following problem.

I have a large file of data, 50,000 rows x 6 columns. I would like to search in blocks of 500 rows for a maximum value in a specific column and compile an output file that prints the complete line for each maximum per 500.

Example, using a 10row x 3column, requested output for blocks of 5 rows:

1 L 3.2
2 F 6.5
3 J 8.4
4 E 5.1
5 A 4.2
6 U 2.2
7 O 6.9
8 H 4.5
9 D 2.3
10 R 21.3

Required Output:

3 J 8.4
10 R 21.3

Any help would be most appreciated,

Rob
# 2  
Old 07-29-2008
Sorry, I have misunderstood the problem.
The following script, output the maximim value for EVERY columns for each block.
Please ignore this post.


Try and adapt the following script :
Code:
awk -v block=5 -v cols=3 '

function init_max(    i) {
   for (i=1; i<=cols; i++)
      max[i] = $i;
}

function set_max(    i) {
   for (i=1; i<=cols; i++)
      if ($i > max[i])
         max[i] = $i;
}

function print_max(    i, out) {
   for (i=1; i<=cols; i++)
      out = out max[i] OFS;
   print out;
}

{
   if (count++ == 0)
      init_max();
   set_max();
   if (count >= block) {
      print_max();
      count = 0;
   }
}

END {
   if (count > 0)
      print_max();
}

' inputfile

Inputfile:
Code:
1 L 3.2
2 F 6.5
3 J 8.4
4 E 5.1
5 A 4.2
6 U 2.2
7 O 6.9
8 H 4.5
9 D 2.3
10 R 21.3

Output:
Code:
5 L 8.4
10 U 21.3

Jean-Pierre.

Last edited by aigles; 07-30-2008 at 05:52 AM.. Reason: Ignore this post
# 3  
Old 07-29-2008
Code:
awk '$c > m { m = $c; r = $0 }
!(NR % n) { print r; m = 0 }
' c=3 n=5 filename


Last edited by radoulov; 07-29-2008 at 05:45 PM..
# 4  
Old 07-30-2008
Three paras, first one is the filename you will process on, second one is how many rows will be included in each block, third one is which column you will compare on.

Here assume the column you are comparing are all integer.

Code:
$file=shift;
$rows=shift;
$col=shift;
open(FH,"<$file");
while(<FH>){
	@arr=split(" ",$_);
	$t=$arr[$col-1];
	$index= ((index $./$rows,".")==-1)?$./$rows: (int $./$rows)+1;
	if($t>=$max){
		$max[$index]=$_;
		$max=$t;
	}
	if($.%$rows==0){
		$max=0;
	}
}
close(FH);
for($i=0;$i<=$#max;$i++){
	print $max[$i];
}

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Format output with blocks - awk

Format output with blocks - Awk ZONESET_A,DBP02_HBA0,DC01,20:00:00:25:b5:b9:a0:28,10,port-channel20,01-VB2 ZONESET_A,DBP02_HBA0,DC01,50:00:14:42:a0:72:74:00,10,fc4/30,O1-CORE-D ZONESET_A,dc01vb,DC01,20:00:00:25:b5:b0:a0:13,10,fc1/24,01-VB1... (1 Reply)
Discussion started by: greycells
1 Replies

2. UNIX for Dummies Questions & Answers

Maximum of the values from the output a query

Following is the output of a command ran inside the script: 2015-01-29-05-38-02 5 2015-01-29-05-38-02 5 2015-01-29-05-38-02 5 2015-01-29-05-38-02 5 2015-01-29-05-38-02 5 2015-01-29-05-38-03 5 2015-01-29-05-38-03 5 2015-01-29-05-38-03 5 2015-01-29-05-38-03 5 2015-01-29-05-38-03 5... (8 Replies)
Discussion started by: Devendra Hupri
8 Replies

3. Shell Programming and Scripting

Output minimum and maximum values for replicates ID

Hi All I hope that someone could help me! I have an input file like this, with 4 colum(ID, feature1, start, end): a x 1 5 b x 3 10 b x 4 9 b x 5 16 c x 5 9 c x 4 8 And my output file should be like this: a x 1 5 b x 3 16 c x 4 9 What I would like to do is to output for each ID... (2 Replies)
Discussion started by: giuliangiuseppe
2 Replies

4. UNIX for Dummies Questions & Answers

Searching a pattern and picking rows underneath them!

Hi, I have a requirement where i need to pick rows right below including the row where the search begins EX: USA CHINA INDIA 10 20 30 30 40 50 60 70 80 HON PAN SAN 50 60 70 80 90 100 So now...I need to get 2... (1 Reply)
Discussion started by: saggiboy10
1 Replies

5. Shell Programming and Scripting

Row blocks to column blocks

Hello, Searched for a while and found some "line-to-column" script. My case is similar but with multiple fields each row: S02 Length Per S02 7043 3.864 S02 54477 29.89 S02 104841 57.52 S03 Length Per S03 1150 0.835 S03 1321 0.96 S03 ... (9 Replies)
Discussion started by: yifangt
9 Replies

6. Shell Programming and Scripting

Output breaking when returning multiple values

I've been trying to write a command-line function to grab a website's MX records and their ip addresses. The code below works with domains that only have one MX record: function kmx { mx=`host -t MX $1 | awk '{ print $7 }'`; ip=`host $mx | sed '/IPv6/d;/handled/d' | awk '{ print $4 }'`; ... (8 Replies)
Discussion started by: Azrael
8 Replies

7. Shell Programming and Scripting

how to split this file into blocks and then send these blocks as input to the tool called Yices?

Hello, I have a file like this: FILE.TXT: (define argc :: int) (assert ( > argc 1)) (assert ( = argc 1)) <check> # (define c :: float) (assert ( > c 0)) (assert ( = c 0)) <check> # now, i want to separate each block('#' is the delimeter), make them separate files, and then send them as... (5 Replies)
Discussion started by: paramad
5 Replies

8. UNIX for Dummies Questions & Answers

Convert 512-blocks to 4k blocks

I'm Unix. I'm looking at "df" on Unix now and below is an example. It's lists the filesystems out in 512-blocks, I need this in 4k blocks. Is there a way to do this in Unix or do I manually convert and how? So for container 1 there is 7,340,032 in size in 512-blocks. What would the 4k block be... (2 Replies)
Discussion started by: rockycj
2 Replies

9. UNIX for Dummies Questions & Answers

Searching partial columns and returning maximum as output

Hello, I am just getting starting with awk and wondering if anyone could help with the following problem. I have a large file of data, 50,000 rows x 6 columns. I would like to search in blocks of 500 rows for a maximum value in a specific column and compile an output file that prints the... (0 Replies)
Discussion started by: xb_analysis
0 Replies

10. Shell Programming and Scripting

Searching and extracting text from output

I have the following output which I need to obtain the values for "Next Seq is xxx" and "Last Seq is xxx" and "Pending count is xxx". You will notice that the number of words prior to that value can be variable hence the reason for asking this question. LINECMD> Line /xxx///ABC9_SND is UP.... (3 Replies)
Discussion started by: sjday
3 Replies
Login or Register to Ask a Question