Find a min,group by.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find a min,group by.
# 1  
Old 08-15-2009
Find a min,group by.

I have a data file with records:
Code:
A123|Peter|20
A123|Jack |10
B222|Helen|15
B222|Jane |13
B222|Guy  |30

I want for find the min for $3 group by $1.

i.e

A123|Jack|10
B222|Jane|13

Thanks.
# 2  
Old 08-15-2009
In sh:
Code:
#!/bin/sh

while read line; do
  c1=$(echo "$line" | cut -d\| -f1)
  c3=$(echo "$line" | cut -d\| -f3)
  [ ! "$p1" ] && p1="$c1"
  [ ! "$min" ] && min="$c3"  
  if [ "$c1" = "$p1" ]; then
    if [ "$c3" -lt "$min" ]; then
      min="$c3"; line1="$line"
    fi
  else
    p1="$c1"; min="$c3"
    echo "$line1"    
  fi
done < file
echo "$line1"

In this code, the data file is called file, on line 16 (done < file)

there are probably better ways doing this , eg with awk Smilie, but could not find my way.
# 3  
Old 08-15-2009
Another way:

Code:
awk -F"|" '$1 in min {if($3 < min[$1]){min[$1]=$3;s[$1]=$2};next}
{min[$1]=$3;s[$1]=$2}
END{for(i in min)print i FS s[i] FS min[i]}
' file

# 4  
Old 08-16-2009
And yet another way:

Code:
$ 
$ cat data.txt
A123|Peter|20
A123|Jack |10
B222|Helen|15
B222|Jane |13
B222|Guy  |30
C333|Ana  |40
C333|Abe  |70
D444|Ben  |50
$ 
$ perl -ne 'chomp; ($a,$b,$c)=split/\|/; $b=~s/\s+$//;
          if ($a ne $pa) {push @y,"$a|$b|$c\n"}
          elsif ($c<$pc) {$y[$#y]="$a|$b|$c\n"}
          $pa=$a; $pc=$c;
          END {print @y}' data.txt
A123|Jack|10
B222|Jane|13
C333|Ana|40
D444|Ben|50
$ 
$

tyler_durden
# 5  
Old 08-16-2009
Hi.

Trusting that the sample data is representative and that one has access to appropriate commands:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate re-ordering for minimum in specific field.

echo
set +o nounset
LC_ALL=C ; LANG=C ; export LC_ALL LANG
echo "Environment: LC_ALL = $LC_ALL, LANG = $LANG"
echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version "=o" $(_eat $0 $1) sort uniq
set -o nounset
echo

FILE=${1-data1}

echo " Data file $FILE:"
cat $FILE

echo
echo " Results:"
sort -t'|' --key=1,1 --key=3,3n $FILE |
uniq -w4

exit 0

producing:
Code:
% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0 
GNU bash 3.2.39
sort (GNU coreutils) 6.10
uniq (GNU coreutils) 6.10

 Data file data1:
A123|Peter|20
B222|Helen|15
A123|Jack |10
B222|Jane |13
B222|Guy  |30

 Results:
A123|Jack |10
B222|Jane |13

See man pages for details ... cheers, drl

---------- Post updated at 07:10 ---------- Previous update was at 07:04 ----------


Using same data set, a variation in perl:
Code:
#!/usr/bin/perl

# @(#) p1	Demonstrate finding minimum value of field, single pass.

use warnings;
use strict;

my ($debug);
$debug = 1;
$debug = 0;
my ( $current, %hash, $id, $grade );

# Read file, make hash entry if new or lower value.

while (<>) {
  chomp;
  ( $id, $grade ) = ( split(/[|]/) )[ 0, 2 ];
  print " id, grade = :$id:, :$grade:\n" if $debug;
  if ( exists $hash{$id} ) {
    $current = ( split( /[|]/, $hash{$id} ) )[2];
    if ( $current < $grade ) {
      next;
    }
  }
  $hash{$id} = $_;
}

# Sort keys and print values from the hash.

for $id ( sort keys %hash ) {
  print "$hash{$id}\n";
}

print STDERR " ( Lines read: $. )\n" if $debug;

exit(0);

Producing:
Code:
% ./p1 data1
A123|Jack |10
B222|Jane |13

Best wishes ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find min and max time taken from a log file

You have a log file as attached in sample input with various operations and time taken by each of them. Write a script to find the min and max time taken for each operation. Sample output is attached. Sample Input is given as below: operation1,83621 operation2,72321 operation3,13288... (1 Reply)
Discussion started by: Chandan_Bose
1 Replies

2. Shell Programming and Scripting

Script to find min for each unique value

I need a script that will search through multiple files and when the first 2 columns match, print out Columns 1 and 2 and the minimum value. File 1 24.01 -81.01 1.0 24.02 -81.02 1.0 24.03 -81.03 3.0 File 2 24.01 -81.01 5.0 24.02 -81.02 3.0 24.03 -81.03 ... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

3. Shell Programming and Scripting

awk script to find min and max value

I need to find the max/min of columns 1 and 2 of a 2 column file what contains the special character ">". I know that this will find the max value of column 1. awk 'BEGIN {max = 0} {if ($1>max) max=$1} END {print max}' input.file But what if I needed to ignore special characters in the... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

4. Shell Programming and Scripting

Find if file is 45 min old if yes delete

Hello, we have process2(my_pro.sh) which check for the file Pr_rn_Chk (generated by other process1). The process runs every 15 min. if exist Pr_rn_Chk the process will exit. Else it will continue. I want to check if the file Pr_rn_Chk is older than 45 min delete the file. so that 4th run... (3 Replies)
Discussion started by: kumar30213
3 Replies

5. Shell Programming and Scripting

to find min and max value for each column!

Hello Experts, I have got a txt files which has multiple columns, I want to get the max, min and diff (max-min) for each column in the same txt file. Example: cat file.txt a 1 4 b 2 5 c 3 6 I want ouput like: cat file.txt a 1 4 b 2 5 c 3 6 Max 3 6 Min 1 4 Diff 2 2 awk 'min=="" ||... (4 Replies)
Discussion started by: dixits
4 Replies

6. Shell Programming and Scripting

How to find the average,min,max ,total count?

Hi , Below is my sample data,I have this 8 column(A,B,C,D,E,F,G,H) in csv file. A , B ,C ,D ,E ,F,G ,H 4141,127337,24,15,20,69,72.0,-3 4141,128864,24,15,20,65,66.0,-1 4141,910053,24,15,4,4,5.0,-1 4141,910383,24,15,22,3,4.0,-1 4141,496969,24,15,14,6,-24.0,-18... (7 Replies)
Discussion started by: vinothsekark
7 Replies

7. Shell Programming and Scripting

Find min.max value if matching columns found using AWK

Input_ File : 2 3 4 5 1 1 0 1 2 1 -1 1 2 1 3 1 3 1 4 1 6 5 6 6 6 6 6 7 6 7 6 8 5 8 6 7 Desired output : 2 3 4 5 -1 1 4 1 6 5 6 8 5 8 6 7 (3 Replies)
Discussion started by: vasanth.vadalur
3 Replies

8. Shell Programming and Scripting

Find all files with group read OR group write OR user write permission

I need to find all the files that have group Read or Write permission or files that have user write permission. This is what I have so far: find . -exec ls -l {} \; | awk '/-...rw..w./ {print $1 " " $3 " " $4 " " $9}' It shows me all files where group read = true, group write = true... (5 Replies)
Discussion started by: shunter63
5 Replies

9. Shell Programming and Scripting

how to find min, max dates in a file

hello friends...:-) i need some help i have a file cantain like this Star1 ,NetWork,09/02/2008 Star1 ,NetWork,10/02/2008 Star1 ,NetWork,11/02/2008 Star2 ,NetWork,08/03/2008 Star2 ,NetWork,09/04/2008 Star2 ,NetWork,10/05/2008 i need to find out min, max dates the output look like... (6 Replies)
Discussion started by: gemini106
6 Replies

10. UNIX for Dummies Questions & Answers

How to find whenther given value is in betwwen min and Max in unix shell scripting

Hi I wanted to write a shell script with an if condition Example MinValue=10 MaxValue=30 logvalue = some integer value that script reads from the command line arguement I wanted to check whether log value greater than or equal to10 and less than equal to 30 and proceed with the rest of... (5 Replies)
Discussion started by: pinky
5 Replies
Login or Register to Ask a Question