Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file
Top Forums Shell Programming and Scripting AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file Post 302600791 by rich@ardz on Wednesday 22nd of February 2012 05:02:28 AM
Old 02-22-2012
AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file
I have a file containing data like so:

2012-01-02 GREEN 4
2012-01-02 GREEN 6
2012-01-02 GREEN 7
2012-01-02 BLUE 4
2012-01-02 BLUE 3
2012-01-02 GREEN 4
2012-01-02 RED 4
2012-01-02 RED 8
2012-01-02 GREEN 4
2012-01-02 YELLOW 5
2012-01-02 YELLOW 2

I can't always predict what the strings are going to be in the second column (so in the example above there are colours but the data file could contain any string in column two). There is always however a number in the third column (which I want the max value of for a paticular string in column two). Is awk able to:
  1. Pull out each of the unique strings in column 2?
  2. For each of the unique strings get the maximum associated value (so using the above you'd end up with the following)?:

    2012-01-02 GREEN 7
    2012-01-02 BLUE 4
    2012-01-02 RED 8
    2012-01-02 YELLOW 5
or would this be easier with Perl (or even shell)? any code examples much appreciated!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

2. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

3. Shell Programming and Scripting

Perl Data Structure - Non unique values

I have the perl data structure and what i need to do is find all values in @{$extractColumns{'2'}{'D'}} which are not there in @{$extractColumns{'2'}{'M'}} but seems like i need to put a flag somewhere and i messed up foreach my $order (keys %extractColumns) { foreach my $value... (2 Replies)
Discussion started by: dinjo_jo
2 Replies

4. Shell Programming and Scripting

for each different entry in column 1 extract maximum values from column 2 in unix/awk

Hello, I have 2 columns (1st column has multiple entries but the corresponding values in the column 2 may be the same or different.) however I want to extract unique values for each entry in column 1 by assigning the max value from column 2 SDF4 -0.211654 SDF4 0.978068 ... (1 Reply)
Discussion started by: Diya123
1 Replies

5. Shell Programming and Scripting

calculating unique strings values

Hi, Im looking for a script which will calculate the unique strings column 2 & 3 values in a log as mentioned in example eg:- bag 12 12 bag 18 15 bags 15 13 bags 15 14 blazer 24 24 blazer 33 32 boots 19 15 Result should be:- bag 30 27 bags 30 27... (9 Replies)
Discussion started by: Paulwintech
9 Replies

6. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

7. Shell Programming and Scripting

Extracting unique values of a column from a feed file

Hi Folks, I have the below feed file named abc1.txt in which you can see there is a title and below is the respective values in the rows and it is completely pipe delimited file ,. ... (4 Replies)
Discussion started by: punpun66
4 Replies

8. UNIX for Dummies Questions & Answers

Using awk to find and use the maximum value in column of data

Dear Unix Gurus, I have a text file with multiple columns, for example, see sample.txt below 0 1 301 1 4 250 2 6 140 3 2 610 7 1 180I want to find the maximum in, say, column 3, normalise all the values to this maximum value (to 4 decimal places) and spit everything into a new... (2 Replies)
Discussion started by: tintin72
2 Replies

9. Shell Programming and Scripting

awk to filter out lines containing unique values in a specified column

Hi, I have multiple files that each contain four columns of strings: File1: Code: 123 abc gfh 273 456 ddff jfh 837 789 ghi u4u 395 File2: Code: 123 abc dd fu 456 def 457 nd 891 384 djh 783 I want to compare the strings in Column 1 of File 1 with each other file and Print in... (3 Replies)
Discussion started by: owwow14
3 Replies

10. Shell Programming and Scripting

Find minimum and maximum values based on column with associative array

Hello, I need to find out the minimum and maximum values based on specific column, and then print out the entire row with the max value. Infile.txt: scf6 290173 290416 . + X_047241 T_00113118-1 scf6 290491 290957 . + X_047241 T_00113118-2 scf6 290898 290957 . + X_047241 T_00113119-3 scf6... (2 Replies)
Discussion started by: yifangt
2 Replies

LEARN ABOUT DEBIAN

primaxscan

PRIMAXSCAN(1)						User Contributed Perl Documentation					     PRIMAXSCAN(1)

NAME

       primaxscan

USAGE

       primaxscan [options] primaxscan -f filename.tiff primaxscan -h

OPTIONS

       -h, --help
	   shows this message

       -v [LEVEL] , --verbose=[LEVEL]
	   increase debugging output

       -P PORT, --port=PORT
	   Hex-address of the parallel-PORT (378)

       -s SPEED, --speed=SPEED
	   SPEED of the carriage {0,1,2...15}

       -S RES, --Scanner=RES
	   max. optical RESolution of the Scanner {300,600} dpi

       -V, --version
	   display version

       -p LxT, --pos=LxT
	   position form Left x Top in inch

       -d WxH, --dim=WxH
	   Width x Heigth of scan area in inch (max. 8.50 x 11.70)

       -c C, --contrast=C
	   Contrast {(-2000)..(1000)}

       -b B, --brightness=B
	   Brightness {(-200)..(200)}

       -g G, --gamma=G
	   Gamma {(0.2)..(5)}

       -r RES, --res=RES
	   RESolution in dpi (max. 300) see option -S

       -f NAME,  --file=NAME
	   NAME for tiff-file

       -C val, --Compression=val, val=0
	   switches compression off

       -l on/off, --lamp=on/off
	   turn lamp on/off

       -D, --RED
	   only RED-channel

       -E, --BLUE
	   only BLUE-channel

       -N, --GREEN
	   only GREEN-channel

       -R, --RGB
	   24-Bit RGB-mode (default)

       -G, --GRAY
	   8-Bit GRAY-mode

       -B, --BW
	   1-Bit Black-White-mode (possible with all other modes)

SEE ALSO

       http://primax.sourceforge.net/faq.html

AUTHORS

       Software authors:
	   Marco Foglia <marco@foglia.org> Thomas Schano <Schano@t-online.de> Christian Ordig <chr.ordig@gmx.net>

       Manual author:
	   Ola Lundqvist <opal@debian.org>

POD ERRORS

       Hey! The above document had some coding errors, which are explained below:

       Around line 13:
	   '=item' outside of any '=over'

       Around line 98:
	   You forgot a '=back' before '=head1'

       Around line 104:
	   '=item' outside of any '=over'

Primaxscan						   Sat Apr 21 13:35:52 UTC 2012 					     PRIMAXSCAN(1)
All times are GMT -4. The time now is 08:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy