Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file
Top Forums Shell Programming and Scripting AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file Post 302600791 by rich@ardz on Wednesday 22nd of February 2012 05:02:28 AM
Old 02-22-2012
AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file
I have a file containing data like so:

2012-01-02 GREEN 4
2012-01-02 GREEN 6
2012-01-02 GREEN 7
2012-01-02 BLUE 4
2012-01-02 BLUE 3
2012-01-02 GREEN 4
2012-01-02 RED 4
2012-01-02 RED 8
2012-01-02 GREEN 4
2012-01-02 YELLOW 5
2012-01-02 YELLOW 2

I can't always predict what the strings are going to be in the second column (so in the example above there are colours but the data file could contain any string in column two). There is always however a number in the third column (which I want the max value of for a paticular string in column two). Is awk able to:
  1. Pull out each of the unique strings in column 2?
  2. For each of the unique strings get the maximum associated value (so using the above you'd end up with the following)?:

    2012-01-02 GREEN 7
    2012-01-02 BLUE 4
    2012-01-02 RED 8
    2012-01-02 YELLOW 5
or would this be easier with Perl (or even shell)? any code examples much appreciated!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

2. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

3. Shell Programming and Scripting

Perl Data Structure - Non unique values

I have the perl data structure and what i need to do is find all values in @{$extractColumns{'2'}{'D'}} which are not there in @{$extractColumns{'2'}{'M'}} but seems like i need to put a flag somewhere and i messed up foreach my $order (keys %extractColumns) { foreach my $value... (2 Replies)
Discussion started by: dinjo_jo
2 Replies

4. Shell Programming and Scripting

for each different entry in column 1 extract maximum values from column 2 in unix/awk

Hello, I have 2 columns (1st column has multiple entries but the corresponding values in the column 2 may be the same or different.) however I want to extract unique values for each entry in column 1 by assigning the max value from column 2 SDF4 -0.211654 SDF4 0.978068 ... (1 Reply)
Discussion started by: Diya123
1 Replies

5. Shell Programming and Scripting

calculating unique strings values

Hi, Im looking for a script which will calculate the unique strings column 2 & 3 values in a log as mentioned in example eg:- bag 12 12 bag 18 15 bags 15 13 bags 15 14 blazer 24 24 blazer 33 32 boots 19 15 Result should be:- bag 30 27 bags 30 27... (9 Replies)
Discussion started by: Paulwintech
9 Replies

6. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

7. Shell Programming and Scripting

Extracting unique values of a column from a feed file

Hi Folks, I have the below feed file named abc1.txt in which you can see there is a title and below is the respective values in the rows and it is completely pipe delimited file ,. ... (4 Replies)
Discussion started by: punpun66
4 Replies

8. UNIX for Dummies Questions & Answers

Using awk to find and use the maximum value in column of data

Dear Unix Gurus, I have a text file with multiple columns, for example, see sample.txt below 0 1 301 1 4 250 2 6 140 3 2 610 7 1 180I want to find the maximum in, say, column 3, normalise all the values to this maximum value (to 4 decimal places) and spit everything into a new... (2 Replies)
Discussion started by: tintin72
2 Replies

9. Shell Programming and Scripting

awk to filter out lines containing unique values in a specified column

Hi, I have multiple files that each contain four columns of strings: File1: Code: 123 abc gfh 273 456 ddff jfh 837 789 ghi u4u 395 File2: Code: 123 abc dd fu 456 def 457 nd 891 384 djh 783 I want to compare the strings in Column 1 of File 1 with each other file and Print in... (3 Replies)
Discussion started by: owwow14
3 Replies

10. Shell Programming and Scripting

Find minimum and maximum values based on column with associative array

Hello, I need to find out the minimum and maximum values based on specific column, and then print out the entire row with the max value. Infile.txt: scf6 290173 290416 . + X_047241 T_00113118-1 scf6 290491 290957 . + X_047241 T_00113118-2 scf6 290898 290957 . + X_047241 T_00113119-3 scf6... (2 Replies)
Discussion started by: yifangt
2 Replies

LEARN ABOUT DEBIAN

vboxrc

vboxrc(5)						    Linux System Administration 						 vboxrc(5)

NAME

       vboxrc - user config file for vbox

Description
       Every user can save settings in ~/.vboxrc.

       Everything  after  "#" is ignored, as are empty lines. All arguments must be separated by whitespace (spaces, tabs). Arguments may not con-
       tain whitespace.

       These keywords are allowed:

	      USERNAME

	      PASSWORD
		     Username and password for vboxd login. These settings are used by vbox and vboxctrl. If they are missing, you have  to  enter
		     them at runtime.

	      VOLUME Default volume for vbox.

	      C_BACKGROUND
		     Background color.

	      C_STATUSBAR
		     Color for normal text in the status bar.

	      C_STATUSBAR_HL
		     Color for highlighted text in the status bar.

	      C_POWERLED_ON
		     Color for powerled (on).

	      C_POWERLED_OFF
		     Color for powerled (off).

	      C_STATUSLED_ON
		     Color for statusled (on).

	      C_STATUSLED_OFF
		     Color for statusled (off).

	      C_LIST Color for list of messages.

	      C_LIST_SELECTED
		     Color for selected entry in the list of messages.

	      C_INFOTEXT
		     Color for info texts in the message list.

       All color settings have the format TEXTCOLOR:BACKGROUNDCOLOR. You may not use whitespace.

       These colors are available for TEXTCOLOR:
	      BLACK RED GREEN BROWN BLUE MAGENTA CYAN GRAY
	      DARKGREY LIGHTRED LIGHTGREEN YELLOW LIGHTBLUE
	      LIGHTMAGENTA LIGHTCYAN WHITE

       These colors are available for BACKGROUNDCOLOR:
	      BLACK RED GREEN BROWN BLUE MAGENTA CYAN GREY

FILES

       ~/.vboxrc
	      this configuration file
       /usr/share/doc/isdnvboxclient/examples/vboxrc
	      an example

SEE ALSO

       vbox(1)

AUTHOR

       This  manual  page  was	written  by Andreas Jellinghaus <aj@dungeon.inka.de>, for Debian GNU/Linux and isdn4linux.  Now maintained by Paul
       Slootman <paul@debian.org>.

ISDN 4 Linux 3.25						    2000/09/15								 vboxrc(5)
All times are GMT -4. The time now is 11:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy