Sponsored Content
Full Discussion: Unique sort with two fields
Top Forums Shell Programming and Scripting Unique sort with two fields Post 302517127 by mirni on Tuesday 26th of April 2011 07:02:50 AM
Old 04-26-2011
This should do the trick:
Code:
$ sort -n input  | awk -F, '{if($1 in a){next}else{a[$1]++;print} }'

---------- Post updated at 01:02 AM ---------- Previous update was at 01:00 AM ----------

@ scottn: Your solution fails in this case:
Code:
$ cat cols
1230,502
1230,506
123,702
123,99
234,101
235,104
456,104
456,100
$ sort  -t, -k1,1 -u cols
123,702
1230,502
234,101
235,104
456,104

Numeric sort is needed
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sort and Unique in Perl

Hi, May I know, if a pipe separated File is large, what is the best method to calculate the unique row count of 3rd column and get a list of unique value of the 3rdcolum? Thanks in advance! (20 Replies)
Discussion started by: deepakwins
20 Replies

2. Shell Programming and Scripting

Perl sort unique by one field only

Hi all, I've searched the forum and I can find some code to sort uniquely in perl but not by a single field. I have a file with data such as the following: 1,test,34 1,test2,65 2,test,35, 1,test3,34 2,test,34 What i want to do is sort it uniqely by the first field only so I'd end... (2 Replies)
Discussion started by: Donkey25
2 Replies

3. Shell Programming and Scripting

What some ideas with sort and get unique data

Hi all, I am writing a script where i can parse through the directory and get common string in two directories i get. The command below SUN_PLATFORM=`$FIND $STREAM_PATH . -depth -name ShareableEntities | $AWK -F"/" '{if($10 ~ /sun5/) print $0}'` gives the following output:- ... (1 Reply)
Discussion started by: asirohi
1 Replies

4. Shell Programming and Scripting

Awk sort and unique

Input file --------- 12:name1:|host1|host1|host2|host1 13:name2:|host1|host1|host2|host3 14:name3: ...... Required output --------------- 12:name1:host1(2)|host1(1) 13:name2:host1(2)|host2(1)|host3(1) 14:name3: where (x) - Count how many times field appears in last column ... (3 Replies)
Discussion started by: greycells
3 Replies

5. Shell Programming and Scripting

Unique sort with three fields

I have another file with three columns A,B,C as below 123,1,502 123,2,506 123,3,702 234,4,101 235,5,104 456,6,104 456,7,100 i want to sort such that i get a unique value in column A, and for those with multiple value in A, i want the lowest value in C. output should be Code:... (3 Replies)
Discussion started by: dealerso
3 Replies

6. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

7. Shell Programming and Scripting

Sort unique by multiple fields

i need to sort to get all the unique records based on the 1st and 2nd column, and keep the record with the highest value on 5th column if there are duplicates, every column with varies length a^2^x^y^z bxc^2xx2^aa^bvxxxx^cdd a^3^1^2^3 a^2^x^1^c I want a result which will only keep the 1st... (2 Replies)
Discussion started by: dtdt
2 Replies

8. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Hi I have a file as below <field1> <field2> <field3> ... <field_num1> <field_num2> Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields. I tried this and it doesn't sort on the difference field .. Appreciate your help. cat... (9 Replies)
Discussion started by: newstart
9 Replies

9. Shell Programming and Scripting

Sort unique

Hi, I have an input file that I have sorted in a previous stage by $1 and $4. I now need something that will take the first record from each group of data based on the key being $1 Input file 1000AAA|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA 1000AAA|"ZZZ"|"Date"|"2"|"Y"|"ABC"|""|AA... (2 Replies)
Discussion started by: Ads89
2 Replies

10. UNIX for Beginners Questions & Answers

Grep or awk a unique and specific word across many fields

Hi there, I have data with similar structure as this: CHR START-SNP END-SNP REF ALT PATIENT1 PATIENT2 PATIENT3 PATIENT4 chr1 69511 69511 A G homo hetero homo hetero chr2 69513 69513 T C . hetero homo hetero chr3 69814 69814 G C . . homo homo chr4 69815 69815 C A hetero . . hetero is... (10 Replies)
Discussion started by: daashti
10 Replies
Regexp::Common::number(3)				User Contributed Perl Documentation				 Regexp::Common::number(3)

NAME
Regexp::Common::number -- provide regexes for numbers SYNOPSIS
use Regexp::Common qw /number/; while (<>) { /^$RE{num}{int}$/ and print "Integer "; /^$RE{num}{real}$/ and print "Real "; /^$RE{num}{real}{-base => 16}$/ and print "Hexadecimal real "; } DESCRIPTION
Please consult the manual of Regexp::Common for a general description of the works of this interface. Do not use this module directly, but load it via Regexp::Common. $RE{num}{int}{-base}{-sep}{-group}{-places}{-sign} Returns a pattern that matches an integer. If "-base => B" is specified, the integer is in base B, with "2 <= B <= 36". For bases larger than 10, upper case letters are used. The default base is 10. If "-sep => P" is specified, the pattern P is required as a grouping marker within the number. If this option is not given, no grouping marker is used. If "-group => N" is specified, digits between grouping markers must be grouped in sequences of exactly N digits. The default value of N is 3. If "-group => N,M" is specified, digits between grouping markers must be grouped in sequences of at least N digits, and at most M digits. This option is ignored unless the "-sep" option is used. If "-places => N" is specified, the integer recognized must be exactly N digits wide. If "-places => N,M" is specified, the integer must be at least N wide, and at most M characters. There is no default, which means that integers are unlimited in size. This option is ignored if the "-sep" option is used. If "-sign => P" is used, it's a pattern the leading sign has to match. This defaults to "[-+]?", which means the number is optionally preceded by a minus or a plus. If you want to match unsigned integers, use $RE{num}{int}{-sign => ''}. For example: $RE{num}{int} # match 1234567 $RE{num}{int}{-sep=>','} # match 1,234,567 $RE{num}{int}{-sep=>',?'} # match 1234567 or 1,234,567 $RE{num}{int}{-sep=>'.'}{-group=>4} # match 1.2345.6789 Under "-keep" (see Regexp::Common): $1 captures the entire number $2 captures the optional sign of the number $3 captures the complete set of digits $RE{num}{real}{-base}{-radix}{-places}{-sep}{-group}{-expon} Returns a pattern that matches a floating-point number. If "-base=N" is specified, the number is assumed to be in that base (with A..Z representing the digits for 11..36). By default, the base is 10. If "-radix=P" is specified, the pattern P is used as the radix point for the number (i.e. the "decimal point" in base 10). The default is "qr/[.]/". If "-places=N" is specified, the number is assumed to have exactly N places after the radix point. If "-places=M,N" is specified, the number is assumed to have between M and N places after the radix point. By default, the number of places is unrestricted. If "-sep=P" specified, the pattern P is required as a grouping marker within the pre-radix section of the number. By default, no separator is allowed. If "-group=N" is specified, digits between grouping separators must be grouped in sequences of exactly N characters. The default value of N is 3. If "-expon=P" is specified, the pattern P is used as the exponential marker. The default value of P is "qr/[Ee]/". If "-sign=P" is specified, the pattern P is used to match the leading sign (and the sign of the exponent). This defaults to "[-+]?", means means that an optional plus or minus sign can be used. For example: $RE{num}{real} # matches 123.456 or -0.1234567 $RE{num}{real}{-places=>2} # matches 123.45 or -0.12 $RE{num}{real}{-places=>'0,3'} # matches 123.456 or 0 or 9.8 $RE{num}{real}{-sep=>'[,.]?'} # matches 123,456 or 123.456 $RE{num}{real}{-base=>3'} # matches 121.102 Under "-keep": $1 captures the entire match $2 captures the optional sign of the number $3 captures the complete mantissa $4 captures the whole number portion of the mantissa $5 captures the radix point $6 captures the fractional portion of the mantissa $7 captures the optional exponent marker $8 captures the entire exponent value $9 captures the optional sign of the exponent $10 captures the digits of the exponent $RE{num}{dec}{-radix}{-places}{-sep}{-group}{-expon} A synonym for $RE{num}{real}{-base=>10}{...} $RE{num}{oct}{-radix}{-places}{-sep}{-group}{-expon} A synonym for $RE{num}{real}{-base=>8}{...} $RE{num}{bin}{-radix}{-places}{-sep}{-group}{-expon} A synonym for $RE{num}{real}{-base=>2}{...} $RE{num}{hex}{-radix}{-places}{-sep}{-group}{-expon} A synonym for $RE{num}{real}{-base=>16}{...} $RE{num}{decimal}{-base}{-radix}{-places}{-sep}{-group} The same as $RE{num}{real}, except that an exponent isn't allowed. Hence, this returns a pattern matching decimal numbers. If "-base=N" is specified, the number is assumed to be in that base (with A..Z representing the digits for 11..36). By default, the base is 10. If "-radix=P" is specified, the pattern P is used as the radix point for the number (i.e. the "decimal point" in base 10). The default is "qr/[.]/". If "-places=N" is specified, the number is assumed to have exactly N places after the radix point. If "-places=M,N" is specified, the number is assumed to have between M and N places after the radix point. By default, the number of places is unrestricted. If "-sep=P" specified, the pattern P is required as a grouping marker within the pre-radix section of the number. By default, no separator is allowed. If "-group=N" is specified, digits between grouping separators must be grouped in sequences of exactly N characters. The default value of N is 3. For example: $RE{num}{decimal} # matches 123.456 or -0.1234567 $RE{num}{decimal}{-places=>2} # matches 123.45 or -0.12 $RE{num}{decimal}{-places=>'0,3'} # matches 123.456 or 0 or 9.8 $RE{num}{decimal}{-sep=>'[,.]?'} # matches 123,456 or 123.456 $RE{num}{decimal}{-base=>3'} # matches 121.102 Under "-keep": $1 captures the entire match $2 captures the optional sign of the number $3 captures the complete mantissa $4 captures the whole number portion of the mantissa $5 captures the radix point $6 captures the fractional portion of the mantissa $RE{num}{square} Returns a pattern that matches a (decimal) square. Because Perl's arithmetic is lossy when using integers over about 53 bits, this pattern only recognizes numbers less than 9000000000000000, if one uses a Perl that is configured to use 64 bit integers. Otherwise, the limit is 2147483647. These restrictions were introduced in versions 2.116 and 2.117 of Regexp::Common. Regardless whether "-keep" was set, the matched number will be returned in $1. This pattern is available for version 5.008 and up. $RE{num}{roman} Returns a pattern that matches an integer written in Roman numbers. Case doesn't matter. Only the more modern style, that is, no more than three repetitions of a letter, is recognized. The largest number matched is MMMCMXCIX, or 3999. Larger numbers cannot be expressed using ASCII characters. A future version will be able to deal with the Unicode symbols to match larger Roman numbers. Under "-keep", the number will be captured in $1. SEE ALSO
Regexp::Common for a general description of how to use this interface. AUTHOR
Damian Conway (damian@conway.org) MAINTAINANCE
This package is maintained by Abigail (regexp-common@abigail.be). BUGS AND IRRITATIONS
Bound to be plenty. For a start, there are many common regexes missing. Send them in to regexp-common@abigail.be. LICENSE and COPYRIGHT This software is Copyright (c) 2001 - 2013, Damian Conway and Abigail. This module is free software, and maybe used under any of the following licenses: 1) The Perl Artistic License. See the file COPYRIGHT.AL. 2) The Perl Artistic License 2.0. See the file COPYRIGHT.AL2. 3) The BSD Licence. See the file COPYRIGHT.BSD. 4) The MIT Licence. See the file COPYRIGHT.MIT. perl v5.18.2 2013-03-11 Regexp::Common::number(3)
All times are GMT -4. The time now is 02:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy