Sponsored Content
Full Discussion: Sort, Uniq, Duplicates
Top Forums Shell Programming and Scripting Sort, Uniq, Duplicates Post 302117872 by ghostdog74 on Thursday 17th of May 2007 01:49:54 AM
Old 05-17-2007
Code:
awk -F"," '{ line[$1.$2] = $0
             arr[$1.$2]++
           }
END{     for (i in arr) {
            if ( arr[i] > 1 ){
	       print line[i] > "duplicates"
	    }
	 } 
 }' file

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sort/uniq

I have a file: Fred Fred Fred Jim Fred Jim Jim If sort is executed on the listed file, shouldn't the output be?: Fred Fred Fred Fred Jim Jim Jim (3 Replies)
Discussion started by: jimmyflip
3 Replies

2. UNIX for Dummies Questions & Answers

Help with Last,uniq, sort and cut

Using the last, uniq, sort and cut commands, determine how many times the different users have logged in. I know how to use the last command and cut command... i came up with last | cut -f1 -d" " | uniq i dont know if this is right, can someone please help me... thanks (1 Reply)
Discussion started by: jay1228
1 Replies

3. Shell Programming and Scripting

sort and uniq in perl

Does anyone have a quick and dirty way of performing a sort and uniq in perl? How an array with data like: this is bkupArr BOLADVICE_VN this is bkupArr MLT6800PROD2A this is bkupArr MLT6800PROD2A this is bkupArr BOLADVICE_VN_7YR this is bkupArr MLT6800PROD2A I want to sort it... (4 Replies)
Discussion started by: reggiej
4 Replies

4. Shell Programming and Scripting

Removing duplicates [sort , uniq]

Hey Guys, I have file which looks like this, Contig201#numbPA Contig1452#nmdynD6PA dm022p15.r#CG6461PA dm005e16.f#SpatPA IGU001_0015_A06.f#CG17593PA I need to remove duplicates based on the chracter matching upto '#'. for example if we consider this.. Contig201#numbPA... (4 Replies)
Discussion started by: sharatz83
4 Replies

5. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

6. Shell Programming and Scripting

sort | uniq question

Hello, I have a large data file: 1234 8888 bbb 2745 8888 bbb 9489 8888 bbb 1234 8888 aaa 4838 8888 aaa 3977 8888 aaa I need to remove duplicate lines (where the first column is the duplicate). I have been using: sort file.txt | uniq -w4 > newfile.txt However, it seems to keep the... (11 Replies)
Discussion started by: palex
11 Replies

7. Shell Programming and Scripting

Sort and uniq after comparision

Hi All, I have a text file with the format shown below. Some of the records are duplicated with the only exception being date (Field 15). I want to compare all duplicate records using subscriber number (field 7) and keep only those records with greater date. ... (1 Reply)
Discussion started by: nua7
1 Replies

8. Shell Programming and Scripting

Sort uniq or awk

Hi again, I have files with the following contents datetime,ip1,port1,ip2,port2,number How would I find out how many times ip1 field shows up a particular file? Then how would I find out how many time ip1 and port 2 shows up? Please mind the file may contain 100k lines. (8 Replies)
Discussion started by: LDHB2012
8 Replies

9. Shell Programming and Scripting

Uniq or sort -u or similar only between { }

Hi ! I am trying to remove doubbled entrys in a textfile only between delimiters. Like that example but i dont know how to do that with sort or similar. input: { aaa aaa } { aaa aaa } output: { aaa } { (8 Replies)
Discussion started by: fugitivus
8 Replies

10. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies
LOCALE_LOOKUP(3)							 1							  LOCALE_LOOKUP(3)

Locale::lookup - Searches the language tag list for the best match to the language

	Object oriented style

SYNOPSIS
publicstatic string Locale::lookup (array $langtag, string $locale, [bool $canonicalize = false], [string $default]) DESCRIPTION
Procedural style string locale_lookup (array $langtag, string $locale, [bool $canonicalize = false], [string $default]) Searches the items in $langtag for the best match to the language range specified in $locale according to RFC 4647's lookup algorithm. PARAMETERS
o $langtag - An array containing a list of language tags to compare to $locale. Maximum 100 items allowed. o $locale - The locale to use as the language range when matching. o $canonicalize - If true, the arguments will be converted to canonical form before matching. o $default - The locale to use if no match is found. RETURN VALUES
The closest matching language tag or default value. EXAMPLES
Example #1 locale_lookup(3) example <?php $arr = array( 'de-DEVA', 'de-DE-1996', 'de', 'de-De' ); echo locale_lookup($arr, 'de-DE-1996-x-prv1-prv2', true, 'en_US'); ?> Example #2 OO example <?php $arr = array( 'de-DEVA', 'de-DE-1996', 'de', 'de-De' ); echo Locale::lookup($arr, 'de-DE-1996-x-prv1-prv2', true, 'en_US'); ?> The above example will output: de_de_1996 SEE ALSO
locale_filter_matches(3). PHP Documentation Group LOCALE_LOOKUP(3)
All times are GMT -4. The time now is 04:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy