Sponsored Content
Top Forums Shell Programming and Scripting Removing duplicates [sort , uniq] Post 302080032 by sharatz83 on Friday 14th of July 2006 04:56:32 PM
Old 07-14-2006
Quote:
Originally Posted by jim mcnamara
Code:
awk -F "#" '!arr[$1]++'  filename > newfilename

Jim ,

It works fine on Linux but when I tried to run it on Solaris , only
awk -F "#" 'arr[$1]++' filename > newfilename seems to work and not
awk -F "#" '!arr[$1]++' filename > newfilename .

-sharat
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sort/uniq

I have a file: Fred Fred Fred Jim Fred Jim Jim If sort is executed on the listed file, shouldn't the output be?: Fred Fred Fred Fred Jim Jim Jim (3 Replies)
Discussion started by: jimmyflip
3 Replies

2. Shell Programming and Scripting

Sort, Uniq, Duplicates

Input File is : ------------- 25060008,0040,03, 25136437,0030,03, 25069457,0040,02, 80303438,0014,03,1st 80321837,0009,03,1st 80321977,0009,03,1st 80341345,0007,03,1st 84176527,0047,03,1st 84176527,0047,03, 20000735,0018,03,1st 25060008,0040,03, I am using the following in the script... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies

3. UNIX for Dummies Questions & Answers

removing duplicates and sort -k

Hello experts, I am trying to remove all lines in a csv file where the 2nd columns is a duplicate. I am try to use sort with the key parameter sort -u -k 2,2 File.csv > Output.csv File.csv File Name|Document Name|Document Title|Organization Word Doc 1.doc|Word Document|Sample... (3 Replies)
Discussion started by: orahi001
3 Replies

4. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

5. Shell Programming and Scripting

sort | uniq question

Hello, I have a large data file: 1234 8888 bbb 2745 8888 bbb 9489 8888 bbb 1234 8888 aaa 4838 8888 aaa 3977 8888 aaa I need to remove duplicate lines (where the first column is the duplicate). I have been using: sort file.txt | uniq -w4 > newfile.txt However, it seems to keep the... (11 Replies)
Discussion started by: palex
11 Replies

6. Shell Programming and Scripting

Sort and uniq after comparision

Hi All, I have a text file with the format shown below. Some of the records are duplicated with the only exception being date (Field 15). I want to compare all duplicate records using subscriber number (field 7) and keep only those records with greater date. ... (1 Reply)
Discussion started by: nua7
1 Replies

7. Shell Programming and Scripting

Sort field and uniq

I have a flatfile A.txt 2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11 How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies

8. Shell Programming and Scripting

Uniq or sort -u or similar only between { }

Hi ! I am trying to remove doubbled entrys in a textfile only between delimiters. Like that example but i dont know how to do that with sort or similar. input: { aaa aaa } { aaa aaa } output: { aaa } { (8 Replies)
Discussion started by: fugitivus
8 Replies

9. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies

10. Shell Programming and Scripting

Sort & Uniq -u

Hi All, Below the actual file which i like to sort and Uniq -u /opt/oracle/work/Antony/Shell_Script> cat emp.1st 2233|a.k. shukula |g.m. |sales |12/12/52 |6000 1006|chanchal singhvi |director |sales |03/09/38 |6700... (8 Replies)
Discussion started by: Antony Ankrose
8 Replies
COLLATOR_SORT_WITH_SORT_KEYS(3) 					 1					   COLLATOR_SORT_WITH_SORT_KEYS(3)

Collator::sortWithSortKeys - Sort array using specified collator and sort keys

	Object oriented style

SYNOPSIS
public bool Collator::sortWithSortKeys (array &$arr) DESCRIPTION
Procedural style bool collator_sort_with_sort_keys (Collator $coll, array &$arr) Similar to collator_sort(3) but uses ICU sorting keys produced by ucol_getSortKey() to gain more speed on large arrays. PARAMETERS
o $coll -Collator object. o $arr -Array of strings to sort RETURN VALUES
Returns TRUE on success or FALSE on failure. EXAMPLES
Example #1 collator_sort_with_sort_keys(3) example <?php $arr = array( 'Kopfe', 'Kypper', 'Kopfe' ); $coll = collator_create( 'sv' ); collator_sort_with_sort_keys( $coll, $arr ); var_export( $arr ); ?> The above example will output: array ( 0 => 'Kopfe', 1 => 'Kypper', 2 => 'Kopfe', ) SEE ALSO
Collator constants, collator_sort(3), collator_asort(3). PHP Documentation Group COLLATOR_SORT_WITH_SORT_KEYS(3)
All times are GMT -4. The time now is 01:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy