Sponsored Content
Top Forums Shell Programming and Scripting Sort and Remove Duplicate on file Post 302521923 by yinyuemi on Thursday 12th of May 2011 06:29:27 PM
Old 05-12-2011
how about this?
Code:
echo 'aaa|1234|2010-12-31
aaa|1234|2010-11-10
bbb|345|2011-01-01
ccc|346|2011-02-01
bbb|345|2011-03-10
aaa|1234|2010-01-01' |sort -t '|' -k1,1 -k3,3r |awk -F"|" '++a[$1"|"$2]==1'
aaa|1234|2010-12-31
bbb|345|2011-03-10
ccc|346|2011-02-01

 

9 More Discussions You Might Find Interesting

1. Solaris

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (2 Replies)
Discussion started by: svenkatareddy
2 Replies

2. Shell Programming and Scripting

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (19 Replies)
Discussion started by: svenkatareddy
19 Replies

3. Shell Programming and Scripting

Shellscript to sort duplicate files listed in a text file

I have many pdf's scattered across 4 machines. There is 1 location where I have other Pdf's maintained. But the issues it the 4 machines may have duplicate pdf's among themselves, but I want just 1 copy of each so that they can be transfered to that 1 location. What I have thought is: 1) I have... (11 Replies)
Discussion started by: deaddevil
11 Replies

4. Shell Programming and Scripting

remove of duplicate line from a file

I have a file a.txt having content like deepak ram sham deepram sita kumar I Want to delete the first line containing "deep" ... I tried using... grep -i 'deep' a.txt It gives me 2 rows...I want to delete the first one.. + need to know the command to delete the line from... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

5. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

6. Shell Programming and Scripting

Remove duplicate chars and sort string [SED]

Hi, INPUT: DCBADD OUTPUT: ABCD The SED script should alphabetically sort the chars in the string and remove the duplicate chars. (5 Replies)
Discussion started by: jds93
5 Replies

7. Shell Programming and Scripting

How to Remove duplicate value from file?

if different branch code is available for same BIC code and one of the branch code is XXX.only one row will be stored and with branch code as XXX .rest of the rows for the BIC code will not be stored. for example if $7 is BIC code and $8 is branch code INPUT file are following... (9 Replies)
Discussion started by: mohan sharma
9 Replies

8. UNIX for Dummies Questions & Answers

Sort and delete partical duplicate file

I want to delete partical duplicate file >gma-miR156d Gm01,PACID=26323927 150.00 -18.28 2 18 17 35 16 75.00% 81.25% >>gma-miR156d Gm01,PACID=26323927 150.00 -18.28 150.00 -18.28 1 21 119 17 I want to order by the second column and delete the... (1 Reply)
Discussion started by: grace_shen
1 Replies

9. Shell Programming and Scripting

Remove duplicate lines, sort it and save it as file itself

Hi, all I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
ToDo(3pm)						User Contributed Perl Documentation						 ToDo(3pm)

NAME
Palm::ToDo - Handler for Palm ToDo databases. SYNOPSIS
use Palm::ToDo; DESCRIPTION
The ToDo PDB handler is a helper class for the Palm::PDB package. It parses ToDo databases. AppInfo block The AppInfo block begins with standard category support. See Palm::StdAppInfo for details. Other fields include: $pdb->{appinfo}{dirty_appinfo} $pdb->{appinfo}{sortOrder} I don't know what these are. Sort block $pdb->{sort} This is a scalar, the raw data of the sort block. Records $record = $pdb->{records}[N] $record->{due_day} $record->{due_month} $record->{due_year} The due date of the ToDo item. If the item has no due date, these are undefined. $record->{completed} This is defined and true iff the item has been completed. $record->{priority} An integer. The priority of the item. $record->{description} A text string. The description of the item. $record->{note} A text string. The note attached to the item. Undefined if the item has no note. new $pdb = new Palm::ToDo; Create a new PDB, initialized with the various Palm::ToDo fields and an empty record list. Use this method if you're creating a ToDo PDB from scratch. new_Record $record = $pdb->new_Record; Creates a new ToDo record, with blank values for all of the fields. "new_Record" does not add the new record to $pdb. For that, you want "$pdb->append_Record". SOURCE CONTROL
The source is in Github: http://github.com/briandfoy/p5-Palm/tree/master AUTHOR
Alessandro Zummo, "<a.zummo@towertech.it>" Currently maintained by brian d foy, "<bdfoy@cpan.org>" SEE ALSO
Palm::PDB(3) Palm::StdAppInfo(3) perl v5.10.1 2010-02-23 ToDo(3pm)
All times are GMT -4. The time now is 07:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy