Shell Programming and Scripting

Hi ,
I have below data inside a file named ref.psv . I want to create a shell script which will do the below 2 points :
(1) sort the file content first based on the latest date which is the last column in the file (actual file its the 175th column)
(2)after sorting the file based on latest date , remove the duplicates based on the first column only

15277105||Common Stick|ESHR||Common Stock|CYRO AB|2013-05-14T00:52:31.662-04:00

16111278||Common Stick|ESHR||Common Stock|STANDARD REGISTER CO|2013-05-14T00:52:31.672-04:00

15277105||Common Stick|ESHR||Common Stock|CYRO AB|2013-05-15T00:52:31.672-04:00

39693766||Common Stick|ESHR||Common Stock|HS AG|2013-05-15T00:52:31.672-04:00

Any help with the script is requested .
thanks,
Sam

Can you provide a breakup of the last date and time field...

Hi Shamrock,
The time is distributed as :

2013-05-14T00:52:31.662-04:00 indicates YYYY-MM-DDThh:mm:ss.[miliseconds]-[GMT-4].

For me YYYY-MM-DDThh:mm:ss is ok if it can be sorted

This sort of thing is best doable in perl...

Code:

#!/usr/bin/perl

use warnings;
use Time::Local;
use Time::localtime;

while (<>) {
    chomp($_);
    @f = split(/\|/, $_);
    @dnt = split(/T/, $f[7]);
    $gmt = 4*60*60;

    my($yr, $mo, $dy) = split(/-/, $dnt[0]);
    my($hr, $mi, $sx) = split(/:/, $dnt[1]);
    my($se, $ms) = split(/\./, $sx);

    $ep = timelocal($se,$mi,$hr,$dy,$mo,$yr) + $gmt;
    push(@{$rec{$ep}}, $_);
}

foreach $t (sort keys %rec) {
    printf("%s\n", @{$rec{$t}});
};

Save the above perl script in a file and run it with your input file as an argument...

Code:

my_perl_script ref.psv