Sponsored Content
Full Discussion: sort | uniq question
Top Forums Shell Programming and Scripting sort | uniq question Post 302493279 by palex on Wednesday 2nd of February 2011 09:47:43 AM
Old 02-02-2011
sort | uniq question

Hello,
I have a large data file:
Code:
1234 8888 bbb
2745 8888 bbb
9489 8888 bbb
1234 8888 aaa
4838 8888 aaa
3977 8888 aaa

I need to remove duplicate lines (where the first column is the duplicate). I have been using:
Code:
sort file.txt | uniq -w4 > newfile.txt

However, it seems to keep the first of the duplicate pair alphabetically. So in the example above,
Code:
1234 8888 aaa    would be kept, and
1234 8888 bbb    would be excluded

I need to modify the command so that the first of the two lines *chronologically* would be kept (In this case, 1234 8888 bbb).

Thanks so much!

Last edited by Scott; 02-02-2011 at 03:47 PM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sort/uniq

I have a file: Fred Fred Fred Jim Fred Jim Jim If sort is executed on the listed file, shouldn't the output be?: Fred Fred Fred Fred Jim Jim Jim (3 Replies)
Discussion started by: jimmyflip
3 Replies

2. UNIX for Dummies Questions & Answers

Help with Last,uniq, sort and cut

Using the last, uniq, sort and cut commands, determine how many times the different users have logged in. I know how to use the last command and cut command... i came up with last | cut -f1 -d" " | uniq i dont know if this is right, can someone please help me... thanks (1 Reply)
Discussion started by: jay1228
1 Replies

3. Shell Programming and Scripting

sort and uniq in perl

Does anyone have a quick and dirty way of performing a sort and uniq in perl? How an array with data like: this is bkupArr BOLADVICE_VN this is bkupArr MLT6800PROD2A this is bkupArr MLT6800PROD2A this is bkupArr BOLADVICE_VN_7YR this is bkupArr MLT6800PROD2A I want to sort it... (4 Replies)
Discussion started by: reggiej
4 Replies

4. Shell Programming and Scripting

Sort, Uniq, Duplicates

Input File is : ------------- 25060008,0040,03, 25136437,0030,03, 25069457,0040,02, 80303438,0014,03,1st 80321837,0009,03,1st 80321977,0009,03,1st 80341345,0007,03,1st 84176527,0047,03,1st 84176527,0047,03, 20000735,0018,03,1st 25060008,0040,03, I am using the following in the script... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies

5. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

6. Shell Programming and Scripting

Sort and uniq after comparision

Hi All, I have a text file with the format shown below. Some of the records are duplicated with the only exception being date (Field 15). I want to compare all duplicate records using subscriber number (field 7) and keep only those records with greater date. ... (1 Reply)
Discussion started by: nua7
1 Replies

7. Shell Programming and Scripting

Sort field and uniq

I have a flatfile A.txt 2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11 How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies

8. Shell Programming and Scripting

Sort uniq or awk

Hi again, I have files with the following contents datetime,ip1,port1,ip2,port2,number How would I find out how many times ip1 field shows up a particular file? Then how would I find out how many time ip1 and port 2 shows up? Please mind the file may contain 100k lines. (8 Replies)
Discussion started by: LDHB2012
8 Replies

9. Shell Programming and Scripting

Uniq or sort -u or similar only between { }

Hi ! I am trying to remove doubbled entrys in a textfile only between delimiters. Like that example but i dont know how to do that with sort or similar. input: { aaa aaa } { aaa aaa } output: { aaa } { (8 Replies)
Discussion started by: fugitivus
8 Replies

10. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies
libapache2-mod-perl2-2.0.7::docs::api::Apache2::URI(3pm)User Contributed Perl Documentatiolibapache2-mod-perl2-2.0.7::docs::api::Apache2::URI(3pm)

NAME
Apache2::URI - Perl API for manipulating URIs Synopsis use Apache2::URI (); $hostport = $r->construct_server(); $hostport = $r->construct_server($hostname); $hostport = $r->construct_server($hostname, $port); $hostport = $r->construct_server($hostname, $port, $pool); $url = $r->construct_url(); $url = $r->construct_url($rel_uri); $url = $r->construct_url($rel_uri, $pool); $parsed_uri = $r->parse_uri($uri); $parsed_uri = $r->parsed_uri(); $url = join '%20', qw(one two three); Apache2::URI::unescape_url($url); Description While "APR::URI" provides a generic API to dissect, adjust and put together any given URI string, "Apache2::URI" provides an API specific to Apache, by taking the information directly from the $r object. Therefore when manipulating the URI of the current HTTP request usually methods from both classes are used. API
"Apache2::URI" provides the following functions and methods: "construct_server" Construct a string made of hostname and port $hostport = $r->construct_server(); $hostport = $r->construct_server($hostname); $hostport = $r->construct_server($hostname, $port); $hostport = $r->construct_server($hostname, $port, $pool); obj: $r ( "Apache2::RequestRec object" ) The current request object opt arg1: $hostname ( string ) The hostname of the server. If that argument is not passed, "$r->get_server_name" is used. opt arg2: $port ( string ) The port the server is running on. If that argument is not passed, "$r->get_server_port" is used. opt arg3: $pool ( "APR::Pool object" ) The pool to allocate the string from. If that argument is not passed, "$r->pool" is used. ret: $hostport ( string ) The server's hostport string since: 2.0.00 Examples: o Assuming that: $r->get_server_name == "localhost"; $r->get_server_port == 8001; The code: $hostport = $r->construct_server(); returns a string: localhost:8001 o The following code sets the values explicitly: $hostport = $r->construct_server("my.example.com", 8888); and it returns a string: my.example.com:8888 "construct_url" Build a fully qualified URL from the uri and information in the request rec: $url = $r->construct_url(); $url = $r->construct_url($rel_uri); $url = $r->construct_url($rel_uri, $pool); obj: $r ( "Apache2::RequestRec object" ) The current request object opt arg1: $rel_uri ( string ) The path to the requested file (it may include a concatenation of path, query and fragment components). If that argument is not passed, "$r->uri" is used. opt arg2: $pool ( "APR::Pool object" ) The pool to allocate the URL from If that argument is not passed, "$r->pool" is used. ret: $url ( string ) A fully qualified URL since: 2.0.00 Examples: o Assuming that the request was http://localhost.localdomain:8529/test?args The code: my $url = $r->construct_url; returns the string: http://localhost.localdomain:8529/test notice that the query (args) component is not in the string. You need to append it manually if it's needed. o Assuming that the request was http://localhost.localdomain:8529/test?args The code: my $rel_uri = "/foo/bar?tar"; my $url = $r->construct_url($rel_uri); returns the string: http://localhost.localdomain:8529/foo/bar?tar "parse_uri" Break apart URI (affecting the current request's uri components) $r->parse_uri($uri); obj: $r ( "Apache2::RequestRec object" ) The current request object arg1: $uri ( string ) The uri to break apart ret: no return value warning: This method has several side-effects explained below since: 2.0.00 This method call has the following side-effects: 1. sets "$r->args" to the rest after '?' if such exists in the passed $uri, otherwise sets it to "undef". 2. sets "$r->uri" to the passed $uri without the "$r->args" part. 3. sets "$r->hostname" (if not set already) using the ("scheme://host:port") parts of the passed $uri. "parsed_uri" Get the current request's parsed uri object my $uri = $r->parsed_uri(); obj: $r ( "Apache2::RequestRec object" ) The current request object ret: $uri ( "APR::URI object" ) The parsed uri since: 2.0.00 This object is suitable for using with "APR::URI::rpath" "unescape_url" Unescape URLs Apache2::URI::unescape_url($url); obj: $url ( string ) The URL to unescape ret: no return value The argument $url is now unescaped since: 2.0.00 Example: my $url = join '%20', qw(one two three); Apache2::URI::unescape_url($url); $url now contains the string: "one two three"; See Also "APR::URI", mod_perl 2.0 documentation. Copyright mod_perl 2.0 and its core modules are copyrighted under The Apache Software License, Version 2.0. Authors The mod_perl development team and numerous contributors. perl v5.14.2 2011-02-08 libapache2-mod-perl2-2.0.7::docs::api::Apache2::URI(3pm)
All times are GMT -4. The time now is 02:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy