Sponsored Content
Top Forums UNIX for Advanced & Expert Users Alternative to sort -ur +1 required Post 302712493 by Don Cragun on Tuesday 9th of October 2012 08:38:57 AM
Old 10-09-2012
Quote:
Originally Posted by Mike Smith
Tried -k2,2 and it binned loads of old entries which I wanted!

Say I have this data, the highest number (newest) one will always be at the top and I only want to keep the top one as everything right of, and including, column 2 is the same (nothing has changed since last sweep).
1349455502 ygtr-1b:3/1/19 10/100/Gig Ethernet SFP
1349246545 ygtr-1b:3/1/19 10/100/Gig Ethernet SFP

However if we get the following then I want both entries retaining
1349455502 ygtr-1b:3/1/19 10/100/Gig Ethernet SFP
1349246545 ygtr-1b:3/1/19 Old customer name typed in here

-k2,2 seems to ONLY consider column 2 which is no good based on my first post's criteria.

This is tricky!
It isn't tricky at all. You can't say that the first field doesn't matter when throwing away duplicates and at the same time say that you want the highest value for the first field to be the one that is kept when throwing away duplicates. If you care which duplicate is kept, you don't think they are duplicates. If you want both, you have to do it in two separate steps:
  1. sort in reverse order with the data from the start of field 2 to the end of the line as your primary sort key and the numeric value in field 1 as your secondary sort key, and then
  2. on a second pass, discard or ignore the second and subsequent lines that match (including field separators other than the separator between the 1st and 2nd fields) from the start of the 2nd field to the end of the line.

So why does it now matter which timestamp is kept when throwing away duplicates if it didn't matter when you were running this application on Solaris 8?
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

sort out the required data

Hi All, I have a file 1.txt which has the duplicate dns entries as shown: Name: 000f9fbc6738.net.in|Addresses: 10.241.66.169, 10.84.2.222,212.241.66.170 Name: 001371e8ed3e.net.in|Addresses: 10.241.65.153, 10.84.1.101 Name: 00e06f5bd42a.net.in|Addresses: 10.72.19.218,... (6 Replies)
Discussion started by: imas
6 Replies

2. Shell Programming and Scripting

Getting required fields from a test file in required fromat in unix

My data is something like shown below. date1 date2 aaa bbbb ccccc date3 date4 dddd eeeeeee ffffffffff ggggg hh I want the output like this date1date2 aaa eeeeee I serached in the forum but didn't find the exact matching solution. Please help. (7 Replies)
Discussion started by: rdhanek
7 Replies

3. Shell Programming and Scripting

How to Sort Floating Numbers Using the Sort Command?

Hi to all. I'm trying to sort this with the Unix command sort. user1:12345678:3.5:2.5:8:1:2:3 user2:12345679:4.5:3.5:8:1:3:2 user3:12345687:5.5:2.5:6:1:3:2 user4:12345670:5.5:2.5:5:3:2:1 user5:12345671:2.5:5.5:7:2:3:1 I need to get this: user3:12345687:5.5:2.5:6:1:3:2... (7 Replies)
Discussion started by: daniel.gbaena
7 Replies

4. UNIX for Dummies Questions & Answers

How to insert alternative columns and sort text from first column to second?

Hi Everybody, I am just new to UNIX as well as to this forum. I have a text file with 10,000 coloumns and each coloumn contains values separated by space. I want to separate them into new coloumns..the file is something like this as ad af 1 A as ad af 1 D ... ... 1 and A are in one... (7 Replies)
Discussion started by: Unilearn
7 Replies

5. UNIX for Advanced & Expert Users

Script to sort the files and append the extension .sort to the sorted version of the file

Hello all - I am to this forum and fairly new in learning unix and finding some difficulty in preparing a small shell script. I am trying to make script to sort all the files given by user as input (either the exact full name of the file or say the files matching the criteria like all files... (3 Replies)
Discussion started by: pankaj80
3 Replies

6. Shell Programming and Scripting

Want to sort a file using awk & sed to get required output

Hi All, Need Suggestion, Want to sort a file using awk & sed to get required, output as below, such that each LUN shows correct WWPN and FA port Numbers correctly: Required output: 01FB 10000000c97843a2 8C 0 01FB 10000000c96fb279 9C 0 22AF 10000000c97843a2 8C 0 22AF 10000000c975adbd ... (10 Replies)
Discussion started by: aix_admin_007
10 Replies

7. Shell Programming and Scripting

Help with sort word and general numeric sort at the same time

Input file: 100%ABC2 3.44E-12 USA A2M%H02579 0E0 UK 100%ABC2 5.34E-8 UK 100%ABC2 3.25E-12 USA A2M%H02579 5E-45 UK Output file: 100%ABC2 3.44E-12 USA 100%ABC2 3.25E-12 USA 100%ABC2 5.34E-8 UK A2M%H02579 0E0 UK A2M%H02579 5E-45 UK Code try: sort -k1,1 -g -k2 -r input.txt... (2 Replies)
Discussion started by: perl_beginner
2 Replies

8. Shell Programming and Scripting

Sort help: How to sort collected 'file list' by date stamp :

Hi Experts, I have a filelist collected from another server , now want to sort the output using date/time stamp filed. - Filed 6, 7,8 are showing the date/time/stamp. Here is the input: #---------------------------------------------------------------------- -rw------- 1 root ... (3 Replies)
Discussion started by: rveri
3 Replies

9. UNIX for Dummies Questions & Answers

Best Alternative for checking input parameter contains required value or not

Any good way to check if code has the required output # /sbin/sysctl net.ipv4.icmp_echo_ignore_broadcasts net.ipv4.icmp_echo_ignore_broadcasts = 1 /sbin/sysctl net.ipv4.icmp_echo_ignore_broadcasts | grep "= 1" net.ipv4.icmp_echo_ignore_broadcasts = 1 What I can think of is above, and it... (16 Replies)
Discussion started by: alvinoo
16 Replies

10. Shell Programming and Scripting

How to Modify a file content in UNIX and sort for only required fields ?

I have the below contents in a file after making the below curl call curl ... | grep -E "state|Rno" | paste -sd',\n' | grep "Disconnected" > test "state" : "Disconnected",, "Rno" : "5554f1d2" "state" : "Disconnected",, "Rno" : "10587563" "state" : "Disconnected",, "Rno" :... (2 Replies)
Discussion started by: Vaibhav H
2 Replies
Fields(3pm)						User Contributed Perl Documentation					       Fields(3pm)

NAME
Sort::Fields - Sort lines containing delimited fields SYNOPSIS
use Sort::Fields; @sorted = fieldsort [3, '2n'], @lines; @sorted = fieldsort '+', [-1, -3, 0], @lines; $sort_3_2n = make_fieldsort [3, '2n'], @lines; @sorted = $sort_3_2n->(@lines); DESCRIPTION
Sort::Fields provides a general purpose technique for efficiently sorting lists of lines that contain data separated into fields. Sort::Fields automatically imports two subroutines, "fieldsort" and "make_fieldsort", and two variants, "stable_fieldsort" and "make_sta- ble_fieldsort". "make_fieldsort" generates a sorting subroutine and returns a reference to it. "fieldsort" is a wrapper for the "make_fieldsort" subroutine. The first argument to make_fieldsort is a delimiter string, which is used as a regular expression argument for a "split" operator. The delimiter string is optional. If it is not supplied, make_fieldsort splits each line using "/s+/". The second argument is an array reference containing one or more field specifiers. The specifiers indicate what fields in the strings will be used to sort the data. The specifier "1" indicates the first field, "2" indicates the second, and so on. A negative specifier like "-2" means to sort on the second field in reverse (descending) order. To indicate a numeric rather than alphabetic comparison, append "n" to the specifier. A specifier of "0" means the entire string ("-0" means the entire string, in reverse order). The order in which the specifiers appear is the order in which they will be used to sort the data. The primary key is first, the secondary key is second, and so on. "fieldsort [1, 2], @data" is roughly equivalent to "make_fieldsort([1, 2])->(@data)". Avoid calling fieldsort repeatedly with the same sort specifiers. If you need to use a particular sort more than once, it is more efficient to call "make_fieldsort" once and reuse the subroutine it returns. "stable_fieldsort" and "make_stable_fieldsort" are like their "unstable" counterparts, except that the items that compare the same are maintained in their original order. EXAMPLES
Some sample data (in array @data): 123 asd 1.22 asdd 32 ewq 2.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 23 erww 4.21 ewet 91 fdgs 3.43 ewet 123 refs 3.22 asdd 123 refs 4.32 asdd # alpha sort on column 1 print fieldsort [1], @data; 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 23 erww 4.21 ewet 32 ewq 2.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet # numeric sort on column 1 print fieldsort ['1n'], @data; 23 erww 4.21 ewet 32 ewq 2.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd # reverse numeric sort on column 1 print fieldsort ['-1n'], @data; 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 91 fdgs 3.43 ewet 51 erwt 34.2 ewet 43 rewq 2.12 ewet 32 ewq 2.32 asdd 23 erww 4.21 ewet # alpha sort on column 2, then alpha on entire line print fieldsort [2, 0], @data; 123 asd 1.22 asdd 51 erwt 34.2 ewet 23 erww 4.21 ewet 32 ewq 2.32 asdd 91 fdgs 3.43 ewet 123 refs 3.22 asdd 123 refs 4.32 asdd 43 rewq 2.12 ewet # alpha sort on column 4, then numeric on column 1, then reverse # numeric on column 3 print fieldsort [4, '1n', '-3n'], @data; 32 ewq 2.32 asdd 123 refs 4.32 asdd 123 refs 3.22 asdd 123 asd 1.22 asdd 23 erww 4.21 ewet 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet # now, splitting on either literal period or whitespace # sort numeric on column 4 (fractional part of decimals) then # numeric on column 3 (whole part of decimals) print fieldsort '(?:.|s+)', ['4n', '3n'], @data; 51 erwt 34.2 ewet 43 rewq 2.12 ewet 23 erww 4.21 ewet 123 asd 1.22 asdd 123 refs 3.22 asdd 32 ewq 2.32 asdd 123 refs 4.32 asdd 91 fdgs 3.43 ewet # alpha sort on column 4, then numeric on the entire line # NOTE: produces warnings under -w print fieldsort [4, '0n'], @data; 32 ewq 2.32 asdd 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 23 erww 4.21 ewet 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet # stable alpha sort on column 4 (maintains original relative order # among items that compare the same) print stable_fieldsort [4], @data; 123 asd 1.22 asdd 32 ewq 2.32 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 23 erww 4.21 ewet 91 fdgs 3.43 ewet BUGS
Some rudimentary tests now. Perhaps something should be done to catch things like: fieldsort '.', [1, 2], @lines; '.' translates to "split /./" -- probably not what you want. Passing blank lines and/or lines containing the wrong kind of data (alphas instead of numbers) can result in copious warning messages under "-w". If the regexp contains memory parentheses ("(...)" rather than "(?:...)"), split will function in "delimiter retention" mode, capturing the contents of the parentheses as well as the stuff between the delimiters. I could imagine how this could be useful, but on the other hand I could also imagine how it could be confusing if encountered unexpectedly. Caveat sortor. Not really a bug, but if you are planning to sort a large text file, consider using sort(1). Unless, of course, your operating system doesn't have sort(1). AUTHOR
Joseph N. Hall, joseph@5sigma.com SEE ALSO
perl(1). perl v5.8.8 2008-03-25 Fields(3pm)
All times are GMT -4. The time now is 09:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy