Sponsored Content
Top Forums Shell Programming and Scripting Why is sort -k not working all the time? Post 302683871 by alan on Wednesday 8th of August 2012 06:51:08 PM
Old 08-08-2012
Why is sort -k not working all the time?

I have a script that puts a list of files in two separate arrays:

First, I get a file list from a ZIP file and fill `FIRST_Array()` with it. Second, I get a file list from a control file within a ZIP file and fill `SECOND_Array()` with it

Code:
                while read length date time filename 
                do
                        FIRST_Array+=( "$filename" )
                        echo "$filename" >> FIRST.report.out
                done < <(/usr/bin/unzip -qql AAA.ZIP |sort -k12 -t~)

Third, I compare both array like so:

Code:
    diff -q <(printf "%s\n" "${FIRST_Array[@]}") <(printf "%s\n" "${SECOND_Array[@]}") |wc -l

I can tell that `Diff` fails because I output each array to files: `FIRST.report.out` and `SECOND.report.out` are simply not sorted properly.

1) FIRST.report.out (what's inside the ZIP file)


Code:
JGS-Memphis~AT1~Pre-Test~X-BanhT~JGMDTV387~6~P~1100~HR24-500~033072053326~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-GuinE~JGMDTV069~6~P~1100~H24-700~033081107519~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-MooreBe~JGM98745~40~P~1100~H21-200~029264526103~20120808~240914.XML
JGS-Memphis~FUN~Pre-Test~X-RossA~jgmdtv168~2~P~1100~H21-200~029415655926~20120808~240914.XML

2) SECOND.report.out (what's inside the ZIP's control file)

Code:
JGS-Memphis~AT1~Pre-Test~X-BanhT~JGMDTV387~6~P~1100~HR24-500~033072053326~20120808~240914.XML
JGS-Memphis~FUN~Pre-Test~X-RossA~jgmdtv168~2~P~1100~H21-200~029415655926~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-GuinE~JGMDTV069~6~P~1100~H24-700~033081107519~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-MooreBe~JGM98745~40~P~1100~H21-200~029264526103~20120808~240914.XML

Using sort -k12 -t~ made sense since ~ is the delimiter for the file's date field ("20120808" : 12th position). But it is not working consistently.

The sort is worse when my script processes bigger ZIP files. Why is sort -k not working all the time? How can I sort both arrays?

Last edited by alan; 08-08-2012 at 07:56 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to numeric sort on field time

i want to sort time field given by who command as a whole i have tried like this who|sort -n +4 -5 (1 Reply)
Discussion started by: rahulspatil_111
1 Replies

2. Shell Programming and Scripting

Sort By Date and Time

Hi , I would like to list or sort by date and time (the files are named in day and time format) where the latest file will be placed at the bottom and the earliest file be placed at the top. Can anybody help me? My files are named in the following manner. EG: abc_071128_144121_data "... (21 Replies)
Discussion started by: Raynon
21 Replies

3. Shell Programming and Scripting

Script not working..."sort" not working properly....

Hello all, I have a file - 12.txt cat 12.txt =============================================== Number of executions = 2 Total execution time (sec.ms) = 0.009883 Number of executions = 8 Total execution time (sec.ms) = 0.001270 Number of... (23 Replies)
Discussion started by: Rahulpict
23 Replies

4. UNIX for Dummies Questions & Answers

Sort two files by time

I had a question about sorting two files which contain data like the following: File 1: 09:12:12 blah blah 0 blah blah 1 blah blah 2 blah blah 09:12:15 blah blah 0 blah blah 1 blah blah 2 blah blah File 2: 09:12:13 blah2 0 blah2 1 blah2 2 ... (1 Reply)
Discussion started by: rowd23
1 Replies

5. Shell Programming and Scripting

Sort by name, time, and size

How do you combine these ls commands so that I can have the outputs by name, time stamp, and size? ls -al |grep name_of_file ls -al | sort +4nr ls -l -t Please advise. (4 Replies)
Discussion started by: Daniel Gate
4 Replies

6. Shell Programming and Scripting

sort not working

// AIX 5.3 & 6.1 I am trying to sort by size. # fuser -OdV /tmp | sort -n -t= -k 3,3 But, it gives me like below. inode=3502 size=193106 fd=1 inode=3502 size=193106 fd=2 inode=3513 size=139474 fd=2 inode=3511 size=261016 fd=1 inode=3502 ... (8 Replies)
Discussion started by: Daniel Gate
8 Replies

7. Shell Programming and Scripting

How to Sort by Date and Time?

Hi, The input file is as follows, 22.06.2012 17:58:38 CPUser: xxxxxxx, billedAfterStatus: Active 13.07.2012 08:46:15 CPUser: xxxxxxx, billedAfterStatus: Active 20.07.2012 08:56:24 CPUser: xxxxxxx, billedAfterStatus: Active 20.03.2012 08:56:24 CPUser: xxxxxxx, billedAfterStatus: Active... (16 Replies)
Discussion started by: nanthagopal
16 Replies

8. Shell Programming and Scripting

Help with sort word and general numeric sort at the same time

Input file: 100%ABC2 3.44E-12 USA A2M%H02579 0E0 UK 100%ABC2 5.34E-8 UK 100%ABC2 3.25E-12 USA A2M%H02579 5E-45 UK Output file: 100%ABC2 3.44E-12 USA 100%ABC2 3.25E-12 USA 100%ABC2 5.34E-8 UK A2M%H02579 0E0 UK A2M%H02579 5E-45 UK Code try: sort -k1,1 -g -k2 -r input.txt... (2 Replies)
Discussion started by: perl_beginner
2 Replies

9. Shell Programming and Scripting

Why is sort not working properly here ?

Platform: RHEL 5.4 In the below text file I have strings like following. $ cat /tmp/mytextfile.txt DISK1 DISK10 DISK101 DISK102 DISK103 DISK104 DISK105 DISK106 DISK107 DISK108 DISK109 DISK110 DISK111 DISK112 DISK113 DISK114 (8 Replies)
Discussion started by: kraljic
8 Replies

10. Shell Programming and Scripting

Sort based on column 1, not working with awk

Hi Guru, I need some help regarding awking the output so it only show the first line (based on column) of each row. So If column has 1, three row, then it only show the first line of that row, based on similar character in column 1. So i am trying to achieve a sort, based on column one and... (3 Replies)
Discussion started by: Junes
3 Replies
sort(3pm)						 Perl Programmers Reference Guide						 sort(3pm)

NAME
sort - perl pragma to control sort() behaviour SYNOPSIS
use sort 'stable'; # guarantee stability use sort '_quicksort'; # use a quicksort algorithm use sort '_mergesort'; # use a mergesort algorithm use sort 'defaults'; # revert to default behavior no sort 'stable'; # stability not important use sort '_qsort'; # alias for quicksort my $current = sort::current(); # identify prevailing algorithm DESCRIPTION
With the "sort" pragma you can control the behaviour of the builtin "sort()" function. In Perl versions 5.6 and earlier the quicksort algorithm was used to implement "sort()", but in Perl 5.8 a mergesort algorithm was also made available, mainly to guarantee worst case O(N log N) behaviour: the worst case of quicksort is O(N**2). In Perl 5.8 and later, quick- sort defends against quadratic behaviour by shuffling large arrays before sorting. A stable sort means that for records that compare equal, the original input ordering is preserved. Mergesort is stable, quicksort is not. Stability will matter only if elements that compare equal can be distinguished in some other way. That means that simple numerical and lexical sorts do not profit from stability, since equal elements are indistinguishable. However, with a comparison such as { substr($a, 0, 3) cmp substr($b, 0, 3) } stability might matter because elements that compare equal on the first 3 characters may be distinguished based on subsequent characters. In Perl 5.8 and later, quicksort can be stabilized, but doing so will add overhead, so it should only be done if it matters. The best algorithm depends on many things. On average, mergesort does fewer comparisons than quicksort, so it may be better when compli- cated comparison routines are used. Mergesort also takes advantage of pre-existing order, so it would be favored for using "sort()" to merge several sorted arrays. On the other hand, quicksort is often faster for small arrays, and on arrays of a few distinct values, repeated many times. You can force the choice of algorithm with this pragma, but this feels heavy-handed, so the subpragmas beginning with a "_" may not persist beyond Perl 5.8. The default algorithm is mergesort, which will be stable even if you do not explicitly demand it. But the stability of the default sort is a side-effect that could change in later versions. If stability is important, be sure to say so with a use sort 'stable'; The "no sort" pragma doesn't forbid what follows, it just leaves the choice open. Thus, after no sort qw(_mergesort stable); a mergesort, which happens to be stable, will be employed anyway. Note that no sort "_quicksort"; no sort "_mergesort"; have exactly the same effect, leaving the choice of sort algorithm open. CAVEATS
This pragma is not lexically scoped: its effect is global to the program it appears in. That means the following will probably not do what you expect, because both pragmas take effect at compile time, before either "sort()" happens. { use sort "_quicksort"; print sort::current . " "; @a = sort @b; } { use sort "stable"; print sort::current . " "; @c = sort @d; } # prints: # quicksort stable # quicksort stable You can achieve the effect you probably wanted by using "eval()" to defer the pragmas until run time. Use the quoted argument form of "eval()", not the BLOCK form, as in eval { use sort "_quicksort" }; # WRONG or the effect will still be at compile time. Reset to default options before selecting other subpragmas (in case somebody carelessly left them on) and after sorting, as a courtesy to others. { eval 'use sort qw(defaults _quicksort)'; # force quicksort eval 'no sort "stable"'; # stability not wanted print sort::current . " "; @a = sort @b; eval 'use sort "defaults"'; # clean up, for others } { eval 'use sort qw(defaults stable)'; # force stability print sort::current . " "; @c = sort @d; eval 'use sort "defaults"'; # clean up, for others } # prints: # quicksort # stable Scoping for this pragma may change in future versions. perl v5.8.0 2002-06-01 sort(3pm)
All times are GMT -4. The time now is 10:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy