Why is sort -k not working all the time?

Login or Register to Ask a Question and Join Our Community

Why is sort -k not working all the time?

Tags

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting Why is sort -k not working all the time?

08-08-2012

Registered User

93, 3

Join Date: Jul 2003

Last Activity: 29 June 2015, 5:58 PM EDT

Location: Earth

Posts: 93

Thanks Given: 8

Thanked 3 Times in 2 Posts

Why is sort -k not working all the time?

I have a script that puts a list of files in two separate arrays:

First, I get a file list from a ZIP file and fill `FIRST_Array()` with it. Second, I get a file list from a control file within a ZIP file and fill `SECOND_Array()` with it

Code:

                while read length date time filename 
                do
                        FIRST_Array+=( "$filename" )
                        echo "$filename" >> FIRST.report.out
                done < <(/usr/bin/unzip -qql AAA.ZIP |sort -k12 -t~)

Third, I compare both array like so:

Code:

    diff -q <(printf "%s\n" "${FIRST_Array[@]}") <(printf "%s\n" "${SECOND_Array[@]}") |wc -l

I can tell that `Diff` fails because I output each array to files: `FIRST.report.out` and `SECOND.report.out` are simply not sorted properly.

1) FIRST.report.out (what's inside the ZIP file)

Code:

JGS-Memphis~AT1~Pre-Test~X-BanhT~JGMDTV387~6~P~1100~HR24-500~033072053326~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-GuinE~JGMDTV069~6~P~1100~H24-700~033081107519~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-MooreBe~JGM98745~40~P~1100~H21-200~029264526103~20120808~240914.XML
JGS-Memphis~FUN~Pre-Test~X-RossA~jgmdtv168~2~P~1100~H21-200~029415655926~20120808~240914.XML

2) SECOND.report.out (what's inside the ZIP's control file)

Code:

JGS-Memphis~AT1~Pre-Test~X-BanhT~JGMDTV387~6~P~1100~HR24-500~033072053326~20120808~240914.XML
JGS-Memphis~FUN~Pre-Test~X-RossA~jgmdtv168~2~P~1100~H21-200~029415655926~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-GuinE~JGMDTV069~6~P~1100~H24-700~033081107519~20120808~240914.XML
JGS-Memphis~PRE~DTV_PREP~X-MooreBe~JGM98745~40~P~1100~H21-200~029264526103~20120808~240914.XML

Using sort -k12 -t~ made sense since ~ is the delimiter for the file's date field ("20120808" : 12th position). But it is not working consistently.

The sort is worse when my script processes bigger ZIP files. Why is sort -k not working all the time? How can I sort both arrays?

Last edited by alan; 08-08-2012 at 07:56 PM..

alan

View Public Profile for alan

Find all posts by alan

08-08-2012

Moderator

3,791, 1,452

Join Date: Oct 2010

Last Activity: 1 August 2020, 1:38 AM EDT

Posts: 3,791

Thanks Given: 183

Thanked 1,452 Times in 1,302 Posts

If your sort supports try --debug option to see what it's sorting by. Field 12 is the filename.

Try sort -t~ -k10,11 to sort by the 12 digit number and then date

Chubler_XL

View Public Profile for Chubler_XL

Find all posts by Chubler_XL

08-08-2012

Registered User

440, 71

Join Date: Oct 2009

Last Activity: 26 June 2018, 6:52 PM EDT

Location: spaceBAR Central

Posts: 440

Thanks Given: 0

Thanked 71 Times in 70 Posts

Don't you want to sort the whole file name(from both lists) so both lists will be in the same order?

spacebar

View Public Profile for spacebar

Find all posts by spacebar

08-08-2012

Registered User

12,315, 4,560

Join Date: Jul 2012

Last Activity: 22 November 2019, 4:29 PM EST

Location: San Jose, CA, USA

Posts: 12,315

Thanks Given: 952

Thanked 4,560 Times in 3,818 Posts

What shell are you using? I'm not familiar with a redirection operator of the form:

Code:

< <(command pipeline)

Sorry. Never mind. It is a bash feature. This form is not available in ksh and is an extension to the POSIX shell.

Last edited by Don Cragun; 08-08-2012 at 09:39 PM..

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

08-08-2012

Registered User

3,231, 978

Join Date: Dec 2009

Last Activity: 11 June 2014, 8:40 PM EDT

Posts: 3,231

Thanks Given: 179

Thanked 978 Times in 791 Posts

Quote:

Originally Posted by Don Cragun

What shell are you using? I'm not familiar with a redirection operator of the form:

Code:

< <(command pipeline)

Sorry. Never mind. It is a bash feature. This form is not available in ksh and is an extension to the POSIX shell.

ksh88 and ksh93 both support that construct. The process substitution, <(cmd ...) is replaced with /dev/fd/n, and the first < is the usual input redirection operator.

Regards,
Alister

alister

View Public Profile for alister

Find all posts by alister

08-08-2012

Registered User

12,315, 4,560

Join Date: Jul 2012

Last Activity: 22 November 2019, 4:29 PM EST

Location: San Jose, CA, USA

Posts: 12,315

Thanks Given: 952

Thanked 4,560 Times in 3,818 Posts

Quote:

Originally Posted by alister

ksh88 and ksh93 both support that construct. The process substitution, <(cmd ...) is replaced with /dev/fd/n, and the first < is the usual input redirection operator.

Regards,
Alister

That's interesting. The ksh on Mac OS X Lion (Version M 1993-12-28 s+ $) says:

Quote:

-ksh: syntax error: `<(' unexpected

when given the command:

Code:

while read line;do echo x $line;done < <(printf "a\nb c\nd e f\n")

The New Kornshell Command and Programming Language book by Nolsky and Korn does say that Process Substitution using this form is a "Possible Extension" that

Quote:

is available only on versions of the UNIX system that support the /dev/fd directory for naming open files.

and OS X does support /dev/fd. It looks like someone at Apple failed to configure OS X's ksh to give us access to this feature.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

08-09-2012

Registered User

93, 3

Join Date: Jul 2003

Last Activity: 29 June 2015, 5:58 PM EDT

Location: Earth

Posts: 93

Thanks Given: 8

Thanked 3 Times in 2 Posts

Quote:

Originally Posted by spacebar

Don't you want to sort the whole file name(from both lists) so both lists will be in the same order?

Yes, I do want to sort the whole file name.

I tried to do that by using "|sort" instead of "sort -k..." but the problem remains.

alan

View Public Profile for alan

Find all posts by alan

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sort based on column 1, not working with awk

Hi Guru, I need some help regarding awking the output so it only show the first line (based on column) of each row. So If column has 1, three row, then it only show the first line of that row, based on similar character in column 1. So i am trying to achieve a sort, based on column one and...

2. Shell Programming and Scripting

Why is sort not working properly here ?

Platform: RHEL 5.4 In the below text file I have strings like following. $ cat /tmp/mytextfile.txt DISK1 DISK10 DISK101 DISK102 DISK103 DISK104 DISK105 DISK106 DISK107 DISK108 DISK109 DISK110 DISK111 DISK112 DISK113 DISK114

3. Shell Programming and Scripting

Help with sort word and general numeric sort at the same time

Input file: 100%ABC2 3.44E-12 USA A2M%H02579 0E0 UK 100%ABC2 5.34E-8 UK 100%ABC2 3.25E-12 USA A2M%H02579 5E-45 UK Output file: 100%ABC2 3.44E-12 USA 100%ABC2 3.25E-12 USA 100%ABC2 5.34E-8 UK A2M%H02579 0E0 UK A2M%H02579 5E-45 UK Code try: sort -k1,1 -g -k2 -r input.txt...

4. Shell Programming and Scripting

How to Sort by Date and Time?

Hi, The input file is as follows, 22.06.2012 17:58:38 CPUser: xxxxxxx, billedAfterStatus: Active 13.07.2012 08:46:15 CPUser: xxxxxxx, billedAfterStatus: Active 20.07.2012 08:56:24 CPUser: xxxxxxx, billedAfterStatus: Active 20.03.2012 08:56:24 CPUser: xxxxxxx, billedAfterStatus: Active...

5. Shell Programming and Scripting

sort not working

// AIX 5.3 & 6.1 I am trying to sort by size. # fuser -OdV /tmp | sort -n -t= -k 3,3 But, it gives me like below. inode=3502 size=193106 fd=1 inode=3502 size=193106 fd=2 inode=3513 size=139474 fd=2 inode=3511 size=261016 fd=1 inode=3502 ...

6. Shell Programming and Scripting

Sort by name, time, and size

How do you combine these ls commands so that I can have the outputs by name, time stamp, and size? ls -al |grep name_of_file ls -al | sort +4nr ls -l -t Please advise.

7. UNIX for Dummies Questions & Answers

Sort two files by time

I had a question about sorting two files which contain data like the following: File 1: 09:12:12 blah blah 0 blah blah 1 blah blah 2 blah blah 09:12:15 blah blah 0 blah blah 1 blah blah 2 blah blah File 2: 09:12:13 blah2 0 blah2 1 blah2 2 ...

8. Shell Programming and Scripting

Script not working..."sort" not working properly....

Hello all, I have a file - 12.txt cat 12.txt =============================================== Number of executions = 2 Total execution time (sec.ms) = 0.009883 Number of executions = 8 Total execution time (sec.ms) = 0.001270 Number of...

9. Shell Programming and Scripting

Sort By Date and Time

Hi , I would like to list or sort by date and time (the files are named in day and time format) where the latest file will be placed at the bottom and the earliest file be placed at the top. Can anybody help me? My files are named in the following manner. EG: abc_071128_144121_data "...

10. Shell Programming and Scripting

how to numeric sort on field time

i want to sort time field given by who command as a whole i have tried like this who|sort -n +4 -5

Login or Register to Ask a Question