sort command


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers sort command
# 1  
Old 02-14-2005
sort command

Hi,

I am going to sort a huge flat file using sort command, this file is about 36 million lines, 179 fields delimitered by Ctrl B (002). eg.

1^B198709.....
17^B200301....
3^B196511....
.....

I want this file being sorted by the first field, the result is like :

1^B198709.....
3^B196511....
17^B200301....

I used to sort it by command like :

$sort -n -T /test filename

It worked. but I am trying to find a more effective way to sort it. I assume only sort the first field would be much quicker, then I used command like :

$sort -n -t ^B -k 1,1 -T /test filename

it worked as well, but the performance was the same.

Can anyone give me an advice ?

Thanks in advance

xli
# 2  
Old 02-15-2005
-s ?

Without having a file that huge to messa round with, it's hard for me to say
(maybe I'll generate one after this and try).

The -S option seems like a possibility, for an increased memory buffer.
I would imagine you've been down that road, but I'm just throwing it out there.
# 3  
Old 02-15-2005
It does help abit

-S did seem to help in my test
scenario.

I had a file that took a little over a minute, and I added
-S 1024

And it took about 50 seconds. That is certainly not a scientific
test or anything, nor Earth shattering improvement, but it at least
seems worth exploring.

Another possibility might be using awk, but it's questionable
whether or not that could help speed things up.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sort Command

Hi All, I have used sort -k1 -n data.txt > output.txt command on a large text data file with over 1,000,000 rows. The command managed to sort the data but the code did not read data according to sequence of occurrence. Given below are the first five lines of the data I need to sort; 1 1... (2 Replies)
Discussion started by: Theo Score
2 Replies

2. Shell Programming and Scripting

Help to sort out... Possible use of sort command

I have an input like 4.3.6.66 4.3.6.67 4.3.6.70 4.3.6.25 4.3.6.15 4.3.6.54 4.3.6.44 4.3.6.34 4.3.6.24 4.3.6.14 4.3.6.53 4.3.6.43 4.3.6.49 4.3.6.33 4.3.6.52 4.3.6.19 4.3.6.58 4.3.6.42 (5 Replies)
Discussion started by: dnam9917
5 Replies

3. Shell Programming and Scripting

Help with sort command

I have a file with the following content:- 181268525,0640613864,B,113,22-dec-2011 14:12:08, 181268525,0640613864,C,113,25-dec-2011 14:18:50, 181268525,0640613864,L,113,26-dec-2011 14:07:46, 181268525,0640613864,X,113,01-jan-2012 16:57:45, 181268525,0640613864,X,113,04-jan-2012 14:13:27,... (3 Replies)
Discussion started by: Yoda
3 Replies

4. Shell Programming and Scripting

Is it Possible to sort a list of hexadecimal numbers using "sort" command?

Hello Everybody :) !!!. i have question in mind, is it possible to sort a list of hexadecimal numbers using "sort" command? (9 Replies)
Discussion started by: Kesavan
9 Replies

5. Homework & Coursework Questions

Sort command

I have file ipaddress.txt 192.168.1.25 127.3.9.12 192.168.12.1 127.21.2.3 127.92.80.6 192.168.4.5 I want to sort as 127.3.9.12 127.21.2.3 127.92.80.6 192.168.1.25 192.168.12.1 192.168.4.5 So what sort command do I have to use. (1 Reply)
Discussion started by: RiderOnsky
1 Replies

6. Shell Programming and Scripting

How to Sort Floating Numbers Using the Sort Command?

Hi to all. I'm trying to sort this with the Unix command sort. user1:12345678:3.5:2.5:8:1:2:3 user2:12345679:4.5:3.5:8:1:3:2 user3:12345687:5.5:2.5:6:1:3:2 user4:12345670:5.5:2.5:5:3:2:1 user5:12345671:2.5:5.5:7:2:3:1 I need to get this: user3:12345687:5.5:2.5:6:1:3:2... (7 Replies)
Discussion started by: daniel.gbaena
7 Replies

7. Shell Programming and Scripting

Need help with sort command

Hi i have a file containing ip addresses and want to sort those IP addresses in the ascending order. file (match.txt) contents are: 192.168.0.100 192.168.0.16 192.168.0.10 192.168.0.23 192.168.0.2 192.168.0.3 192.168.0.1 192.168.0.222 i tried: sort -n match.txt output is :... (3 Replies)
Discussion started by: manmeet
3 Replies

8. UNIX for Dummies Questions & Answers

sort command...

Hi IŽd like to get a few explanations about how the sort command works when cascading the options. cscyabl@comet:(develop)> more file 2:A2 2:A1 5:A2 5:A2 10:A1 cscyabl@comet:(develop)> sort -n -u file 2:A1 5:A2 10:A1 cscyabl@comet:(develop)> sort -u -n file 2:A1 5:A2 10:A1... (8 Replies)
Discussion started by: Indalecio
8 Replies

9. UNIX for Dummies Questions & Answers

Help with the Sort command

Can someone please tell me how to sort a file, based on a particular position within the file? I have a line sequential file that is 152 bytes per record, in which i need to sort the file based on the numeric data in positions 142-152. I have done the "man sort" command and see the -k option... (1 Reply)
Discussion started by: rjjenkin
1 Replies
Login or Register to Ask a Question