How do i sort lines lexigraphical in bash?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How do i sort lines lexigraphical in bash?
# 8  
Old 12-19-2016
Hi.

Here is a comparison of 100,000 of your data lines shuffled, then sorted with msort and sort:
Code:
#!/usr/bin/env bash

# @(#) s2       Demonstrate time comparison, msort vs. sort.

LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C specimen msort sort

FILE=${1-sort-timing-input.txt}

p=$( basename $0 ) t1="$Revision: 1.14 $" v=${t1//[!0-9.]/}
[[ $# -gt 0 ]] && [[ "$1" =~ -version ]] &&  { echo "$p (local) $v" ; exit 0 ; }

pl " Sample of test input:"
specimen $FILE

pl " Time for msort:"
rm -f zz1
time msort -a S -j -q -l -n 1,1 $FILE > zz1

pl " Time for system sort:"
rm -f zz2
time sort -k 1,1 $FILE > zz2

pl " Compare results:"
# echo x >> zz2 # to test cmp.
if cmp zz1 zz2
then
  pe " Sort outputs compare exactly."
  v1=$( pe "0.612 / 0.056" | bc )       # using a sample if times
  pe " Command msort is slower by factor of $v1 in user time, algorithm shellsort."
else
  pe " Sort outputs do NOT compare -- FAIL!"
fi
rm -f zz1 zz2

exit 0

producing:
Code:
$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.6 (jessie) 
bash GNU bash 4.3.30
specimen (local) 1.17
msort 8.53
sort (GNU coreutils) 8.23

-----
 Sample of test input:
Edges: 5:0:5 of 100000 lines in file "sort-timing-input.txt"
fkdo-b-cen7 fkdo
fkdo-b-cen7 fkdo
flrp-b-an21 flrp
flrp-b-cen1 flrp
flrp-b-an23 flrp
   ---
flrp-b-an22 flrp
fkdo-b-cen8 fkdo
fkdo-b-cen6 fkdo
flrp-b-an24 flrp
flrp-b-an21 flrp

-----
 Time for msort:

real    0m1.834s
user    0m0.688s
sys     0m1.140s

-----
 Time for system sort:

real    0m0.059s
user    0m0.048s
sys     0m0.008s

-----
 Compare results:
 Sort outputs compare exactly.
 Command msort is slower by factor of 10 in user time, algorithm shellsort.

On one hand, an order of magnitude sounds really big. On the other hand, we are comparing about 1/10 second to 2 seconds over 100K lines. That's good enough for me in most cases, compared to the time one would put in to fiddle around with modifying the input file to satisfy sort.

Best wishes ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to sort file with certain criteria (bash)?

I am running a command that is part of a script and this is what I am getting when it is sorted by the command: command: ls /tmp/test/*NDMP*.z /tmp/test/CARS-GOLD-NET_CHROMJOB-01-XZ-ARCHIVE-NDMP.z /tmp/test/CARS-GOLD-NET_CHROMJOB-01-XZ-NDMP.z... (2 Replies)
Discussion started by: newbie2010
2 Replies

2. UNIX for Dummies Questions & Answers

awk - (URGENT!) Print lines sort and move lines if match found

URGENT HELP IS NEEDED!! I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but - Currently, script is not printing lines to File3.txt in order. - Also the matching lines are not moving out of File1.txt ... (1 Reply)
Discussion started by: High-T
1 Replies

3. Shell Programming and Scripting

How to sort lines according words?

Hello I greped some lines from an xml file and generated a new file. but some entries are missing my table is unsorted. e.g. NAME="Adel" ADDRESS="Donaustr." NUMBER="2" POSTCODE="33333" NAME="Adel" ADDRESS="Donaustr." NUMBER="2" POSTCODE="33333" NAME="Adel" NUMBER="2" POSTCODE="33333"... (5 Replies)
Discussion started by: witchblade
5 Replies

4. UNIX for Dummies Questions & Answers

Bash script to sort files

I've got a disorganized list of items and quantities for each. I've been using a combination of grep and sort to find out how much to buy of each item. I'm tired of having to constantly using these commands so I've been trying to write a shell script to make it easier, but I can't figure out how... (3 Replies)
Discussion started by: PTcharger
3 Replies

5. Shell Programming and Scripting

Bash - remove duplicates without sort

I need to use bash to remove duplicates without using sort first. I can not use: cat file | sort | uniq But when I use only cat file | uniq some duplicates are not removed. (4 Replies)
Discussion started by: locoroco
4 Replies

6. Shell Programming and Scripting

grep from 3 lines and sort

Pseudo name=hdiskpower54 Symmetrix ID=000190101757 Logical device ID=0601 state=alive; policy=SymmOpt; priority=0; queued-IOs=0 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW... (7 Replies)
Discussion started by: Daniel Gate
7 Replies

7. Shell Programming and Scripting

BASH: Sort four lines based on first line

I am in the process of sorting an AutoHotkey script's contents so as to make it easier for me to find and view its nearly 200 buzzwords (when I forget which one corresponds with what phrase, which I do now and then). About half to two-thirds of the script's key phrases correspond to locations... (7 Replies)
Discussion started by: SilversleevesX
7 Replies

8. Shell Programming and Scripting

Need Help to sort text lines

I need to sort input file as below to display as below: input.txt User: my_id File: oracle/scripts/ssc/ssc_db_info User: your_id File: pkg_files/BWSwsrms/request User: your_id File: pkg_files/BWSwsco/checkConfig.sh OUTPUT: User: my_id File: ... (3 Replies)
Discussion started by: tqlam
3 Replies

9. Shell Programming and Scripting

How to sort decimal values in bash

Hi, I have a list of values from associative array from 0,..till 1.0000. I tried various sort options; sort -g, sort -nr but it still couldnt work. In other words, the numbers are not sorted accordingly. Please help. Thanks. (1 Reply)
Discussion started by: ahjiefreak
1 Replies

10. Shell Programming and Scripting

Sort (bash command)

I did a search on this, and found lots on SORT but no answer to my question. I have a C program that fetches all of our users from Netware, and I have that it makes a file that I later include in a html as a select tag drop-down menu. Here is what 1 line looks like: <option... (5 Replies)
Discussion started by: booboo
5 Replies
Login or Register to Ask a Question