awk - Why does output order change?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk - Why does output order change?
# 1  
Old 01-09-2014
awk - Why does output order change?

I have this END section in an awk script:


Code:
END     {
                for (j in littlebin)
                        {
                        print (j, int(log(j)/log(2)) , littlebin[j])
                        lbsum+=littlebin[j]
                        }

                for (i in bins)
                        {
                        print (i / (1024^2) "MB" , bins[i])
                        }
        }

As you can see I have 2 for loops, each outputting the contents of an array, with a little extra calculation. So, it prints in this order:

(1) Contents of the littlebin array, then
(2) Contents of the bins array

Ok, good. But, I'd like the contents of each of those arrays to be sorted numerically on the first element printed. So, I add a sort pipe:

Code:
END     {
                for (j in littlebin)
                        print (j, int(log(j)/log(2)), littlebin[j]) | "sort -nk 1"
                        lbsum+=littlebin[j]

                for (i in bins)
                        print (i / (1024^2) "MB" , bins[i]) | "sort -nk 1"
        }

This causes the output to be somewhat reversed, to become:

(1) Contents of the bins array, numerically sorted, then
(2) Contents of the littlebin array, numerically sorted

Why is that happening? And what is the best way of sorting as I'm attempting, on the index of each array? I've tried asorti, but it doesn't sort numerically. Thank you.
# 2  
Old 01-09-2014
The order in which a for ( idx in array ) loop scans an array is not defined; it is generally based upon the internal implementation of arrays inside awk.

By the way you can use asort() / asorti() functions to sort array values / indices.

Post the code fragment that you used to sort numerically.
# 3  
Old 01-09-2014
Quote:
Originally Posted by Yoda
The order in which a for ( idx in array ) loop scans an array is not defined; it is generally based upon the internal implementation of arrays inside awk.

By the way you can use asort() / asorti() functions to sort array values / indices.

Post the code fragment that you used to sort numerically.
I attempted to use asorti, but I was unable to get it to sort numerically.

The code that I used is above. I simply appended " | sort -nk 1" to each "print" line. Here's the full script:

Code:
BEGIN   {
        binwidth = (512 * 1024)
        }

$1 >= 1387515600 && $3 == "GET" {
        mybin = (binwidth * int($2 / binwidth))
        bins[mybin]++
        }

$1 >= 1387515600 && mybin < binwidth && $3 == "GET"     {
        littlebin[2^(int(log($2)/log(2)))]++
        }

END     {
                for (j in littlebin)
                        print (j, int(log(j)/log(2)), littlebin[j]) | "sort -nk 1"

                for (i in bins)
                        print (i / (1024^2) "MB" , bins[i]) | "sort -nk 1"
        }

These are a couple of lines from the file it's parsing:

Code:
1387499749,6324891,POST,1387497600
1387501190,1120178,GET,1387497600

Thanks for the reply.
# 4  
Old 01-09-2014
Quote:
Originally Posted by treesloth
I attempted to use asorti, but I was unable to get it to sort numerically.
I don't see asorti function used in the code that you posted.

All you have to do is call asorti function and copy the sorted result from source array to destination array. Finally scan the destination array to print sorted result.

By the way the sort -nk 1 command usage in your code is logically wrong and is not going to help.
# 5  
Old 01-09-2014
Quote:
Originally Posted by Yoda
I don't see asorti function used in the code that you posted.

All you have to do is call asorti function and copy the sorted result from source array to destination array. Finally scan the destination array to print sorted result.

By the way the sort -nk 1 command usage in your code is logically wrong and is not going to help.
That's correct; asorti is not in my code presently. When I attempted to use it, it sorted my indices alphabetically, not numerically. It was done like this:

Code:
n = asorti(littlebin, lbidx, "ascending")
      for (j = 1 ; j <= n ; j++)
      print lbidx[j], littlebin[lbidx[j]]

The output was:

Code:
1024 142
131072 810
16384 249
2048 82
262144 720
32768 677
4096 68
512 181
65536 1128
8192 181

As you can see, it is not sorted numerically.
# 6  
Old 01-09-2014
Here is an example sorting by value:
Code:
awk  '
        BEGIN {
                n = split ("1024 131072 16384 2048 262144 32768 4096 512 65536 8192", A)
        }
        END {
                print "Before sorting array"
                for ( i = 1; i <= n; i++ )
                        print A[i]

                n = asort ( A, B )

                print "After sorting array"
                for ( i = 1; i <= n; i++ )
                        print B[i]

        }
' /dev/null

Producing output:
Code:
Before sorting array
1024
131072
16384
2048
262144
32768
4096
512
65536
8192
After sorting array
512
1024
2048
4096
8192
16384
32768
65536
131072
262144

# 7  
Old 01-09-2014
An alternative approach that should work (although I haven't tested it) would be to change the following lines in your original code:
Code:
END     {
                for (j in littlebin)
                        print (j, int(log(j)/log(2)), littlebin[j]) | "sort -nk 1"

                for (i in bins)
                        print (i / (1024^2) "MB" , bins[i]) | "sort -nk 1"
        }

to:
Code:
END     {
                for (j in littlebin)
                        print (j, int(log(j)/log(2)), littlebin[j]) | "sort -n"
                close("sort -n")

                for (i in bins)
                        print (i / (1024^2) "MB" , bins[i]) | "sort -n"
        }

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to change row by order nr?

Hello, I have a file with thousands of rows and I need to change sequence of lines. Sample file: #NAME #SERVICE 112233 #DESCRIPTION AABBCCDD #SERVICE 738292 #DESCRIPTION FFYYRRTT ... ... ... Desired output: #NAME (5 Replies)
Discussion started by: baris35
5 Replies

2. Shell Programming and Scripting

Change of fields order inline

Hi everyone, What is the best solution to check every line in the xml file and change order of found field along with its value without touching value. Pattern will be given i.e. one line can look like this one: <widget position="value,value" size="value,value" name="value"... (5 Replies)
Discussion started by: TiedCone
5 Replies

3. UNIX for Dummies Questions & Answers

change service order

hi guys I have a service that depends on some shares (NFS shares ) that need to be mounted before before the service start so the service-app finds the NFS shares and starts correctly... I am confused here this is what I found but I am not sure what to do in order to change it BTW is Suse... (2 Replies)
Discussion started by: karlochacon
2 Replies

4. Shell Programming and Scripting

change order

I have inside a file 22 25 80 111 631 694 861 875 I need this in the form using awk or sed 22,25,80,111,631,694,861,875 (4 Replies)
Discussion started by: anil510
4 Replies

5. Shell Programming and Scripting

using awk for setting variable but change the output of this variable within awk

Hi all, Hope someone can help me out here. I have this BASH script (see below) My problem lies with the variable path. The output of the command find will give me several fields. The 9th field is the path. I want to captured that and the I want to filter this to a specific level. The... (6 Replies)
Discussion started by: Cowardly
6 Replies

6. Shell Programming and Scripting

Change order

Good evening I have a file as below and want to change the order, as in the second column, sed awk Pearl Thanks aaaaaaaaaa bbbbbbbbb cccccccc aaaaaaaaaa bbbbbbbbb cccccccc aaaaaaaaaa cccccccc bbbbbbbbb aaaaaaaaaa cccccccc bbbbbbbbb (8 Replies)
Discussion started by: Novice-
8 Replies

7. Shell Programming and Scripting

How can I change is output format by awk ?

Hello, Can you tell me how can I change this format by awk Input 0.2057422D-01 0.2463722D-01 -0.1068047D-02 Output 0.02057422 0.02463722 -0.001068047 Thanks wan (8 Replies)
Discussion started by: wanchem
8 Replies

8. Shell Programming and Scripting

Change many columns position/order

Hi everyone, Please some help over here. (I´m using cygwing) I have files with 40 columns and 2000 lines in average. I´m trying to change the order position as follow. Original columns position:... (3 Replies)
Discussion started by: cgkmal
3 Replies

9. AIX

How to change the order of authentication in aix?

I have an AIX system configured as NIS client. I have an local user on the system called "batman" and i have a user by the same name in NIS as well. Now when i try to login with Batman user, the local batman gets in. How do I tell the AIX machine to authenticate Batman as NIS user? ... (2 Replies)
Discussion started by: balaji_prk
2 Replies

10. UNIX for Dummies Questions & Answers

How to change the order of a string ?

Hi , I want to change the order of a string using sed command . Is it possible ? $echo "abc123xyz" | sed 's/\()*\) \(*\)/\2\1/' abc123xyz $ echo "abc123xyz" |sed 's/\()*\) \(*\) \()*\)/\2\1\3/' abc123xyz I want to change the string , abc123xyz as xyz123abc . Is it... (5 Replies)
Discussion started by: rajavu
5 Replies
Login or Register to Ask a Question