Sponsored Content
Top Forums Shell Programming and Scripting Strange variable comparison result in awk Post 302432263 by treesloth on Thursday 24th of June 2010 12:36:17 PM
Old 06-24-2010
Perhaps some context will help. This is incomplete, of course. I haven't yet added outputs for all results-- I'm just working on the "lowest" bin and rudimentary output for the rest. Once that works, I'll finish it, which should be easy enough.

So, what I do is take a number and increment a particular "bin"-- an array element-- depending on the value of that number. That's handled by this line:

Code:
bin[ ( i * int(($0 - 1) / i) + i)]++

Now, sometimes it's useful to be able to lump together everything below a certain value. Since the index of the array is simply the number describing the histogram values contained in the bin, I can just tell the script, "take all array elements whose index is less than the user-specified start value and dump them all into a single bin". I do that by the simple numeric test I described above. The problem, though, is that those values are not being properly tested. Why in the world would this script believe that 10000 is less than 2000? As special troubleshooting output, I have the script tell me which "bins" it's putting in the "everything less than user threshold" group. It puts these lines at the top, followed by the "normal" output. Here's a sample of numbers such as those that it will be used to test, sorted for reader convenience:

Code:
139
384
515
552
556
2128
2183
2234
2277
2284
2331
2385
2879
3281
3327
5035
5294
9401
9948
11458
13363

As you can see, there are 5 values below 2000-- yet, for some reason, the test says that there are 3, and the 3 it gives are over 10000. It almost seems to be applying some sort of alphabetic test, not numeric.

To test, save the values above as, say, "testvals". Save the script below as "hist.awk" and run:

./hist.awk -v i="125" -v s="2000" testvals

Code:
#! /usr/bin/awk -f

        {
        bin[ ( i * int(($0 - 1) / i) + i)]++
        }

END     {
        for (bindex in bin)
                {
                if ( bindex < s )
                        {
                        print bindex "\t" bin[bindex] "\t" s
                        hist[less] = hist[less] + bin[bindex]
                        }
                else
                        {
                        hist[bindex] = bin[bindex]
                        }
                }
                print "less\t" hist[less]
        }

Any thoughts? I'm thoroughly confused.

Last edited by treesloth; 06-24-2010 at 04:17 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Strange Number comparison issue

Hi, I am comparing two numbers, but it gives strange results: My Code: if then echo "True" else echo "False" fi This code gives False for the follwoing comparison where as True for the following: Any reason for this? Both Should have given False... I am using... (9 Replies)
Discussion started by: shihabvk
9 Replies

2. Shell Programming and Scripting

assign awk command result to a variable

#!/bin/sh # ## MYSTRING = `awk '/myApp.app/' /Users/$USER/Library/Preferences/loginwindow.plist` if then echo String not found defaults write /Users/$USER/Library/Preferences/loginwindow AutoLaunchedApplicationDictionary -dict-add -string Hide -bool YES -string Path -string... (9 Replies)
Discussion started by: dedmakar
9 Replies

3. Shell Programming and Scripting

awk printing: strange result

Dear all, I am using awk in a bash script to extract a list of x y z coordinates from a file such as: %BEGIN 3D-SPACE COORDINATES 0.2085627338147950 0.2471306816410478 0.2085627338147950 0.1242549179185660 0.2755539793525220 0.4147884486606120 0.2030669560265720 ... (6 Replies)
Discussion started by: pauli
6 Replies

4. Shell Programming and Scripting

awk comparison with variable

hi, I want to compare i variable in the awk statement but its not working out. Pl help me out If we do the comparison like this its OK, cat sample | awk -F" ", '{if ($1=="1-Sep-2009") print $1,$2,$3,$4,$5}' But if u use a variable instead of "1-Sept-2009", it does not return anything,... (2 Replies)
Discussion started by: asadlone
2 Replies

5. Shell Programming and Scripting

Awk multiple variable array: comparison

Foo.txt 20 40 57 50 22 51 66 26 17 15 63 18 80 46 78 99 87 2 14 14 51 47 49 100 58 Bar.txt 20 22 51 15 63 78 99 55 51 58 How to get output using awk 20 22 57 50 51 15 26 17 63 78 80 46 99 55 - - 51 58 49 100 (5 Replies)
Discussion started by: genehunter
5 Replies

6. Shell Programming and Scripting

Strange result of eval, how does eval really work with ssh?

Hi all, some small script with eval turned me to crazy. my OS is linux Linux s10-1310 2.6.16.53-0.8.PTF.434477.3.TDC.0-smp #1 SMP Fri Aug 31 06:07:27 PDT 2007 x86_64 x86_64 x86_64 GNU/Linux below script works well #!/bin/bash eval ssh remotehost date eval ssh remotehost ls below... (1 Reply)
Discussion started by: summer_cherry
1 Replies

7. Shell Programming and Scripting

Strange result

Hi, I have following codes which looks ok: $ string1="123456789 abc2" $ string2="abc" $ position_of_string2=`expr index "$string1" "$string2"` $ echo $position_of_string2 $ 11however, when string2="abc2", it gives me the following result: $ string1="123456789 abc2" $... (5 Replies)
Discussion started by: littlewenwen
5 Replies

8. Shell Programming and Scripting

Assign awk gsub result to a variable

Hello, I have searched but failed to find what exactly im looking for, I need to eliminate first "." in a output so i can use something like the following echo "./abc/20141127" | nawk '{gsub("^.","");print}' what i want is to use gsub result later on, how could i achieve it? Let say... (4 Replies)
Discussion started by: EAGL€
4 Replies

9. UNIX for Dummies Questions & Answers

Strange result using find command.

I created a file with the permissions of 776. When I ran the command find /root/Desktop -perm -644 -type f The created file shows up as part of the results. Doesn't -perm -mode mean that for global, only 4(read) and 2(write) can be accepted ? (2 Replies)
Discussion started by: Hijanoqu
2 Replies

10. Shell Programming and Scripting

awk if comparison with variable

Hi , Need help, p for value in `awk -F, '{print $1 }' ad | uniq` do x=$(echo $value) echo $x echo `awk -F, '{if( $1 == $x) sum = sum + $8 } END{print sum}' ad` --- not working echo `awk -F, '{if($1 == “MT”) sum = sum + $8 } END{print sum}' ad` -- Working but hard coded done; ad... (4 Replies)
Discussion started by: nadeemrafikhan
4 Replies
funhist(1)							SAORD Documentation							funhist(1)

NAME
funhist - create a 1D histogram of a column (from a FITS binary table or raw event file) or an image SYNOPSIS
funhist [-n|-w|-T] <iname> [column] [[lo:hi:]bins] OPTIONS
-n # normalize bin value by the width of each bin -w # specify bin width instead of number of bins in arg3 -T # output in rdb/starbase format (tab separators) DESCRIPTION
funhist creates a one-dimensional histogram from the specified columns of a FITS Extension binary table of a FITS file (or from a non-FITS raw event file), or from a FITS image or array, and writes that histogram as an ASCII table. Alternatively, the program can perform a 1D projection of one of the image axes. The first argument to the program is required, and specifies the Funtools file: FITS table or image, raw event file, or array. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, and filters. For a table, the second argument also is required. It specifies the column to use in generating the histogram. If the data file is of type image (or array), the column is optional: if "x" (or "X"), "y" (or "Y") is specified, then a projection is performed over the x (dim1) or y (dim2) axes, respectively. (That is, this projection will give the same results as a histogram performed on a table containing the equiva- lent x,y event rows.) If no column name is specified or "xy" (or "XY") is specified for the image, then a histogram is performed on the values contained in the image pixels. The argument that follows is optional and specifies the number of bins to use in creating the histogram and, if desired, the range of bin values. For image and table histograms, the range should specify the min and max data values. For image histograms on the x and y axes, the range should specify the min and max image bin values. If this argument is omitted, the number of output bins for a table is calcu- lated either from the TLMIN/TLMAX headers values (if these exist in the table FITS header for the specified column) or by going through the data to calculate the min and max value. For an image, the number of output bins is calculated either from the DATAMIN/DATAMAX header val- ues, or by going through the data to calculate min and max value. (Note that this latter calculation might fail if the image cannot be fit in memory.) If the data are floating point (table or image) and the number of bins is not specified, an arbitrary default of 128 is used. For binary table processing, the -w (bin width) switch can be used to specify the width of each bin rather than the number of bins. Thus: funhist test.ev pha 1:100:5 means that 5 bins of width 20 are used in the histogram, while: funhist -w test.ev pha 1:100:5 means that 20 bins of width 5 are used in the histogram. The data are divvied up into the specified number of bins and the resulting 1D histogram (or projection) is output in ASCII table format. For a table, the output displays the low_edge (inclusive) and hi_edge (exclusive) values for the data. For example, a 15-row table con- taining a "pha" column whose values range from -7.5 to 7.5 can be processed thus: [sh] funhist test.ev pha # data file: /home/eric/data/test.ev # column: pha # min,max,bins: -7.5 7.5 15 bin value lo_edge hi_edge ------ --------- --------------------- --------------------- 1 22 -7.50000000 -6.50000000 2 21 -6.50000000 -5.50000000 3 20 -5.50000000 -4.50000000 4 19 -4.50000000 -3.50000000 5 18 -3.50000000 -2.50000000 6 17 -2.50000000 -1.50000000 7 16 -1.50000000 -0.50000000 8 30 -0.50000000 0.50000000 9 16 0.50000000 1.50000000 10 17 1.50000000 2.50000000 11 18 2.50000000 3.50000000 12 19 3.50000000 4.50000000 13 20 4.50000000 5.50000000 14 21 5.50000000 6.50000000 15 22 6.50000000 7.50000000 [sh] funhist test.ev pha 1:6 # data file: /home/eric/data/test.ev # column: pha # min,max,bins: 0.5 6.5 6 bin value lo_edge hi_edge ------ --------- --------------------- --------------------- 1 16 0.50000000 1.50000000 2 17 1.50000000 2.50000000 3 18 2.50000000 3.50000000 4 19 3.50000000 4.50000000 5 20 4.50000000 5.50000000 6 21 5.50000000 6.50000000 [sh] funhist test.ev pha 1:6:3 # data file: /home/eric/data/test.ev # column: pha # min,max,bins: 0.5 6.5 3 bin value lo_edge hi_edge ------ --------- --------------------- --------------------- 1 33 0.50000000 2.50000000 2 37 2.50000000 4.50000000 3 41 4.50000000 6.50000000 For a table histogram, the -n(normalize) switch can be used to normalize the bin value by the width of the bin (i.e., hi_edge-lo_edge): [sh] funhist -n test.ev pha 1:6:3 # data file: test.ev # column: pha # min,max,bins: 0.5 6.5 3 # width normalization (val/(hi_edge-lo_edge)) is applied bin value lo_edge hi_edge ------ --------------------- --------------------- --------------------- 1 16.50000000 0.50000000 2.50000000 2 6.16666667 2.50000000 4.50000000 3 4.10000000 4.50000000 6.50000000 This could used, for example, to produce a light curve with values having units of counts/second instead of counts. For an image histogram, the output displays the low and high image values (both inclusive) used to generate the histogram. For example, in the following example, 184 pixels had a value of 1, 31 had a value of 2, while only 2 had a value of 3,4,5,6, or 7: [sh] funhist test.fits # data file: /home/eric/data/test.fits # min,max,bins: 1 7 7 bin value lo_val hi_val ------ --------------------- --------------------- --------------------- 1 184.00000000 1.00000000 1.00000000 2 31.00000000 2.00000000 2.00000000 3 2.00000000 3.00000000 3.00000000 4 2.00000000 4.00000000 4.00000000 5 2.00000000 5.00000000 5.00000000 6 2.00000000 6.00000000 6.00000000 7 2.00000000 7.00000000 7.00000000 For the axis projection of an image, the output displays the low and high image bins (both inclusive) used to generate the projection. For example, in the following example, 21 counts had their X bin value of 2, etc.: [sh] funhist test.fits x 2:7 # data file: /home/eric/data/test.fits # column: X # min,max,bins: 2 7 6 bin value lo_bin hi_bin ------ --------------------- --------------------- --------------------- 1 21.00000000 2.00000000 2.00000000 2 20.00000000 3.00000000 3.00000000 3 19.00000000 4.00000000 4.00000000 4 18.00000000 5.00000000 5.00000000 5 17.00000000 6.00000000 6.00000000 6 16.00000000 7.00000000 7.00000000 [sh] funhist test.fits x 2:7:2 # data file: /home/eric/data/test.fits # column: X # min,max,bins: 2 7 2 bin value lo_bin hi_bin ------ --------------------- --------------------- --------------------- 1 60.00000000 2.00000000 4.00000000 2 51.00000000 5.00000000 7.00000000 You can use gnuplot or other plotting programs to graph the results, using a script such as: #!/bin/sh sed -e '1,/---- .*/d /^$/,$d' | awk ' BEGIN{print "set nokey; set title "funhist"; set xlabel "bin"; set ylabel "counts"; plot "-" with boxes"} {print $3, $2, $4-$3}' | gnuplot -persist - 1>/dev/null 2>&1 Similar plot commands are supplied in the script funhist.plot: funhist test.ev pha ... | funhist.plot gnuplot SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funhist(1)
All times are GMT -4. The time now is 10:22 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy