They are pretty small numbers so you're right, double precision won't be enough. I didn't put the actual input on here becuase it is quite wide and I wasn't sure it would fit, but here are the first 29 lines of the input (some are in range some not):
will basically write $5^2 + $5^2.... and so on, where each $5 is from a different line that meets the criteria. Awk is not actually doing the arithmetic, just passing one big equation to bc. So, if I wanted precision out to the 20th decimal place, would I make the following change?
Code:
{print "scale=20;x += "$5" ^ "2}
---------- Post updated at 10:41 PM ---------- Previous update was at 10:33 PM ----------
I'm a bit confused. Is this doing the arithmetic in awk? Something isn't working out, it is giving a negative number.
Last edited by butson; 12-03-2013 at 10:34 PM..
Reason: code tag didn't work because I removed a bracket
To get the precision you want it is a little more complicated than that; we still need to split the mantissa and exponent out of $5 and have bc perform arithmetic that you were doing in your script. For example: If we expand on MadeInGermany's awk script adding some error checking and using bc to get the desired precision we get something like:
Code:
#!/bin/ksh
IAm=${0##*/}
lf="$IAm.$$"
xmin=-18
ymin=-2
zmin=-6
xmax=-8
ymax=8
zmax=4
totden=$(awk -v xmin="$xmin" -v xmax="$xmax" -v ymin="$ymin" -v ymax="$ymax" \
-v zmin="$zmin" -v zmax="$zmax" -v lf="$lf" '
BEGIN { printf("scale=20\nx=0\n") # Initialize bc
}
$5!="" && $2<=xmax && $2>=xmin && $3<=ymax && $3>=ymin && $4<=zmax && $4>=zmin {
if(match($5, /E([+])?/))
printf("x+=(%s*10^%s)^2\n", substr($5, 1, RSTART - 1),
substr($5, RSTART + RLENGTH))
else printf("NR=%d: $5 bad format: %s\n", NR, $5) > lf
}
END { printf("x\nquit\n")
}' "${1:-density.mesh_index}" | bc)
es=$?
if [ -s "$lf" ]
then printf "%s: Log file (%s) is not empty\n" "$IAm" "$lf" >&2
cat "$lf" >&2
if [ "$es" -eq 0 ]
then es=1
fi
fi
printf "totden: %s\n" "$totden"
exit $es
and we expand density.mesh_index to contain several duplicates of lines you showed us in message #8 in this thread to expand it to 800 lines (and add 40 empty lines for fun) and modify a few of the exponents on the numbers in field 5 to use E+004 and E+005 instead of E-004 and E-005 (to test out the logic of converting values with positive and negative exponents) it produces the output:
Code:
totden: 29418651008124.442882561190769208458247
I also tried a slightly modified version of your script (to get rid of the diagnostics from bc induced by processing empty lines):
Code:
#!/bin/bash
xmin=-18
ymin=-2
zmin=-6
xmax=-8
ymax=8
zmax=4
totden=0
o=$2
s=$3
for (( e = $o; e <= $s; e++ ))
do
x=$( sed -n ${e}p $1 | awk '{print $2}' | sed 's/E/\*10\^/' | sed 's/\+//' )
if [ "$x" == "" ]
then continue
fi
xbool=$( echo "$x <= $xmax && $x >= $xmin" | bc )
if [ "$xbool" == 1 ]
then
y=$( sed -n ${e}p $1 | awk '{print $3}' | sed 's/E/\*10\^/' | sed 's/\+//' )
ybool=$( echo "$y <= $ymax && $y >= $ymin" | bc )
if [ "$ybool" == 1 ]
then
z=$( sed -n ${e}p $1 | awk '{print $4}' | sed 's/E/\*10\^/' | sed 's/\+//' )
zbool=$( echo "$z <= $zmax && $z >= $zmin" | bc )
if [ "$zbool" == 1 ]
then
psi=$( sed -n ${e}p $1 | awk '{print $5}' | sed 's/E/\*10\^/' | sed 's/\+//' )
dens=$( echo "scale=20; ($psi)^2" | bc )
totden=$( echo "scale=20;$totden + $dens" | bc )
fi
fi
fi
done
echo $totden
which when invoked with the operands density.mesh_index 1 840 to process all of the lines in the same file produces the output:
Code:
29418651008124.442882561190769208458247
Running the modified copy of your script with bash as the interpreter 10 times took from 22.94 to 23.76 seconds of wall clock time each. Changing the modified copy of your script to use ksh instead of bash produced the same results with execution times from 22.25 to 23.27 seconds each.
Using ksh to process the same data using the script I based on MadeInGermany's code 10 times took from .02 to .03 seconds each. And modifying that script to use bash instead of ksh took .02 to .04 seconds each. All of these tests were run on a 4 year old MacBook Pro running OS X version 10.7.5.
Hopefully, this streamlined script will cut your processing time for your 160 files from the 1.13 years you estimated to a something closer to 1 day.
In case you're having trouble following what the awk script is doing to $5 as it passes work to bc, an abbreviated version of what awk writes through the pipe to bc when given the input you showed us in message #8 in this thread follows:
Thank you guys so much for all the help. I think I will spend my winter break learning how to use awk properly. Maybe I will try to learn FORTRAN as well. I know it would help with this kind of thing, but it seems way more complicated than shell scripting. I think it will help me in my future endeavors though. Again, thanks guys!
Hi
I am getting an argument which specifies the range of numbers. eg: 7-15
Is there a way that i can easily (avoiding loop) print the range of number between and including the specified above.
The above example should translate to 7,8,9,10,11,12,13,14,15 (3 Replies)
Dear Friends,
I want to know how to grep for the lines that has a number between given range(start and end).
I have tried the following sed command.
sed -n -e '/20030101011442/,/20030101035519/p'
However this requires both start and end to be part of the content being grepped. However... (4 Replies)
Hello,
I have a column where there are values from 1 to 150.
I want to get the frequency of values in the following ranges:
1-5
6-10
11-15
....
....
....
146-150
How can I do this in a for loop?
Thanks,
Guss (1 Reply)
Howdy experts,
We have some ranges of number which belongs to particual group as below.
GroupNo StartRange EndRange
Group0125 935300 935399
Group2006 935400 935476
937430 937459
Group0324 935477 935549
... (6 Replies)
Hi im new to unix and need to find a way to grep the top 5 numbers in a file and put them into another file. For example my file looks like this
abcdef 50000
abcdef 45000
abcdef 40000
abcdef 35000
abcdef 30000
abcdef 25000
abcdef 20000
abcdef 15000
abcdef 10000
and so on...
How can... (1 Reply)
Ok, so this is a small part of a script I wrote to build disk groups using VXVM. The only problem is that I am limited to a count of 8 maximum. If I want more, I will have to add more lines of "if" statements. How can I accomplish the same thing, in a few lines, but not be limited in the max... (13 Replies)
Suppose u have a file like
1 30
ABCSAAHSNJQJALBALMKAANKAMLAMALK
4562676268836826826868268468368282972982
2863923792102370179372012792701739729291
31 60... (8 Replies)
I want to grep a range of numbers in a log file. My log file looks like this:
20050807070609Z;blah blah
That is a combination of yr,month,date,hours,minutes,seconds.
I want to search in the log file events that happened between a particular time.
like between 20050807070000 to 20050822070000... (1 Reply)
12-04-2013
