Use decimal value of array in bc ends with illegal character
hi all
I have to read a long cvs file every 4 columns with decimal "3,45" numbers.
The 9th row in this cvs is the first line I need, so it I tail -n+9.
I use sed -e 's/,/./g' to get decimal values with . delimiter.
So far no problem.
Goal is to get two maximum negative forces in ranges 56-66 degrees and 33-55 degrees.
Just for better understanding:
one test case is a set of two direction (open and close)
open and close have both an value of angel in degrees and a value force in Newton.
Here is a table header of three test cases. The original csv file has thousands and around 340 rows of angel/force combinations.
Code:
;test no 3;;;;test no 2;;;;test no 1;;;;
;process open;;process close;process open;;process close;process open;;process close
;angel;force;angel;force;angel;force;angel;force;angel;force;angel;force
;68,09;8,88;0,4;0,6;68,27;8,46;0,21;0,7;68,27;8,8;0,21;0,64
;67,44;0,19;0,12;3,41;67,72;0,35;0,03;2,65;67,72;0,26;0,03;2,2
;66,62;1,5;0,05;0,73;67,08;0,37;0,24;0,05;66,89;0,28;0,24;0,38
;60,46;-4,39;4,65;0,92;60,46;-4,79;5,02;0,82;60,55;-4,65;5,2;0,92
.....
What I need to get working is the commented if-statement.
Code:
#!/bin/bash
csv=${1:-"/home/my/test.csv"}
val1_angel_high=66
val1_angel_low=56
val2_angel_high=55
val2_angel_low=33
no_cols=$(head -n 1 "${csv}" | awk -F";" '{ print NF}')
no_val_pairs=$((((no_cols--))/4))
while [ $no_cols -gt 1 ]; do
angelCollection_process_1=( $(tail -n+9 "${csv}" | cut -d ';' -f "$no_cols" | sed -e 's/,/./g') )
forceCollection_process_1=( $(tail -n+9 "${csv}" | cut -d ';' -f "$((no_cols+1))" | sed -e 's/,/./g') )
#printf "Process 1 - Angel: %s Force: %s\n" "${angelCollection_process_1[@]}" "${forceCollection_process_1[@]}"
position=0
max_force=0.0;
for t in "${angelCollection_process_1[@]}"; do
if (( $(bc <<< "$t <= $val1_angel_high") && $(bc <<< "$t >= $val1_angel_low") )); then
#echo $t : $max_force">"${forceCollection_process_1[$position]}; #works
force="${forceCollection_process_1[$position]}";
echo $max_force $force; #works
force2=$(bc <<< "$force*100");
echo $force2; # not working (standard_in) 1: illegal character: ^M
#if (( $(bc <<< "${forceCollection_process_1[$position]} < $max_force") )); then
# max_force="${forceCollection_process_1[$position]}";
#fi
fi
((position++))
done
echo $max_force
no_cols=$((no_cols-4))
done
Hope you find a solution how I can use the value from array in bc.
Other solutions are also welcome. I try to use perl also, but I failed with reading csv every 4 columns.
^M is the visible, readable representation of the <CR> (carriage return, 0X0D, \r) character and as such not recognized by sed. Your sed might accept the \r representation?
I am assuming that you are really dealing with angles (the slope between two lines) and forces (instead of angels (benevelant attendant spirits) and forces) and have changed variable names to match; but I have not changed the typos in your sample input file. I also assume that the val2_* variables are intended to be used to select ranges of angles to be processed using the 3rd and 4th values in each set of 4 values comprising a set (even though these variables are not referenced at all in your sample code).
The code you provided seemed to be saving a minimum value in the variable named max_force instead of a maximum value. The code below sets max_force1 to the maximum value found in the 2nd field in each test set with an angle in in the inclusive range $val1_angle_low <= field1 <= $val1_angle_high and sets max_force2 to the maximum value found in the 4th field in each test set with an angle in in the inclusive range $val2_angle_low <= field3 <= $val2_angle_high. Your code also multiplied force values by 100 for no obvious reason. At the end max_force1, max_force2, and the maximum of those two values are all printed. The code below does not modify input values, but does print maximum force values at the end after multiplying them by 100.
I do not know why the tail commands in your code start collecting data on line 9 of your input file. The sample data file given seems to have data starting on line 4 of the. The code below assumes data starts on the line following the 1st line in the file where the 3rd semicolon separated field is the string force (which appears to be the last line of the headers in your sample input).
The code below makes wild guesses at what in your sample code was desired output and what was intended to be printed only as debugging information. With the unconditional output and the debugging output, I hope that you will find ways to print what you want. I don't know if you wanted a single maximum force value or one value for the data from the 1st two values in each set of values and one value for the data from the 2nd two values in each set. The following codes prints individual and combined data.
The sample data you provided didn't now have any column three values in the range 33 through 55, so no value was selected from the 3rd and 4th values in each set. The following code strips DOS <carriage-return> characters from the input and (when run in a Locale where the LC_NUMERIC category has period as the radix character) can process input files that have period, comma, or a mixture of both as the radix character.
The following code was written and tested using the Korn shell, but should work with any shell that recognizes Bourne shell syntax (rather than csh shell syntax). It won't work with a pure Bourne shell because it needs basic POSIX parameter expansions (copied from your sample script). So, it should work with ash, bash, dash, ksh, zsh, and other shells that recognize the syntax used by these shells.
If you want to enable debugging printouts, switch the line just before the exit at the end of the script with the line just after the exit.
If someone wants to try this code on a Solaris/SunOS system, change awk in this script to /usr/xpg4/bin/awk or nawk.
Code:
#!/bin/ksh
csv=${1:-"/home/my/test.csv"}
val1_angle_high=66
val1_angle_low=56
val2_angle_high=55
val2_angle_low=33
awk -F';' -v v1low=$val1_angle_low -v v1high=$val1_angle_high \
-v v2low=$val2_angle_low -v v2high=$val2_angle_high '
FNR == 1 {
# Note that we are looking for the end of the header in this file.
hdr = 1
}
hdr { # Look for the last header line in this file.
if($3 == "force")
hdr = 0
if(debug) printf("hdr %d deleted: %s\n", FNR, $0)
next
}
{ # Process data lines...
# Convert commas to periods (assume European data collector with
# "," as radix character and American data processor with "." as
# radix character) and get rid of DOS <carriage-return>s.
gsub(",", ".")
gsub("\r", "")
if(debug) printf("line %d: %s\n", FNR, $0)
# Process all test sets on the current line...
for(col = 2; col < NF; col += 4) {
# Look for val1 angle in range.
if($col >= v1low && $col <= v1high)
# Look for new maximum...
if(found1++) {
if($(col + 1) > max_force1)
max_force1 = $(col + 1)
} else max_force1 = $(col + 1)
# Look for val2 angle in range.
if($(col + 2) >= v2low && $(col + 2) <= v2high)
# Look for new maximum...
if(found2++) {
if($(col + 3) > max_force2)
max_force2 = $(col + 3)
} else max_force2 = $(col + 3)
if(debug) {
printf("subscript: %d\n", col)
printf("range1 cnt: %d, angle1=%s, max=%s\n",
found1, $col,
found1 ? max_force1 : "undefined")
printf("range2 cnt: %d, angle2=%s, max=%s\n",
found2, $(col + 2),
found2 ? max_force2 : "undefined")
}
}
}
END { # Print results.
printf("# of values found in range1 (%f <= angle1 <= %f): %d\n%s ",
v1low, v1high, found1, "Maximum selected range1 value: ")
if(found1)
printf("%f\n", max_force1 * 100)
else print "undefined"
printf("# of values found in range2 (%f <= angle2 <= %f): %d\n%s ",
v2low, v2high, found2, "Maximum selected range2 value: ")
if(found2)
printf("%f\n", max_force2 * 100)
else print "undefined"
printf("# of values found in range: %d\nMaximum selected value: ",
found1 + found2)
if(found1 + found2)
printf("%f\n",
((found1 && found2) ? \
max_force1 > max_force2 ? max_force1 : max_force2 : \
found1 ? max_force1 : max_force2) * 100)
else print "undefined"
}' "$csv"
exit
}' debug=1 "$csv"
With the sample data you provided (and debugging disabled), the above code produces the output:
Code:
# of values found in range1 (56.000000 <= angle1 <= 66.000000): 3
Maximum selected range1 value: -439.000000
# of values found in range2 (33.000000 <= angle2 <= 55.000000): 0
Maximum selected range2 value: undefined
# of values found in range: 3
Maximum selected value: -439.000000
Hi all.
I want to use sed to remove a word that ends with "!" in the first page of a file. The word I want to remove is: "DNA!".
I have search for an answer and nothing of what I found helped me.
~faizlo (2 Replies)
somedata | awk -F"#soils#" '{split($1,a,"NITNUM="); print a}'
how can i edit the content of array 2 above?
what i want to do is get rid of single quotes and double quotes. and then turn the "NewLine" into new lines.
the data in array 2 (a) is:
... (2 Replies)
Hi ,
seq can be 0...128
int windex = seq / 8;
int bindex = seq % 8;
unsigned char bitvalue = '\x01' << (7-bindex) ;
bpv.bitmapvalue = bitvalue;
This is the part of a program to convert decimal to bitmap value of hexadecimal.
I want this to change to convert only to... (1 Reply)
Trying to do some control flow parsing based on the index postion of an array member. Here is the pseudo code I am trying to write in (preferably in pure bash) where possible. I am thinking regex with do the trick, but need a little help.
pesudo code
if == ENDSINFIVEINTS ]]; then
do... (4 Replies)
HI All ,
I am using AIX version 6 . having issue with below perl code,
sub Article ($procudure, @params) {
my ($procudure, @params) = @_;
#Get handle TO Dataware House DATABASE
try {
my $dbHandle = &Cobol::DBAccess::getDBDwConnection();
,,,,,,,,,,,,,
,,,,,,,,,,,,... (3 Replies)
I am seeing an special character in my file
when i do the
cat filename | od-bc
I see a value of 376 for that special character.
I would like to find the decimal value for the character.
For example the decimal value for ctrl-Y is char(25).
Appreciate help on this. (11 Replies)
Hi all,
I have a input like this
3AF9:3B01
and need to expand to the below output
3AF9
3AFA
3AFB
3AFC
3AFD
3AFE
3AFF
3B00
3B01
Please let me know the easiest way for achieving this. Thanks for the help in advance... (6 Replies)
Hi !
I'm having problem with assigning values to a multidimensional character array.
i wanted to have an array with 48 fields ,each filed being of varying size and hence have declared the array as
char struct_arr;
I am trying to assign values to the fileds as
struct_arr = token
... (1 Reply)
If i have a variable which is a decimal number, i.e 34.05 How can you make decimal point fall on the 15th character on the screen? Or any other that you can specify? Can you do it using sed or awk? (3 Replies)
09-17-2015
