I am trying to do standardization (subtracting the mean from original value and dividing by stdev) using an automated script.
I have a file with 10000 rows. I need to initially calculate mean and stdev for each column and then using those values I have to obatin the new standardized values.. I can do this pretty easily in excel.. But I am looking for an automated script.
input
The mean for column 1 is 6.880 and stdev is 2.066
I will now subtract the mean from my observation and divide by stdev to (10.29006-6.880)/2.066. I will do this on all subsequent observations row-wise in column 1. For column 2 again I will find its mean and corresponding stdev and follow the same procedure.
In bash, you could read each record into a simple array with read -a, put the first column values in a simple array and put the numeric values in an associative array that mimics a two dimensional array by keying it with "row:col". Then you can walk the rows column by column and do your calculations and modifications. When done, you regenerate the file from the arrays for rows 0-n, columns 0-m. I suppose awk can do this, too.
You could use one dimensional arrays tricked into another dimension by concatenation: $row$col, as $col is always one digit. typeset'ing variables can prevent repeated conversion from numeric to character form and back. For problems like this, life is simpler in PERL, C/C++, JAVA.
Hello
with examples and numbers you provide i wrote this code .
my shell is bourne . save the code in a file called diya123 and chmod it to 755 :
save your data in a file and , i assume that program and file both are in currrent working directory , invoke the program with
Hello
I don't know your bussiness and i has not work with excell !
I just test this script with one line of example you provided . can this scripts works as expected?
Hi there,
I'm a newbie in unix and am fishing for options related to how raw input data files are handled. The scenario, as I'm sure y'all must be very familiar with, is this : we receive upwards of 50 data files in ASCII format from various source systems - now each file has its own structure... (3 Replies)