Sponsored Content
Full Discussion: Normalize a dataset with AWK
Top Forums Shell Programming and Scripting Normalize a dataset with AWK Post 302294549 by otheus on Thursday 5th of March 2009 11:49:02 AM
Old 03-05-2009
I don't know what you mean by "normalize". Do you mean to put every column in terms of number of deviations from the mean? Would you have one output row, or one row for each row of input?

Your code is pretty good, but has a few bugs.
Code:
BEGIN{ FS="," }
{
    for(i=1;i <=  NF;i++)
    {
        total[i]+=$i;
        totalSquared[i]+=$i^2;
    }
    numberColumn=NF;
}
END{
    for (i=1;i <= numberColumn;i++)
    {
        media=total[i]/NR;
        printf("%.2f|%.2f\n",media,sqrt((totalSquared[i]/NR)-media^2));
    }
}

 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Accessing Mainframe Dataset

Hi May I know is there a way to read/copy a mainframe (IBM OS/390) dataset (sequential file) into a UNIX directory? Thank you for your time. IcyGuava (4 Replies)
Discussion started by: IcyGuava
4 Replies

2. Shell Programming and Scripting

Numbers of records in SAS dataset

I'm declaring a variable within a Korn shell to represent the total number of records in a SAS dataset and could use a little help with the syntax. This is what I have thus far: #!/usr/bin/ksh RecCount = `sas -x "select count(*) from /users/abc/123/sas_dataset.sas7bdat"` (2 Replies)
Discussion started by: sasaliasim
2 Replies

3. Shell Programming and Scripting

Computing dataset for a specific record

Hello everybody, I want to compute a data file in awk. I am new in awk and I need your help. The data file has the following fields. It has thousands of records. Col1 Col2 Col3 Col4 Col5 0.85 0.07 Fre 42:86 25 0.73 0.03 frp 21:10 28 0.64... (12 Replies)
Discussion started by: ubeejani
12 Replies

4. Programming

Dataset Library for C?

I am looking for an opensource dataset library for C. Something equivalent to ADO.Net. Specifically, I am looking for the following features: 1. Create a Dataset from a file (XML or CSV). 2. Create a Dataset from a select query using an ODBC connection. 3. Load a created Dataset into a... (1 Reply)
Discussion started by: a_programmer
1 Replies

5. UNIX for Dummies Questions & Answers

Normalize Data and write to a flat file

All, Can anyone please help me with the below scenario. I have a Flat file of the below format. ID|Name|Level|Type|Zip|MAD|Risk|Band|Salesl|Dealer|CID|AType|CValue|LV|HV|DCode|TR|DU|NStartDate|UserRole|WFlag|EOption|PName|NActivationDate|Os|Orig|Cus|OType|ORequired|DType 03|... (10 Replies)
Discussion started by: sp999
10 Replies

6. Solaris

flarecreate for zfs root dataset and ignore multiple dataset

Hi All, I want to write a script to create flar images on multiple servers. In non zfs filesystem I am using -X option to refer a file to exclude mounts on different servers. but on ZFS -X option is not working. I want multiple mounts to be ignore on ZFS base system during flarecreate. I... (0 Replies)
Discussion started by: uxravi
0 Replies

7. Shell Programming and Scripting

How to insert a column inside a dataset with awk?

Hello folks I have a file called fill1.txt which contains: 1 2 2 1 1 2 1 2 my other file is called fill2.txt which contains: 1 2 1 2 2 2 1 2 1 2 1 1 2 1 1 2 1 1 1 1 2 2 2 1 1 2 2 1 1 2 1 1 1 2 2 2 1 2 2 1 Now, I am looking for a awk command which could insert fill1.txt between... (1 Reply)
Discussion started by: sajmar
1 Replies

8. Programming

Need sql query to string split and normalize data

Hello gurus, I have data in one of the oracle tables as as below: Column 1 Column 2 1 NY,NJ,CA 2 US,UK, 3 AS,EU,NA fyi, Column 2 above has data delimited with a comma as shown. I need a sql query the produce the below output in two columns... (5 Replies)
Discussion started by: calredd
5 Replies

9. UNIX for Advanced & Expert Users

SAS dataset to CSV

Hi Guys, Is there a way to export a sas file i.e .sas7bdat file to .csv file with header and data using unix. I dont want to use SAS program instead using unix tool or unix scripting is it possible ? (25 Replies)
Discussion started by: Master_Mind
25 Replies
PSC(1)							      General Commands Manual							    PSC(1)

NAME
psc - prepare sc files SYNOPSIS
psc [-fLkrSPv] [-s cell] [-R n] [-C n] [-n n] [-d c] DESCRIPTION
Psc is used to prepare data for input to the spreadsheet calculator sc(1). It accepts normal ascii data on standard input. Standard out- put is a sc file. With no options, psc starts the spreadsheet in cell A0. Strings are right justified. All data on a line is entered on the same row; new input lines cause the output row number to increment by one. The default delimiters are tab and space. The column for- mats are set to one larger than the number of columns required to hold the largest value in the column. OPTIONS
-f Omit column width calculations. This option is for preparing data to be merged with an existing spreadsheet. If the option is not specified, the column widths calculated for the data read by psc will override those already set in the existing spreadsheet. -L Left justify strings. -k Keep all delimiters. This option causes the output cell to change on each new delimiter encountered in the input stream. The default action is to condense multiple delimiters to one, so that the cell only changes once per input data item. -r Output the data by row first then column. For input consisting of a single column, this option will result in output of one row with multiple columns instead of a single column spreadsheet. -s cell Start the top left corner of the spreadsheet in cell. For example, -s B33 will arrange the output data so that the spreadsheet starts in column B, row 33. -R n Increment by n on each new output row. -C n Increment by n on each new output column. -n n Output n rows before advancing to the next column. This option is used when the input is arranged in a single column and the spreadsheet is to have multiple columns, each of which is to be length n. -d c Use the single character c as the delimiter between input fields. -P Plain numbers only. A field is a number only when there is no imbedded [-+eE]. -S All numbers are strings. -v Print the version of psc SEE ALSO
sc(1) AUTHOR
Robert Bond PSC 7.16 19 September 2002 PSC(1)
All times are GMT -4. The time now is 06:39 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy