Sponsored Content
Top Forums Shell Programming and Scripting Calculate Correlation between two fields ! Post 302499278 by csierra on Wednesday 23rd of February 2011 10:22:52 PM
Old 02-23-2011
Question Calculate Correlation between two fields !

Hello,

I request your help with a shell script (awk) that ask for two inputs in order to calculate the correlation of the last rows between two fields ( 3 and 4).

Data:
Code:
EC-GLD,1/25/2011,41.270000,129.070000
EC-GLD,1/26/2011,41.550000,129.280000
EC-GLD,1/27/2011,42.260000,127.800000
EC-GLD,1/28/2011,41.940000,127.950000
EC-GLD,1/31/2011,42.380000,129.250000
EC-GLD,2/1/2011,42.580000,129.330000
EC-GLD,2/2/2011,42.650000,129.450000
EC-GLD,2/3/2011,41.990000,129.280000
EC-GLD,2/4/2011,41.170000,131.230000
EC-GLD,2/7/2011,41.300000,131.230000
EC-GLD,2/8/2011,41.650000,132.800000
EC-GLD,2/9/2011,41.900000,132.490000
EC-GLD,2/10/2011,41.400000,132.000000
EC-GLD,2/11/2011,41.240000,132.090000
EC-GLD,2/14/2011,41.200000,132.700000
EC-GLD,2/15/2011,41.010000,133.630000
EC-GLD,2/16/2011,41.040000,133.450000
EC-GLD,2/17/2011,40.540000,134.470000
EC-GLD,2/18/2011,39.690000,134.880000
EC-GLD,2/22/2011,39.660000,136.190000

For example:
----> Correlation 1 : 15 (Last rows)
----> Correlation 2 : 10 (Last rows)

Output
Code:
EC-GLD,15,-0.8979
EC-GLD,10,-0.9037

Thanks a lot for your help

Last edited by Franklin52; 02-24-2011 at 03:36 AM.. Reason: Please use code tags, thank you
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

chmod and cgi correlation

How much do chmod settings affect cgi scripts?? I have a "webmaster" at my work that says I cannot change the permissions on the cgi scripts, and that they work with only certain permissions. They are set for 644, I want to change them to 775 and put her in her own group, like she should be, not... (6 Replies)
Discussion started by: bigmacc
6 Replies

2. Shell Programming and Scripting

correlation coefficient - Awk

Hi guys I have an input file with multiple columns and and rows. Is it possible to calculate correlation of certain value of certain No (For example x of S1 = 112) with all other values (for example start with x 112 corr a 3 of S1 = x-a 0.2 ) INPUT ******* No S1 S2 S3 S4 Sn a 3 ... (2 Replies)
Discussion started by: quincyjones
2 Replies

3. Shell Programming and Scripting

Calculate age of a file | calculate time difference

Hello, I'm trying to create a shell script (#!/bin/sh) which should tell me the age of a file in minutes... I have a process, which delivers me all 15 minutes a new file and I want to have a monitoring script, which sends me an email, if the present file is older than 20 minutes. To do... (10 Replies)
Discussion started by: worm
10 Replies

4. Shell Programming and Scripting

AWK - calculating simple correlation of rows

Is there any way to calculate a simple correlation of few selected rows with all the rows in input ? In the below example I selected Row01,02,03 and correlated with all the rows. I was trying to run in R. But the this big data matrix is too much to handle for R and eventually my system is... (3 Replies)
Discussion started by: quincyjones
3 Replies

5. Shell Programming and Scripting

Pearson correlation between two files

Hi, I want a quick way to determine the pearson correlation between two files. The two files have the same format with only the 3rd column varying. E.g. of file 1 chr1 0 62 chr1 1 260 chr1 2 474 chr1 3 562 chr1 4 633 chr1 5 870 chr1 6 931 chr1 7 978 chr1 8 1058 chr1 9 1151 E.g.... (1 Reply)
Discussion started by: kylle345
1 Replies

6. Shell Programming and Scripting

3 column .csv --> correlation matrix; awk, perl?

Greetings, salutations. I have a 3 column csv file with ~13 million rows and I would like to generate a correlation matrix. Interestingly, you all previously provided a solution to the inverse of this problem. Thread title: "awk? adjacency matrix to adjacency list / correlation matrix to list"... (6 Replies)
Discussion started by: R3353
6 Replies

7. Shell Programming and Scripting

awk to calculate fields only if match is found

Trying to combine the matching $5 values between file1 and file2. If a match is found then the last $6 value in the match and the sum of $7 are outputted to a new file. The awk below I hope is a good start. Thank you :). file1 chr12 9221325 9221448 chr12:9221325-9221448 A2M 1... (5 Replies)
Discussion started by: cmccabe
5 Replies

8. Shell Programming and Scripting

Correlation Between 3 Different Loops using Bash

I have 3 loops that I use to determine the permission level of AWS user accounts. This array lists the AWS policy ARN (Amazon Resource Name): for ((policy_index=0;policy_index<${#aws_managed_policies};++policy_index)); do aws_policy_arn="${aws_managed_policies}" ... (1 Reply)
Discussion started by: bluethundr
1 Replies
MooseX::Getopt::GLD(3pm)				User Contributed Perl Documentation				  MooseX::Getopt::GLD(3pm)

NAME
MooseX::Getopt::GLD - A Moose role for processing command line options with Getopt::Long::Descriptive VERSION
version 0.59 SYNOPSIS
## In your class package My::App; use Moose; with 'MooseX::Getopt::GLD'; # or with 'MooseX::Getopt::GLD' => { getopt_conf => [ 'pass_through', ... ] }; has 'out' => (is => 'rw', isa => 'Str', required => 1); has 'in' => (is => 'rw', isa => 'Str', required => 1); # ... rest of the class here ## in your script #!/usr/bin/perl use My::App; my $app = My::App->new_with_options(); # ... rest of the script here ## on the command line % perl my_app_script.pl -in file.input -out file.dump OPTIONS
This role is a parameterized role. It accepts one configuration parameter, "getopt_conf". This parameter is an ArrayRef of strings, which are Getopt::Long configuration options (see "Configuring Getopt::Long" in Getopt::Long) AUTHOR
Stevan Little <stevan@iinteractive.com> COPYRIGHT AND LICENSE
This software is copyright (c) 2007 by Infinity Interactive, Inc. This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself. perl v5.18.2 2013-11-30 MooseX::Getopt::GLD(3pm)
All times are GMT -4. The time now is 05:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy