Sponsored Content
Top Forums Shell Programming and Scripting How many studies have unequal values for each pair? Post 302978103 by senhia83 on Tuesday 26th of July 2016 09:53:10 AM
Old 07-26-2016
How many studies have unequal values for each pair?

I have several Studies (s) which has points (p) having Values (v).
My goal is to determine for each pair of points, how many studies have different values ( if available ).

Code:
Study	Point	Value
1	p1	value1
1	p2	value2
1	p3	value1
1	p4	value3
1	p5	value3
2	p2	value1
2	p4	value1
3	p1	value1
3	p5	value5
3	p3	value1
4	p2	value1
4	p4	value5


For example, the pair (p1,p5) are involved in 2 studies , STUDY 1 (value1,value3 ) and STUDY 3 (value1, value5 ) where both values are different. So the count for this pair is 2. Pair (p1,p3) is present in both studies 1 and 3 with same values. So the count is 0.


So my desired output is


Code:
Point1	Point2	#StudiesWhereValuesAreDifferentForThisPair
p1	p2	1
p1	p4	1
p1	p5	2
p2	p3	1
p2	p4	2
p2	p5	1
p3	p4	1
p3	p5	2
p4	p5	1

I do have a working solution for this which works for a small data-set for runs forever for the actual dataset which has several thousand factors in each column

Here is my solution

Code:
awk '{sp[$1 FS $2]=$3;s[$1];p[$2];next }END { for(ss in s) { for (p1s in p) { for (p2s in p)   { if (p1s !=p2s) { print ss,p1s,p2s,sp[ss FS p1s],sp[ss FS p2s] } }}}}'  | awk 'NF==5' | awk '$4!=$5' dataset

Please help me achieve the same in a smarter way.

Last edited by senhia83; 07-26-2016 at 11:34 AM..
 

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting a file into unequal parts

How do I split a file into many parts but with different amounts of lines per part? I looked at the split command but that only splits evenly. I'd like a range specified to determine how many lines each output file should have. For example, if the input file has 1000 lines and the range is... (1 Reply)
Discussion started by: revax
1 Replies

2. UNIX for Dummies Questions & Answers

Merge two files with common IDs but unequal number of rows

Hi, I have two files that I would like to merge and think that there should be a solution using awk. The files look something like this: file 1 IDX1 IDY1 IDX2 IDY2 IDX3 IDY3 file 2 IDY1 dataA data1 IDY2 dataB data2 IDY3 dataC data3 Desired output IDX1 IDY1 dataA data1 IDX2 ... (5 Replies)
Discussion started by: katie8856
5 Replies

3. Shell Programming and Scripting

Newline between unequal record fields

Assume the following 5 records (field separator is a space): 0903 0903 0910 0910 0910 0910 0910 0910 0917 0917 0917 0917 0924 1001 1001 1001 1001 1008 1008 1008 1008 1015 1015 1015 1015 1022 1029 1029 1029 1029 1105 1105 1105 1105 1112 1112 1112 1112 1119 1126 1126 1126 1126 1203 1203 1203 1203... (8 Replies)
Discussion started by: tree
8 Replies

4. Shell Programming and Scripting

Pair wise comparisons

Hi, I have 25 groups and I need to perform all possible pairwise compariosns between them using the formula n(n-1)/2. SO in my case it will be 25(25-1)/2 which is equal to 300 comparisons. my 25 groups are FG1 FG2 FG3 FG4 FG5 NT5E CD44 CD44 CD44 AXL ADAM19 CCDC80 L1CAM L1CAM CD44... (1 Reply)
Discussion started by: Diya123
1 Replies

5. Shell Programming and Scripting

Compare two unsorted unequal files extracted from xml

I have two files for comparison which are extracts from set of xml files. file1 has: Comparing File: BRCSH1to320140224CC3.xml :: TZZZ:BR :: TAZZ:OUT UIZZ:0 :: ERAZ:1.000000 UIZZ:0 :: CTZZ:B UIZZ:0 :: CCAZ:MYR Comparing File: BRMY20140224CC18REG013SPFNSY13.xml :: TZZZ:BR :: TAZZ:INB... (1 Reply)
Discussion started by: vamsi gunda
1 Replies

6. Shell Programming and Scripting

Finding difference between two columns of unequal length

Hi, I have two files which look like this cat waitstate.txt 18.2 82.1 cat gostate.txt 5.6 5.8 6.1 6.3 6.6 6.9 7.2 7.5 (4 Replies)
Discussion started by: jamie_123
4 Replies

7. Shell Programming and Scripting

awk name pair values

Team, I have a file like below FILE: NAM1,KEY1,VAL1 NAM1,KEY2,VAL2 NAM1,KEY3,VAL3 NAM2,KEY1,VALA NAM2,KEY2,VALB NAM2,KEY3,VALCOutput: I have to build commands like below <Script> VAL1 VAL2 VAL3 NAME1 <Script> VALA VALB VALC NAME2Can you please help with awk command i can use... (4 Replies)
Discussion started by: mallak
4 Replies
MIN(3)									 1								    MIN(3)

min - Find lowest value

SYNOPSIS
mixed min (array $values) DESCRIPTION
mixed min (mixed $value1, mixed $value2, [mixed $...]) If the first and only parameter is an array, min(3) returns the lowest value in that array. If at least two parameters are provided, min(3) returns the smallest of these values. Note Values of different types will be compared using the standard comparison rules. For instance, a non-numeric string will be com- pared to an integer as though it were 0, but multiple string values will be compared alphanumerically. The actual value returned will be of the original type with no conversion applied. PARAMETERS
o $values - An array containing the values. o $value1 - Any comparable value. o $value2 - Any comparable value. o $... - Any comparable value. RETURN VALUES
min(3) returns the parameter value considered "lowest" according to standard comparisons. If multiple values of different types evaluate as equal (e.g. 0 and 'abc') the first provided to the function will be returned. EXAMPLES
Example #1 Example uses of min(3) <?php echo min(2, 3, 1, 6, 7); // 1 echo min(array(2, 4, 5)); // 2 // The string 'hello' when compared to an int is treated as 0 // Since the two values are equal, the order they are provided determines the result echo min(0, 'hello'); // 0 echo min('hello', 0); // hello // Here we are comparing -1 < 0, so -1 is the lowest value echo min('hello', -1); // -1 // With multiple arrays of different lengths, min returns the shortest $val = min(array(2, 2, 2), array(1, 1, 1, 1)); // array(2, 2, 2) // Multiple arrays of the same length are compared from left to right // so in our example: 2 == 2, but 4 < 5 $val = min(array(2, 4, 8), array(2, 5, 1)); // array(2, 4, 8) // If both an array and non-array are given, the array is never returned // as comparisons treat arrays as greater than any other value $val = min('string', array(2, 5, 7), 42); // string // If one argument is NULL or a boolean, it will be compared against // other values using the rule FALSE < TRUE regardless of the other types involved // In the below examples, both -10 and 10 are treated as TRUE in the comparison $val = min(-10, FALSE, 10); // FALSE $val = min(-10, NULL, 10); // NULL // 0, on the other hand, is treated as FALSE, so is "lower than" TRUE $val = min(0, TRUE); // 0 ?> SEE ALSO
max(3), count(3). PHP Documentation Group MIN(3)
All times are GMT -4. The time now is 06:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy