Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Compare Values between column in the same file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Compare Values between column in the same file
# 8  
Old 09-28-2016
Quote:
Originally Posted by Nina2910
---------- Post updated at 12:59 AM ---------- Previous update was at 12:58 AM ----------

@Don... Sorry for numerous updates . Trust me I was just trying to make it as clear as possible and I am done updating .. Sorry for inconvenience today.
Great. I am glad that you are done updating.

Now, please answer all of the questions I asked in post #6 in this thread. And, please give us a clear description of what fields need to be checked for all of the possible combinations that might appear in field #3 in your input.
This User Gave Thanks to Don Cragun For This Post:
Nina2910
# 9  
Old 09-28-2016
@Don ...I have to answer your questions in blue color.
You have now told us what columns need to be added when column 3 is one of A, C, E, M, N, P, S, SE, or EAMS. And you have told us that column 3 can have numerous other values with absolutely no specification of what is supposed to happen in those cases. (Unless you have changed things again and I missed it.)
column 3 can be any of these Values A, C, E, M, N, P, S or combinations of these characters eg. AS,SE,MP,PS,SAE,EAM, CEPS and more
I updated the intial post yesterday to clarify what columns should be added when column 3 have any values among A, C, E, M, N, P, S.
for the combination, code should check only common columns for all single character in combination.
example 1-
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if it is ES or SE, then code should check common columns which in this case are 9,10,11,12,13 and sum of these should zero.

example 2-
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.

so if column 3 Value is SAE, EAS,ASE, then code should check common columns which in this case are 10,11,12,13 and sum of these should zero.


Please give us a clear description of what needs to be done and then ask for help. I'm tired to trying to keep up with a constantly changing, incomplete description of what you want done.
You constantly state that a list of fields have to be added, but from your sample data that seems like a waste of time. Since none of your input fields seem to have negative values.

Row 4 in the INPUT file in which column 3 has Values SE , so code should check 9,10,11,12,13 columns and sum of should be zero which is not the case.
Input:-
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0
sum of the column 9+10+11+12+13 >0 in the above case. That's the reason output file has WARNING in the Newcolumn.
Output:-
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING

can your script just add warning (which is what is in your description) or Warning (which is what is in your code) or WARNING (which is what is in your sample output) instead of Good (which is in your description) or Goodd (which is what is in your code) or GOOD (which is what is in your sample output) when any of the specified fields contains a non-zero value (instead of adding them all together and then looking for a non-zero sum)?

Updated it already in the initial post.

---------- Post updated at 11:20 AM ---------- Previous update was at 10:47 AM ----------

@Ravinder..Thank you for the code however for row number 4 and 6 the code should have output

COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING

column 3 can be any of these Values A, C, E, M, N, P, S or combinations of these characters eg. AS,SE,MP,PS,SAE,EAM, CEPS and more

Mainly it would be in A ,E, M,P,S,C,N,AM,EA,EM,SC,SM,ES,SE ,AS,SA,ESA ,SEA, EMS,ESM,SEM ,SCE,SEC, EAM,SAM,EAMS,EASM,ESAM,SEAM,ESCM.
I updated the intial post yesterday to clarify what columns should be added when column 3 have any values among A, C, E, M, N, P, S.
for the combination, code should check only common columns for all single character in combination.
example 1-
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if it is ES or SE, then code should check common columns which in this case are 9,10,11,12,13 and sum of these should zero.

example 2-
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.

so if column 3 Value is SAE, EAS,ASE, then code should check common columns which in this case are 10,11,12,13 and sum of these should zero.

---------- Post updated at 11:26 AM ---------- Previous update was at 11:20 AM ----------

@Don ..One more update

I Dig more and found that the expected column 3 values would be either of these ::- A ,E, M,P,S,C,N,AM,EA,EM,SC,SM,ES,SE ,AS,SA,ESA ,SEA, EMS,ESM,SEM ,SCE,SEC, EAM,SAM,EAMS,EASM,ESAM,SEAM,ESCM.

however if the script is able to Calculate the values based on Dynamic combinations that would really helpful.

Thanks again

---------- Post updated at 01:28 PM ---------- Previous update was at 11:26 AM ----------

@Ravinder I have update the code you according to me. however it will work only if I know values of column $3 in advance.

Code:
 
 awk 'BEGIN{FS=OFS="\t"} NR<2{print $0,"NewColumn";next} function sum_check(a,b,c,d,e,f){ 
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="GOOD"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="WARNING"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  }
if(length($w)==2)         {
                                                                        if($w=="SE"||$w=="ES"){
                                                                                        sum_check(9,10,11,12,13)
                                                                                   };
                                                                  }
if(length($w)==3)         {
                                                                        if($w=="SEA"){
                                                                                        sum_check(10,11,12,13)
                                                                                   };
                                                                  }
if(length($w)==4)         {
                                                                        if($w=="EAMS"){
                                                                                        sum_check(11,12,13)
                                                                                   };
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    INPUT_file

Thank you. I was thinking if I can have more dynamic values.
# 10  
Old 09-28-2016
How about
Code:
awk '
BEGIN   {MX = split ("      SEAMPNC", CH, _)
        }
NR > 1  {SUM = 0
         for (i = 7; i<=13; i++) if ($3 !~ CH[i]) SUM += $i
         $(NF+1) = SUM?"WARNING":"GOOD"
        }
1
 ' file
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0 GOOD
VEGE Onion S W 17 0 17 0 0 0 0 0 0 GOOD
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 GOOD
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 GOOD
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING

Please be aware that your conditions in post#1 seem to be inconsistent for "P" and "N".
This User Gave Thanks to RudiC For This Post:
Nina2910
# 11  
Old 09-28-2016
Quote:
Originally Posted by Nina2910
@Ravinder I have update the code you according to me. however it will work only if I know values of column $3 in advance.
Code:
 
 awk 'BEGIN{FS=OFS="\t"} NR<2{print $0,"NewColumn";next} function sum_check(a,b,c,d,e,f){ 
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="GOOD"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="WARNING"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  }
if(length($w)==2)         {
                                                                        if($w=="SE"||$w=="ES"){
                                                                                        sum_check(9,10,11,12,13)
                                                                                   };
                                                                  }
if(length($w)==3)         {
                                                                        if($w=="SEA"){
                                                                                        sum_check(10,11,12,13)
                                                                                   };
                                                                  }
if(length($w)==4)         {
                                                                        if($w=="EAMS"){
                                                                                        sum_check(11,12,13)
                                                                                   };
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    INPUT_file

Thank you. I was thinking if I can have more dynamic values.
Hello Nina2910,

Let's say I have edited your Input_file as follows.
Code:
cat Input_file
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0
VEGE Onion S W 17 0 17 0 0 0 0 0 0
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 10
FRUIT APPLE EA W 397 32 309 56 309 309 0 0 0
FRUIT APPLE MS W 397 32 309 56 309 309 0 0 10
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0
FRUIT APPLE EAS W 397 32 309 56 309 309 0 0 0
FRUIT APPLE EAM W 397 32 309 56 309 309 0 0 0
FRUIT APPLE SMEAC W 808 58 663 87 488 20 0 10 0
FRUIT APPLE CSEAN W 808 58 663 87 488 20 0 10 0
FRUIT APPLE MCSEA W 808 58 663 87 488 20 0 01 0

Then when I run my code posted in POST#3 as follows is the result.
Code:
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0 Good
VEGE Onion S W 17 0 17 0 0 0 0 0 0 Good
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 warning
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 10 warning
FRUIT APPLE EA W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE MS W 397 32 309 56 309 309 0 0 10 warning
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 Good
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE EAS W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE EAM W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE SMEAC W 808 58 663 87 488 20 0 10 0 warning
FRUIT APPLE CSEAN W 808 58 663 87 488 20 0 10 0 warning
FRUIT APPLE MCSEA W 808 58 663 87 488 20 0 01 0 warning

So could you please let me know about following question's answers here.
i- Is above code working as per your expectations or not?
ii- So as per your requirements only strings AEMCSN or specific strings mentioned by you could be in $3, if this is true then in code for single character conditions are there(for different single character different fields we need to add and compare so they are different in conditions there), you have mentioned it needs to be more dynamic, could you please elaborate it more.
Please try to rephrase your requirements in a single post with code tags please, with sample Input_file and expected output sample too.

EDIT: BY seeing your try to edit my code got a little pulse of your objective here(may be), so could you try follwing and let us know if this helps you.
Code:
awk 'function sum_check(a,b,c,d,e,f){
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="Good"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="warning"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  };
                                        if(length($w)==2 && $w ~ /[SEAMPC][SEAMPC]/)         {
												sum_check(9,10,11,12,13);
											     }
					if(length($w)==3 && $w ~ /[SEAMPC][SEAMPC]/)          { 
											 	sum_check(10,11,12,13);
											     }	
									
					if(length($w)==4 && $w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '   Input_file

Thanks,
R. Singh
Last edited by RavinderSingh13; 09-28-2016 at 03:33 PM.. Reason: Added one more solution and checking with user on same too now.
This User Gave Thanks to RavinderSingh13 For This Post:
Nina2910
# 12  
Old 09-28-2016
@Ravinder your code is partially satisfying the expectation.
I am stating the conditions again on which code will decide GOOD or WARNING for Single character Values of column 3 (A, C, E, M, N, P, S).
Code:
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= M then check column 7,8,9,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= P then check column 7,8,9,10,11,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= N then check column 7,8,9,10,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= C then check column 7,8,9,10,11,12 (values should be zero in each) and hence sum of these columns should be zero.

as column 3 can be any of these Values A, C, E, M, N, P, S or combinations of these characters eg. AS,SE,MP,PS,SAE,EAM, CEPS and more.The code should be able to Calculate the values based on Dynamic combinations.

for the combination, code should check only common columns for all single character in combination.
example 1-
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if it is ES or SE, then code should check common columns which in this case are 9,10,11,12,13 and sum of these should zero.
example 2-
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if column 3 Value is SAE, EAS,ASE, then code should check common columns which in this case are 10,11,12,13 and sum of these should zero.

---------- Post updated at 03:06 PM ---------- Previous update was at 02:54 PM ----------

@Rudi ...row 3,4,6 should have "WARNING" in the output file. I have fixed the my output file in post #1.
Last edited by Nina2910; 09-28-2016 at 09:00 PM..
# 13  
Old 09-28-2016
If you ignore the heading and start counting from the row after it, row 4 and 6 in my result HAVE "WARNING".
Are you sure that "P" and "N" need to ignore the SAME column (=11)? Or, shouldn't "N" ignore col 12?
This User Gave Thanks to RudiC For This Post:
Nina2910
# 14  
Old 09-28-2016
@Rudi...yes you are right and code is working perfectly fine..Thank you.

would it be possible if you can explain the code ?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Compare values in multiple rows in one column using awk

I would like to compare values in column 8, and grep the ones where the different is > 1, columns 1 and 2 are the key for array. Every 4 rows the records values in columns 1 and 2 changed. Then, the comparison in the column 8 need to be done for the 4 rows everytime columns 1 and 2 changed ... (4 Replies)
Discussion started by: jiam912
4 Replies

2. Shell Programming and Scripting

Compare two files column values using awk

Judi # cat File1 judi /export/home 76 judi /usr 83 judi # judi # cat File2 judi /export/home 79 judi /usr 82 judi # if COLUMN3 of File2 is greater that COLUMN3 of File1, then print File2's lines juid /export/home 79 Code tags please (2 Replies)
Discussion started by: judi
2 Replies

3. Shell Programming and Scripting

How to compare the values of a column in a same file using awk?

Dear Unix experts, I have got a file where I would like to compare the values of second column if first column is same in such a way that the difference between the values is >50. If not, I would like to discard both values. For example, my input file looks like - comp275_c0_seq2 73... (7 Replies)
Discussion started by: utritala
7 Replies

4. UNIX for Dummies Questions & Answers

Compare values of fields from same column with awk

Hi all ! If there is only one single value in a column (e.g. column 1 below), then return this value in the same output column. If there are several values in the same column (e.g. column 2 below), then return the different values separated by "," in the output. pipe-separated input: ... (11 Replies)
Discussion started by: lucasvs
11 Replies

5. Shell Programming and Scripting

Compare values in two files. For matching rows print corresponding values from File 1 in File2.

- I have two files (File 1 and File 2) and the contents of the files are mentioned below. - I am trying to compare the values of Column1 of File1 with Column1 of File2. If a match is found, print the corresponding value from Column2 of File1 in Column5 of File2. - I tried to modify and use... (10 Replies)
Discussion started by: Santoshbn
10 Replies

6. Shell Programming and Scripting

Take values from a column and put it in a variable and compare

Hi, I have a table in unix from which i want to read the contents line by line, then filter out the values from 6th column one by one and compare it a fixed value. How to do this? (7 Replies)
Discussion started by: arijitsaha
7 Replies

7. Shell Programming and Scripting

How to compare the values of a column in awk in a same file and consecutive lines..

I would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line. Input File ========== PDB 2500 RTDB 123 RTDB-EAGLE 122 VSCCP 2565... (4 Replies)
Discussion started by: manuswami
4 Replies

8. UNIX for Dummies Questions & Answers

Compare two files using awk or sed, add values in a column if their previous fields are same

Hi All, I have two files file1: abc,def,ghi,5,jkl,mno pqr,stu,ghi,10,vwx,xyz cba,ust,ihg,4,cdu,oqw file2: ravi,def,kishore ramu,ust,krishna joseph,stu,mike I need two output files as follows In my above example, each row in file1 has 6 fields and each row in file2 has 3... (1 Reply)
Discussion started by: yerruhari
1 Replies

9. UNIX for Advanced & Expert Users

Compare two files using awk or sed, add values in a column if their previous fields are same

Hi All, I have two files file1: abc,def,ghi,5,jkl,mno pqr,stu,ghi,10,vwx,xyz cba,ust,ihg,4,cdu,oqw file2: ravi,def,kishore ramu,ust,krishna joseph,stu,mike I need two output files as follows In my above example, each row in file1 has 6 fields and each row in file2 has 3... (1 Reply)
Discussion started by: yerruhari
1 Replies

10. Shell Programming and Scripting

I need to extract last column of a file and compare the values

Hi, I am new to unix and I need help in solving below mentioned issue, really appreciate ur help. I have a file sam, john, 2324, 07142007 tom, thomson, 2343, 07142007 john, scott, 2478, 07142007 its a comma delimited file, I need to extract the last column from each line and this... (4 Replies)
Discussion started by: vukkusila
4 Replies
Login or Register to Ask a Question