Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Compare Values between column in the same file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Compare Values between column in the same file
# 1  
Old 09-27-2016
Compare Values between column in the same file
Input File:-
Code:
 
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0
VEGE Onion S W 17 0 17 0 0 0 0 0 0
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0

The value in Column 3 should determine that among columns $7(SMS),$8(EMAIL),$9(AO),$10(Mail),$11(P),$12(N) and $13(cell), which column will have a non-zero integer value and which will have value zero.

eg.
example 1- if column 3 has value E then code should check only column 7,9,10,11,12,13 and sum of these columns (7+9+10+11+12+13) should be zero . If it is Zero then add GOOD else add WARNING to Newcolumn.
like in row 1 in the above file the sum of the 7,9,10,11,12,13 columns is zero so added Good in the out put file.
example 2 -if Column 3 has value SE then code should only check columns 9,10,11,12,13 and sum of these columns (9+10+11+12+13) should be zero.If it is Zero then add GOOD else add WARNING to Newcolumn.
like in row 4 the sum of columns 9,10,11,12,13 is not zero so added warning to the output file.

Now, single character value like E, S, A etc is fine but when there is a string like SE or AM or EAMS then there is a problem.

Like row 5 column 3 has values EAMS so it should only check column 11,12,13 and if those columns are greater then zero then add WARNING to Newcolumn.

Column 3 can be combination of any predefined values in no particular sequence eg. EM,ES,E,S,M,EAMS,EMAS,EP,P,SEC,SEA and more.


Conditions based on which code will decide GOOD or WARNING:-

Code:
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= M then check column 7,8,9,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= P then check column 7,8,9,10,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= N then check column 7,8,9,10,11,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= C then check column 7,8,9,10,11,12 (values should be zero in each) and hence sum of these columns should be zero.

I know its is very confusing. Please help. For single values(can be E or ES or EAM or ...) i am using below code. I have taken E as an example.

Code:
awk 'BEGIN{FS=OFS="\t"} NR<2{print $0,"NewColumn";next} {(($3 == "E") && ((($7+$9+$10+$11+$12+$13) >0))) ?c="WARNING"$14:c="GOOD"$14;printf "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,c;}' Input_File

Desired OUTPUT
Code:
 
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
VEGE Potato E W 396 12 0 384 0 0 0 0 0 GOOD
VEGE Onion S W 17 0 17 0 0 0 0 0 0 GOOD
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 GOOD
 FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 GOOD
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING

Last edited by Nina2910; 09-28-2016 at 09:03 PM.. Reason: Fixed output file row3
# 2  
Old 09-27-2016
You are correct. This is very confusing. You show us code, but do not tell us if it is doing what you want for the single case it seems to try to handle, and for all of the other cases you don't even say what output you do want. How do you expect us to help you write code if you don't clearly explain what that code is supposed to do?

Please show us the exact output that you want your script to produce for the sample input you provided, and explain in detail what output is supposed to be produced for any given value found in field 3 in your input file.

Is there supposed to be a header line in the output produced? If so, what header is supposed to appear in the output?
This User Gave Thanks to Don Cragun For This Post:
Nina2910
# 3  
Old 09-28-2016
@Don. Thank you for reading such a long post and your feedback. I tried to edit it.Please let me know if it is readable to you now.Otherwise I will try to edit it more
# 4  
Old 09-28-2016
Quote:
Originally Posted by Nina2910
@Don. Thank you for reading such a long post and your feedback. I tried to edit it.Please let me know if it is readable to you now.Otherwise I will try to edit it more
Hello Nina2910,

Could you please try following and let me know if this helps you.
Code:
awk 'function sum_check(a,b,c,d,e,f){if(($a+$b+$c+$d+$e+$f)==0){value="Good"} else {value="warning"}}  function cal(w){if(length($w)==1){if($w=="E"){sum_check(7,9,10,11,12,13)};if($3=="S"){sum_check(8,9,10,11,12,13)};if($3=="A"){sum_check(7,8,10,11,12,13)};if($3=="M"){sum_check(7,8,9,11,12,13)};if($3=="N"){sum_check(7,8,9,10,12,13)};if($3=="C"){sum_check(7,8,9,10,11,12)};$(NF+1)=value};if($3 ~ /[SEAMPC][SEAMPC]/){sum_check(11,12,13);$(NF+1)=value}} {cal(3);print}'  Input_file

Output will be as follows.
Code:
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0 Good
VEGE Onion S W 17 0 17 0 0 0 0 0 0 Good
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 warning
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 Good
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 Good

Also not sure how you output shown in last row and 3rd last row has warningin them as I could see their 11st, 12th and 13th field's sum is 0 only.
If it was a typo then please try above and let me know if this helps. Also rushing to somewhere so will add non-one liner form little later.
EDIT: Adding a non-one liner form of solution too now.
Code:
awk 'function sum_check(a,b,c,d,e,f){
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="Good"
                                                                  }
                                        else                      {
                                                                        value="warning"
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        $(NF+1)=value
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                         $(NF+1)=value;
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    Input_file

EDIT2: Improving the solution above by putting logic where once the match found in any of if condition then it shouldn't execute further conditions to save time of execution here.
Code:
awk 'function sum_check(a,b,c,d,e,f){
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="Good"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="warning"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    Input_file

Thanks,
R. Singh
Last edited by RavinderSingh13; 09-28-2016 at 06:46 AM.. Reason: Adding a non-one liner form of solution too successfully now.
This User Gave Thanks to RavinderSingh13 For This Post:
Nina2910
# 5  
Old 09-28-2016
Code:
#!/bin/bash

INPUT_FILE=input.txt

awk 'BEGIN{
E_Len=split("7,9,10,11,12,13",E_Arr,",");
}
{	
	if($3 == "E")
	{
		for(i=1;i<=E_Len;i++)
		{
			col_num=E_Arr[i];
			sum+=$col_num;
		}
		if ( sum == 0)
		{
			print $0",Good";
		}
		else
		{
			print $0",Bad";
		}
	}
	if(length($3)>1)
	{
		sum=$11+$12+$13
		if (sum>0)
		{
			print $0",Good";
		}
		else
		{
			print $0",Bad";
		}
	}
}' ${INPUT_FILE}

you have to repeat the read color coding for other checks as well
This User Gave Thanks to itkamaraj For This Post:
Nina2910
# 6  
Old 09-28-2016
Please STOP changing your posts. If you keep updating post #1 in your threads, no one can follow the conversation we are having trying to get a clear specification of your problem. In the future, please stop and think about what you are asking us to help you do before you submit your first post!

If you need to add more details, do so in subsequent posts; don't keep changing post #1 in a thread. During the time that I have been trying to compose this message, you have edited post #1 at least a dozen times. How can you expect anyone to help you if you keep changing the description of what you want done!

You have now told us what columns need to be added when column 3 is one of A, C, E, M, N, P, S, SE, or EAMS. And you have told us that column 3 can have numerous other values with absolutely no specification of what is supposed to happen in those cases. (Unless you have changed things again and I missed it.)

Please give us a clear description of what needs to be done and then ask for help. I'm tired to trying to keep up with a constantly changing, incomplete description of what you want done.

You constantly state that a list of fields have to be added, but from your sample data that seems like a waste of time. Since none of your input fields seem to have negative values, can your script just add warning (which is what is in your description) or Warning (which is what is in your code) or WARNING (which is what is in your sample output) instead of Good (which is in your description) or Goodd (which is what is in your code) or GOOD (which is what is in your sample output) when any of the specified fields contains a non-zero value (instead of adding them all together and then looking for a non-zero sum)?
# 7  
Old 09-28-2016
Quote:
Originally Posted by itkamaraj
Code:
#!/bin/bash

INPUT_FILE=input.txt

awk 'BEGIN{
E_Len=split("7,9,10,11,12,13",E_Arr,",");
}
{	
	if($3 == "E")
	{
		for(i=1;i<=E_Len;i++)
		{
			col_num=E_Arr[i];
			sum+=$col_num;
		}
		if ( sum == 0)
		{
			print $0",Good";
		}
		else
		{
			print $0",Bad";
		}
	}
	if(length($3)>1)
	{
		sum=$11+$12+$13
		if (sum>0)
		{
			print $0",Good";
		}
		else
		{
			print $0",Bad";
		}
	}
}' ${INPUT_FILE}

you have to repeat the read color coding for other checks as well
---------- Post updated at 12:59 AM ---------- Previous update was at 12:58 AM ----------

@Don... Sorry for numerous updates . Trust me I was just trying to make it as clear as possible and I am done updating .. Sorry for inconvenience today.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Compare values in multiple rows in one column using awk

I would like to compare values in column 8, and grep the ones where the different is > 1, columns 1 and 2 are the key for array. Every 4 rows the records values in columns 1 and 2 changed. Then, the comparison in the column 8 need to be done for the 4 rows everytime columns 1 and 2 changed ... (4 Replies)
Discussion started by: jiam912
4 Replies

2. Shell Programming and Scripting

Compare two files column values using awk

Judi # cat File1 judi /export/home 76 judi /usr 83 judi # judi # cat File2 judi /export/home 79 judi /usr 82 judi # if COLUMN3 of File2 is greater that COLUMN3 of File1, then print File2's lines juid /export/home 79 Code tags please (2 Replies)
Discussion started by: judi
2 Replies

3. Shell Programming and Scripting

How to compare the values of a column in a same file using awk?

Dear Unix experts, I have got a file where I would like to compare the values of second column if first column is same in such a way that the difference between the values is >50. If not, I would like to discard both values. For example, my input file looks like - comp275_c0_seq2 73... (7 Replies)
Discussion started by: utritala
7 Replies

4. UNIX for Dummies Questions & Answers

Compare values of fields from same column with awk

Hi all ! If there is only one single value in a column (e.g. column 1 below), then return this value in the same output column. If there are several values in the same column (e.g. column 2 below), then return the different values separated by "," in the output. pipe-separated input: ... (11 Replies)
Discussion started by: lucasvs
11 Replies

5. Shell Programming and Scripting

Compare values in two files. For matching rows print corresponding values from File 1 in File2.

- I have two files (File 1 and File 2) and the contents of the files are mentioned below. - I am trying to compare the values of Column1 of File1 with Column1 of File2. If a match is found, print the corresponding value from Column2 of File1 in Column5 of File2. - I tried to modify and use... (10 Replies)
Discussion started by: Santoshbn
10 Replies

6. Shell Programming and Scripting

Take values from a column and put it in a variable and compare

Hi, I have a table in unix from which i want to read the contents line by line, then filter out the values from 6th column one by one and compare it a fixed value. How to do this? (7 Replies)
Discussion started by: arijitsaha
7 Replies

7. Shell Programming and Scripting

How to compare the values of a column in awk in a same file and consecutive lines..

I would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line. Input File ========== PDB 2500 RTDB 123 RTDB-EAGLE 122 VSCCP 2565... (4 Replies)
Discussion started by: manuswami
4 Replies

8. UNIX for Dummies Questions & Answers

Compare two files using awk or sed, add values in a column if their previous fields are same

Hi All, I have two files file1: abc,def,ghi,5,jkl,mno pqr,stu,ghi,10,vwx,xyz cba,ust,ihg,4,cdu,oqw file2: ravi,def,kishore ramu,ust,krishna joseph,stu,mike I need two output files as follows In my above example, each row in file1 has 6 fields and each row in file2 has 3... (1 Reply)
Discussion started by: yerruhari
1 Replies

9. UNIX for Advanced & Expert Users

Compare two files using awk or sed, add values in a column if their previous fields are same

Hi All, I have two files file1: abc,def,ghi,5,jkl,mno pqr,stu,ghi,10,vwx,xyz cba,ust,ihg,4,cdu,oqw file2: ravi,def,kishore ramu,ust,krishna joseph,stu,mike I need two output files as follows In my above example, each row in file1 has 6 fields and each row in file2 has 3... (1 Reply)
Discussion started by: yerruhari
1 Replies

10. Shell Programming and Scripting

I need to extract last column of a file and compare the values

Hi, I am new to unix and I need help in solving below mentioned issue, really appreciate ur help. I have a file sam, john, 2324, 07142007 tom, thomson, 2343, 07142007 john, scott, 2478, 07142007 its a comma delimited file, I need to extract the last column from each line and this... (4 Replies)
Discussion started by: vukkusila
4 Replies
Login or Register to Ask a Question