Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sum of three columns - in 4N columns file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sum of three columns - in 4N columns file
# 1  
Old 01-06-2010
Sum of three columns - in 4N columns file
Hi All,
happy new year.

I have a file with 4xN columns like

Code:
0.0000e+00	0.0000e+00	7.199E+07	7.123E+07	6.976E+07	6.482E+07	5.256E+07	2.523E+07
0.0000e+00	0.0000e+00	8.641E+07	8.550E+07	8.373E+07	7.780E+07	6.309E+07	3.028E+07
0.0000e+00	0.0000e+00	1.017E+08	1.007E+08	9.857E+07	9.159E+07	7.427E+07	3.565E+07
0.0000e+00	0.0000e+00	1.178E+08	1.166E+08	1.142E+08	1.061E+08	8.603E+07	4.130E+07
0.0000e+00	0.0000e+00	1.347E+08	1.333E+08	1.305E+08	1.213E+08	9.832E+07	4.719E+07
0.0000e+00	0.0000e+00	1.521E+08	1.505E+08	1.474E+08	1.370E+08	1.111E+08	5.331E+07

In this case N=2, but it can be what ever number.
I would like to replace
the 0's in column 1 with the sum of the column N+1,2N+1,3N+1
the 0's in column 2 with the sum of the column N+2,2N+2,3N+2
...
the 0's in column N with the sum of the column N+N,2N+N,3N+N

At the moment I'm using the following script which works fine if N=1

Code:
awk -F'\t' '
  (/^[0-9]/ && $1 == 0 && $1 = sprintf("%.4E",$2 + $3 + $4)) || -3
  ' OFS='\t' filename

Any idea how to genaralize the code?
Thank you,
Sarah
Last edited by f_o_555; 01-06-2010 at 11:58 AM..
# 2  
Old 01-06-2010
Try this and let me know it is what you want.use nawk or gawk
or /usr/xpg4/bin/awk

from your description above 4*N=number of columns (NF always even number) so N=NF/4


Code:
nawk '{ for (i=1;i<=NF;i++) { $i=$((NF/4)+i) + $(2*(NF/4)+i) + $(3*(NF/4)+i)printf "%.4E\t",$i}
print "" }'  infile.txt

SmilieSmilieSmilie
Last edited by ahmad.diab; 01-06-2010 at 12:28 PM..
# 3  
Old 01-06-2010
Hi, f_o_555:

I did my best to not make any assumptions regarding your assumptions (I was somewhat puzzled by the choice of negative three to trigger the printing of a record, but I kept it Smilie)

Code:
$ cat f_o_555.awk 
BEGIN { OFS="\t" }
/^[0-9]/ && (N=NF/4) && (i=NF+1) {
    delete sum
    while (--i)
        i>N ? (sum[i%N ? i%N : N]+=$i) : ($i=sprintf("%.4E", sum[i]))
}
-3

$ cat data3
0 0 0 1 2 3 1 2 3 1 2 3
0 0 0 1 2 3 1 2 3 1 2 3
0 0 0 1 2 3 1 2 3 1 2 3

$ awk -f f_o_555.awk data3
3.0000E+00      6.0000E+00      9.0000E+00      1       2       3       1      2       3       1       2       3
3.0000E+00      6.0000E+00      9.0000E+00      1       2       3       1      2       3       1       2       3
3.0000E+00      6.0000E+00      9.0000E+00      1       2       3       1      2       3       1       2       3

$ cat data4
0 0 0 0 1 2 3 4 1 2 3 4 1 2 3 4
0 0 0 0 1 2 3 4 1 2 3 4 1 2 3 4
0 0 0 0 1 2 3 4 1 2 3 4 1 2 3 4

$ awk -f f_o_555.awk data4
3.0000E+00      6.0000E+00      9.0000E+00      1.2000E+01      1       2      3      4       1       2       3       4       1       2       3       4
3.0000E+00      6.0000E+00      9.0000E+00      1.2000E+01      1       2      3      4       1       2       3       4       1       2       3       4
3.0000E+00      6.0000E+00      9.0000E+00      1.2000E+01      1       2      3      4       1       2       3       4       1       2       3       4

Regards,
alister




---------- Post updated at 03:01 PM ---------- Previous update was at 12:29 PM ----------

As i understand the problem, ahmad.diab's solution is incorrect but it did make me realize that i was over-engineering. A simpler approach inspired by his post (the only advantage of the modulus-based solution above is that it can readily handle MxN, and not just 4xN, if the hardcoded 4 is parameterized).

The following gives the same output given the same sample data files (data3 and data4) used above.

Code:
$ awk '/^[0-9]/ && $1==0 && (N=NF/4) { for (i=1;i<=N;i++) $i=sprintf("%.4E", $(N+i)+$(2*N+i)+$(3*N+i))} -3' OFS='\t' data

alister
Last edited by alister; 01-06-2010 at 04:18 PM.. Reason: perhaps i have ocd :)
# 4  
Old 01-06-2010
What's the meaning of " -3 " ?
# 5  
Old 01-06-2010
Quote:
Originally Posted by rdcwayx
What's the meaning of " -3 " ?
I don't know. heheh. In the original post's code, it triggers printing of a record that doesn't match the criteria which seem to identify records with data to process. I couldn't part with it.

Technically, any pattern expression that evaluates to non-zero/true will cause its corresponding action to execute. -3 evaluates to true and its corresponding action is absent and so it default to printing $0.
# 6  
Old 01-07-2010
Quote:
Originally Posted by alister
Hi, f_o_555:

As i understand the problem, ahmad.diab's solution is incorrect but it did make me realize that i was over-engineering. A simpler approach inspired by his post (the only advantage of the modulus-based solution above is that it can readily handle MxN, and not just 4xN, if the hardcoded 4 is parameterized).

The following gives the same output given the same sample data files (data3 and data4) used above.

Code:
$ awk '/^[0-9]/ && $1==0 && (N=NF/4) { for (i=1;i<=N;i++) $i=sprintf("%.4E", $(N+i)+$(2*N+i)+$(3*N+i))} -3' OFS='\t' data

alister
the number of columns is 4*N not only "N" as in alister code above and as f_o_555 mention in his post below.

Quote:
Originally Posted by alister
I have a file with 4xN columns like
So we need f_o_555 to decide what is the number of columns need to be proceed in the for loop ? is it N or 4*N (NF)

BR
SmilieSmilieSmilie
# 7  
Old 01-07-2010
Hello ahmad.diab:

The number of columns is 4*N. The problem statement says that they are divided into four groups. The only columns that will be modified to contain the sum of their counterparts are the first N. Your code is going beyond the first N columns and modifying ones it shouldn't.

Take care,
alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies

2. UNIX for Beginners Questions & Answers

Copy columns from one file into another and get sum of column values and row count

I have a file abc.csv, from which I need column 24(PurchaseOrder_TotalCost) to get the sum_of_amounts with date and row count into another file say output.csv abc.csv- UTF-8,,,,,,,,,,,,,,,,,,,,,,,,, ... (6 Replies)
Discussion started by: Tahir_M
6 Replies

3. UNIX for Beginners Questions & Answers

Group by columns and add sum in new columns

Dear Experts, I have input file which is comma separated, has 4 columns like below, BRAND,COUNTRY,MODEL,COUNT NIKE,USA,DUMMY,5 NIKE,USA,ORIGINAL,10 PUMA,FRANCE,DUMMY,20 PUMA,FRANCE,ORIGINAL,15 ADIDAS,ITALY,DUMMY,50 ADIDAS,ITALY,ORIGINAL,50 SPIKE,CHINA,DUMMY,1O And expected output add... (2 Replies)
Discussion started by: ricky1991
2 Replies

4. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ... (4 Replies)
Discussion started by: Khushbu
4 Replies

5. Shell Programming and Scripting

Get the SUM of TWO columns SEPARATELY by doing GROUP BY on other columns

My File looks like: "|" -> Field separator A|B|C|100|1000 D|E|F|1|2 G|H|I|0|7 D|E|F|1|2 A|B|C|10|10000 G|H|I|0|7 A|B|C|1|100 D|E|F|1|2 I need to do a SUM on Col. 5 and Col.6 by grouping on Col 1,2 & 3 My expected output is: A|B|C|111|11100 (2 Replies)
Discussion started by: machomaddy
2 Replies

6. Shell Programming and Scripting

Sum numeric columns contained in a plain text file

Hi everyone, Here are the contents of a plain text file created by a SQL query: SUM(T.TRNQTY) COUNT(D.TRNSEQ) ---------------- ---------------- 1380 46 1393 59 2680 134 740 37 ... (5 Replies)
Discussion started by: gacanepa
5 Replies

7. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

8. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

9. Shell Programming and Scripting

Sum of two columns in a file

Hi, I need to do a sum of two columns in a file where delimiter is |^ input 1|^2|^3|^4|^50|^2|^3|^100 2|^3|^4|^6|^100|^7|^2|^50 3|^4|^2|^3|^50|^6|^3|^50 4|^2|^5|^7|^25|^2|^8|^25 Output required: Sum of 2 columns 5 & 8 which is 450 (11 Replies)
Discussion started by: Jram
11 Replies

10. UNIX for Dummies Questions & Answers

Sum of all columns in all files in one output file

If I have say 4 files like this: File1: 1 3 4 7 7 0 5 7 5 9 1 2 7 4 8 File2: 1 4 6 2 5 7 1 2 3 6 0 3 0 3 8 File3: (5 Replies)
Discussion started by: cosmologist
5 Replies
Login or Register to Ask a Question