Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sum the columns of files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sum the columns of files
# 8  
Old 01-14-2011
I think it is sort that caused the problem, but I don't know how to display output in the same order as the input,
Code:
awk '{a[$1]+=$2;b[$1]+=$3;if(c<=$1);c=$1}END{for(i=i;i<=c;i++)print i"\t"a[i]"\t"b[i]}' *.csv

this code can do this without sort, but the time cost didn't decrease significantly, I want a faster one
# 9  
Old 01-14-2011
try this:
Code:
awk '!($1 in a) {key[++key[0]]=$1} {a[$1]+=$2;b[$1]+=$3}END{for(i=1;i<=key[0];i++) print key[i],a[key[i]],b[key[i]]}' OFS='\t' *csv

# 10  
Old 01-14-2011
Lose the ;
Code:
awk '{a[$1]+=$2;b[$1]+=$3;if(c<=$1)c=$1}END{for(i=i;i<=c;i++)print i"\t"a[i]"\t"b[i]}' *.csv

What about the suggestion to go parallel?
# 11  
Old 01-14-2011
Quote:
Originally Posted by vgersh99
try this:
Code:
awk '!($1 in a) {key[++key[0]]=$1} {a[$1]+=$2;b[$1]+=$3}END{for(i=1;i<=key[0];i++) print key[i],a[key[i]],b[key[i]]}' OFS='\t' *csv

thanks for your code,they need similar time cost

---------- Post updated at 09:38 AM ---------- Previous update was at 09:37 AM ----------

Quote:
Originally Posted by Scrutinizer
Lose the ;
Code:
awk '{a[$1]+=$2;b[$1]+=$3;if(c<=$1)c=$1}END{for(i=i;i<=c;i++)print i"\t"a[i]"\t"b[i]}' *.csv

What about the suggestion to go parallel?
thanks for your suggestion,but parallel is a little difficult for meSmilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Group by columns and add sum in new columns

Dear Experts, I have input file which is comma separated, has 4 columns like below, BRAND,COUNTRY,MODEL,COUNT NIKE,USA,DUMMY,5 NIKE,USA,ORIGINAL,10 PUMA,FRANCE,DUMMY,20 PUMA,FRANCE,ORIGINAL,15 ADIDAS,ITALY,DUMMY,50 ADIDAS,ITALY,ORIGINAL,50 SPIKE,CHINA,DUMMY,1O And expected output add... (2 Replies)
Discussion started by: ricky1991
2 Replies

2. Shell Programming and Scripting

Sum of Columns

HI Guys, I gave Input file F.Txt ID H1 H2 H3 H4 H5 A 5 6 7 8 9 B 4 65 4 4 7 C 4 4 4 4 4 D 4 4 4 4 4 Output :- ID H1 H2 H3 H4 H5 Total 17 79 19 20 24 Sum of Each Columns (8 Replies)
Discussion started by: pareshkp
8 Replies

3. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ... (4 Replies)
Discussion started by: Khushbu
4 Replies

4. Shell Programming and Scripting

Get the SUM of TWO columns SEPARATELY by doing GROUP BY on other columns

My File looks like: "|" -> Field separator A|B|C|100|1000 D|E|F|1|2 G|H|I|0|7 D|E|F|1|2 A|B|C|10|10000 G|H|I|0|7 A|B|C|1|100 D|E|F|1|2 I need to do a SUM on Col. 5 and Col.6 by grouping on Col 1,2 & 3 My expected output is: A|B|C|111|11100 (2 Replies)
Discussion started by: machomaddy
2 Replies

5. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

6. Shell Programming and Scripting

Sum columns

Hi All, I'm new to this forum. So please be patience with me! :) I have a file that looks like this (all rows have the same number of columns): 19 20 30 15 17 38 51 60 74 85 96 07 .... 10 20 44 59 39 88 13 77 30 10 11 12 .... . . . I want to sum the value of first field to all the... (2 Replies)
Discussion started by: Aderson Nascime
2 Replies

7. Shell Programming and Scripting

Sum up values of columns in 4 files using shell script

I am new to shell script.I have records like below in 4 different files which have about 10000 records each, all records unique and sorted based on column 2. 1 2 3 4 5 6 --------------------------- SR|1010478|000044590|1|0|0| SR|1014759|000105790|1|0|0| SR|1016609|000108901|1|0|0|... (2 Replies)
Discussion started by: reach.sree@gmai
2 Replies

8. UNIX for Dummies Questions & Answers

Sum of all columns in all files in one output file

If I have say 4 files like this: File1: 1 3 4 7 7 0 5 7 5 9 1 2 7 4 8 File2: 1 4 6 2 5 7 1 2 3 6 0 3 0 3 8 File3: (5 Replies)
Discussion started by: cosmologist
5 Replies

9. Shell Programming and Scripting

Sum of three columns - in 4N columns file

Hi All, happy new year. I have a file with 4xN columns like 0.0000e+00 0.0000e+00 7.199E+07 7.123E+07 6.976E+07 6.482E+07 5.256E+07 2.523E+07 0.0000e+00 0.0000e+00 8.641E+07 8.550E+07 8.373E+07 7.780E+07 6.309E+07 3.028E+07... (8 Replies)
Discussion started by: f_o_555
8 Replies

10. Shell Programming and Scripting

sum of three columns

Hi All, I have like this M17XX-050-01 0100000000 QQSSS 0.0000e+00 1.712E+06 1.255E+07 0.0000e+00 0.0000e+00 1.722E+06 1.263E+07 0.0000e+00 ... 0.0000e+00 1.204E+06 8.829E+06 0.0000e+00 M17XX-050-01 0100000000 WWSSS 0.0000e+00 7.564E+03 1.165E+01 0.0000e+00... (6 Replies)
Discussion started by: f_o_555
6 Replies
Login or Register to Ask a Question