need [HELP] sum array multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting need [HELP] sum array multiple files
# 1  
Old 06-18-2011
need [HELP] sum array multiple files

Hi.. I'm very newbie here..

I wonder if somebody can help me..

I have multiple directories with same out file name for each directories..

./dirA/out.dat
./dirB/out.dat
./dirC/out.dat
..and so on..

for ./dirA/out.dat here is the structure content :
Quote:
1 10 20 30 40
2 50 60 50 10
3 ..
for ./dirB/out.dat the same structure but different data :
Quote:
1 11 23 40 30
2 20 30 10 50
3 ..
and so on for other files..

What I want to do is to sum the data in the same cell for every files..

so for example above my expected output file is :
Quote:
1 21 43 70 70
2 70 90 60 60
3 ...
How can I make the script using awk ..? or I should make fortran code..? I have no idea to do both.. Smilie

Thank you in advance..

Best Regards..
--- agiantz
# 2  
Old 06-18-2011
which data to be added..? i mean "What SUM" you need?

21+ 43+ 70+ 70 = *

or 21 + 70 = * ???
# 3  
Old 06-18-2011
Quote:
Originally Posted by linuxadmin
which data to be added..? i mean "What SUM" you need?

21+ 43+ 70+ 70 = *

or 21 + 70 = * ???
the last example is the result that I want..
21 43 70 70 is the result

I want to sum from several files that have same dimension,
example :
file 1 :
1 10 20 30 40
2 50 60 50 10
3 ... (first coulomb just number, not the data)
+ file 2 :
1 11 23 40 30
2 20 30 10 50
3 .. (and so on with other files..)

so i will have output file like this :
1 21 43 70 70
2 70 90 60 60
3 ...
(where 21 is sum from 10 in the first file with 11 in 2nd file..)
the position for every cell is the same..
# 4  
Old 06-18-2011
Browse before posting.

I googled like this:

shell get sum from two files

and i got a closed post on 07-21-08.

https://www.unix.com/shell-programmin...ple-files.html

.
# 5  
Old 06-18-2011
Try this little Perl script:
Code:
#!/usr/bin/perl
chomp(@files=`find . -name "out.dat"`);
for $f (@files){
  open F, "$f";
  chomp(@rows=<F>);
  for $r (0..$#rows){
    @cells=split / /,$rows[$r];
    for $c (0..$#cells){
      $sum[$r][$c]+=$cells[$c] if $c>0;
      $sum[$r][$c]=$cells[$c] if $c==0;
    }
  }
}
for $r (0..$#rows){
  print "@{$sum[$r]}\n";
}

Save it in the directory containing dirA, dirB etc. Then run it without arguments: ./script.pl
This User Gave Thanks to bartus11 For This Post:
# 6  
Old 06-18-2011
Quote:
Originally Posted by linuxadmin
Browse before posting.

I googled like this:

shell get sum from two files

and i got a closed post on 07-21-08.

...

.
I already try that, but the problem is different because I need sum only for every cells in same dimension..

Quote:
Originally Posted by bartus11
Try this little Perl script:
Code:
#!/usr/bin/perl
chomp(@files=`find . -name "out.dat"`);
for $f (@files){
  open F, "$f";
  chomp(@rows=<F>);
  for $r (0..$#rows){
    @cells=split / /,$rows[$r];
    for $c (0..$#cells){
      $sum[$r][$c]+=$cells[$c] if $c>0;
      $sum[$r][$c]=$cells[$c] if $c==0;
    }
  }
}
for $r (0..$#rows){
  print "@{$sum[$r]}\n";
}

Save it in the directory containing dirA, dirB etc. Then run it without arguments: ./script.pl
You are my savior.. Smilie
Thank you very much.. it's work like magic.. ^_^

Save my time for my hundreds of data....

Arigatou gozaimasu...
# 7  
Old 06-18-2011
Code:
awk '{i=1;A[FNR":"i]=$1
        do {x=FNR":"(++i)
                A[x]+=$i
        }while(i<NF)
}END{f=1;do {
        k=1;do{
                y=(y?y OFS:z) A[f":"k]
        }while(++k<=NF)
        print y;y=z
}while(f++<FNR)}' dir*/out.dat

---------- Post updated at 10:50 AM ---------- Previous update was at 10:45 AM ----------

Code:
$ cat dirA/out.dat
1 10 20 30 40
2 50 60 50 10
$ cat dirB/out.dat
1 11 23 40 30
2 20 30 10 50
$ cat my.awk
{i=1;A[FNR":"i]=$1
        do {x=FNR":"(++i)
                A[x]+=$i
        }while(i<NF)
}END{f=1;do {
        k=1;do{
                y=(y?y OFS:z) A[f":"k]
        }while(++k<=NF)
        print y;y=z
}while(f++<FNR)}
$ awk -f my.awk OFS="\t" dir*/out.dat
1       21      43      70      70
2       70      90      60      60
$

You could even choose the Output Field Separator

Last edited by ctsgnb; 06-18-2011 at 07:10 AM.. Reason: indentation ...
This User Gave Thanks to ctsgnb For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script count lines and sum numbers from multiple files

I want to count the number of lines, I need this result be a number, and sum the last numeric column, I had done to make this one at time, but I need to make this for a crontab, so, it has to be an script, here is my lines: It counts the number of lines: egrep -i String file_name_201611* |... (5 Replies)
Discussion started by: Elly
5 Replies

2. Shell Programming and Scripting

Help reading the array and sum of the array elements

Hi All, need help with reading the array and sum of the array elements. given an array of integers of size N . You need to print the sum of the elements in the array, keeping in mind that some of those integers may be quite large. Input Format The first line of the input consists of an... (1 Reply)
Discussion started by: nishantrefound
1 Replies

3. Shell Programming and Scripting

Need to sum up a column value from multiple files into Credit and Debit categories using awk command

i have multiple files with Batch Header, Record detail & Batch trailer data in the files like : BH 20150225950050N8262 RD 20140918000000 99999999 unk Deferred Settlement -13950 BT01 -13950 *Above sample data donot have the spaces coorectly defined. I do have multiple batch trailer... (1 Reply)
Discussion started by: kcdg859
1 Replies

4. Shell Programming and Scripting

Sum values of specific column in multiple files, considering ranges defined in another file

I have a file (let say file B) like this: File B: A1 3 5 A1 7 9 A2 2 5 A3 1 3 The first column defines a filename and the other two define a range in that specific file. In the same directory, I have also three more files (File A1, A2 and A3). Here is 10 sample lines... (3 Replies)
Discussion started by: Bastami
3 Replies

5. Shell Programming and Scripting

Bash 3d associative array with bash3 AND multiple files

Hello again guru’s (big apologies for wall of text) Still working on that DNS updater for my production team and while there is a ton of hit in searches i can't seem to find the answer to this. Context: We have apps that switch from let’s say host1 to host2. REAL basic DNS clustering... (5 Replies)
Discussion started by: maverick72
5 Replies

6. Shell Programming and Scripting

Sum of a column in multiple files

I am performing the following operation on a file that looks like this 1000 0 10 479.0 1115478.07497 0.0 0.0 0.0872665 1000 10 20 1500.0 3470012.29304 0.0 0.0 0.261799 1000 20 30 2442.0 5676346.87758 0.0 0.0 0.436332 1000 30 40 3378.0 7737905.30957 0.0 0.0 0.610865 1000 40 50 4131.0... (2 Replies)
Discussion started by: kayak
2 Replies

7. Shell Programming and Scripting

sum numbers of multiple files

Hi, I want to count the number of occurrences of numbers from a file of 6,000,000 lines. Because its too large, I decided to split the counts up in multiple files. So I have files of the counts of 5,000 lines. Now I want to add up the counts of all those files. The "counts file" looks like... (9 Replies)
Discussion started by: linseyr
9 Replies

8. UNIX for Dummies Questions & Answers

need [HELP] sum array multiple files

Hi.. I'm very newbie here.. I wonder if somebody can help me.. I have multiple directories with same out file name for each directories.. ./dirA/out.dat ./dirB/out.dat ./dirC/out.dat ..and so on.. for ./dirA/out.dat here is the structure content : for ./dirB/out.dat the same... (4 Replies)
Discussion started by: agiantz
4 Replies

9. Shell Programming and Scripting

Read the data from multiple files and sum the value

Hi all, I have a requirement where i have to read multiple files using Shell Script in Korn Shell. each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files. any idea on how to achieve this?? (i think i can achieve it using a loop,... (9 Replies)
Discussion started by: nvuradi
9 Replies

10. Shell Programming and Scripting

sum numbers in multiple files

I have 11 directories with around 200 files in each. In each directory the files are labeled out.0 through out.201 . Each file has around 118 numbers in a single column. I need to sum the files in each directory so each directory will have a resultant vector that is 118 numbers long. I then... (5 Replies)
Discussion started by: pattywac
5 Replies
Login or Register to Ask a Question