Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sum numbers in multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sum numbers in multiple files
# 1  
Old 07-21-2008
sum numbers in multiple files
I have 11 directories with around 200 files in each. In each directory the files are labeled out.0 through out.201 . Each file has around 118 numbers in a single column. I need to sum the files in each directory so each directory will have a resultant vector that is 118 numbers long. I then need to combine the vectors into a single file that is 11*118 (1298) numbers long. Someone mentioned using 'cat' but since i only wrote my first csh script over the weekend I have no idea where to start.

thanks in advance
# 2  
Old 07-21-2008
Question Can you provide more info / example?
Code:
DirA
 out.0
  contains 118 rows, each with a number
 out.1
  contains 118 rows, each with a number
 ...
DirB
 repeats

Fairly straight-forward to hav a loop add up all of the numbers.
But, not sure what you mean by resulting vector.
# 3  
Old 07-21-2008
you have the directory structure right. Im not trying to sum up each number within the file. Within each dir all 200 of the first entries would be summed, same for the second ..... through the 118th entry of each file. This would result in each directory having a summed column that is 118 numbers long. I then need to combine those numbers from each dir into one file so it would contain the 118 numbers * 11 dir = 1298 numbers in the final data file.

so inside a dir:

out.1:
0
5
10

out.2
0
5
100

sum=
0
10
110
# 4  
Old 07-21-2008
Question continuing with my last notes
Code:
DirA
 out.0
  contains 118 rows, each with a number
 out.1
  contains 118 rows, each with a number
 ...
DirB
 repeats

Code:
inside DirA
 open out.0
  add up all rows in the file, get a number 31543
  write that to a temp file call DirA_sums
 open out.1
  add up all rows in the file, get a number 19548
  write that to a temp file call DirA_sums
  DirA_sums now contains 31543 19548
 continue through all of the files
inside DirB
 process the out.? files in same manner

continue through DirK (11th directory)

After the above, I would have eleven files, each containing 118 numbers (which are the sums from the files within the directory)

Do I got that right?
# 5  
Old 07-21-2008
not quite.....
should be more like this:

Code:
inside DirA
 open out.0
  store all rows in the file to array
 open out.1
  store all rows in the file to temparray
  set array = array + temparray
 open out.2
  store all rows in the file to temparray
  set array = array + temparray


 continue through all of the files
 Write array to tempfile out.DirA

inside DirB
 process the out.0-200 files in same manner

continue through DirK (11th directory)

after 11th directory
Write out.DirA to finaloutput.file
append out.DirB to finaloutput.file
...
append out.DirK to finaloutput.file

# 6  
Old 07-21-2008
Tools starting to code, if understand your question
given a sample structure (2 dirs, 4 files in each)
each file has five numbers in it, one per line
./DirA/out.0
./DirA/out.1
./DirA/out.2
./DirA/out.3
./DirB/out.0
./DirB/out.1
./DirB/out.2
./DirB/out.3

Start of the script programming
Code:
> cat manip  
#! /usr/bin/bash

find . -name out* | sort >file_list

rm *.arr 2>/dev/null
while read zf
  do
  echo $zf
  tfile=$(echo $zf | cut -d "/" -f2)
  arr_txt=$(cat $zf | tr "\n" " ")
  echo "$arr_txt" >>"$tfile".arr
done <file_list

program execution
Code:
> manip
./DirA/out.0
./DirA/out.1
./DirA/out.2
./DirA/out.3
./DirB/out.0
./DirB/out.1
./DirB/out.2
./DirB/out.3

see the resultant two files created
Code:
> cat DirA.arr
2 4 6 8 10  
1 3 5 7 9  
21 22 31 32 33  
12 13 14 15 16  
> cat DirB.arr
3 4 5 6 7  
6 7 19 10 11  
5 10 9 6 14  
19 20 2 3 10

I know that the files were read properly, to create each line in my .arr files.

So, if this looks to be on-course, it is fairly straight-forward to combine my .arr files into one larger file. So...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script count lines and sum numbers from multiple files

I want to count the number of lines, I need this result be a number, and sum the last numeric column, I had done to make this one at time, but I need to make this for a crontab, so, it has to be an script, here is my lines: It counts the number of lines: egrep -i String file_name_201611* |... (5 Replies)
Discussion started by: Elly
5 Replies

2. Shell Programming and Scripting

Need to sum up a column value from multiple files into Credit and Debit categories using awk command

i have multiple files with Batch Header, Record detail & Batch trailer data in the files like : BH 20150225950050N8262 RD 20140918000000 99999999 unk Deferred Settlement -13950 BT01 -13950 *Above sample data donot have the spaces coorectly defined. I do have multiple batch trailer... (1 Reply)
Discussion started by: kcdg859
1 Replies

3. Shell Programming and Scripting

Sum of numbers in three or more files

I have files : cat file1 15 88 44 667 33 4cat file2 445 66 77 3 56 (12 Replies)
Discussion started by: Natalie
12 Replies

4. Shell Programming and Scripting

Sum of a column in multiple files

I am performing the following operation on a file that looks like this 1000 0 10 479.0 1115478.07497 0.0 0.0 0.0872665 1000 10 20 1500.0 3470012.29304 0.0 0.0 0.261799 1000 20 30 2442.0 5676346.87758 0.0 0.0 0.436332 1000 30 40 3378.0 7737905.30957 0.0 0.0 0.610865 1000 40 50 4131.0... (2 Replies)
Discussion started by: kayak
2 Replies

5. Shell Programming and Scripting

sum numbers of multiple files

Hi, I want to count the number of occurrences of numbers from a file of 6,000,000 lines. Because its too large, I decided to split the counts up in multiple files. So I have files of the counts of 5,000 lines. Now I want to add up the counts of all those files. The "counts file" looks like... (9 Replies)
Discussion started by: linseyr
9 Replies

6. UNIX for Dummies Questions & Answers

need [HELP] sum array multiple files

Hi.. I'm very newbie here.. I wonder if somebody can help me.. I have multiple directories with same out file name for each directories.. ./dirA/out.dat ./dirB/out.dat ./dirC/out.dat ..and so on.. for ./dirA/out.dat here is the structure content : for ./dirB/out.dat the same... (4 Replies)
Discussion started by: agiantz
4 Replies

7. Shell Programming and Scripting

need [HELP] sum array multiple files

Hi.. I'm very newbie here.. I wonder if somebody can help me.. I have multiple directories with same out file name for each directories.. ./dirA/out.dat ./dirB/out.dat ./dirC/out.dat ..and so on.. for ./dirA/out.dat here is the structure content : for ./dirB/out.dat the same... (6 Replies)
Discussion started by: agiantz
6 Replies

8. Shell Programming and Scripting

Sum Numbers from different files

Hi All, I need to print the sum of numbers from different files. Input files: file1.out 10 20 30 file2.out 10 20 30 (5 Replies)
Discussion started by: saint2006
5 Replies

9. Shell Programming and Scripting

Read the data from multiple files and sum the value

Hi all, I have a requirement where i have to read multiple files using Shell Script in Korn Shell. each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files. any idea on how to achieve this?? (i think i can achieve it using a loop,... (9 Replies)
Discussion started by: nvuradi
9 Replies

10. Shell Programming and Scripting

how to find a sum of multiple numbers

I have a command which returns some numbers as follows: $ls -l ${dbname}.ix* | awk '{print $5 }' 929792 36864 57344 73728 53248 114688 How can I find the sum of those numbers by piping this output into 'awk' or some other editor/command? Thanks a lot -A (3 Replies)
Discussion started by: aoussenko
3 Replies
Login or Register to Ask a Question