Read the data from multiple files and sum the value


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read the data from multiple files and sum the value
# 1  
Old 09-08-2008
Read the data from multiple files and sum the value

Hi all,

I have a requirement where i have to read multiple files using Shell Script in Korn Shell.

each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files.

any idea on how to achieve this?? (i think i can achieve it using a loop, getting the number files and using a loop for all the files and sum the amount)

any simpler approach than this??

the number of files to be read may vary.

highly appreciate your help.

Thanks,
Naren
# 2  
Old 09-08-2008
create a file with all of the filenames you need to access, call it filenames.txt
Code:
awk '{ if (FNR==3) {sum+=$0} } END {print sum} ' $(cat filenames.txt)

This assumes that the amount field is the whole line on line #3.
# 3  
Old 09-08-2008
Quote:
Originally Posted by jim mcnamara
create a file with all of the filenames you need to access, call it filenames.txt
Code:
awk '{ if (FNR==3) {sum+=$0} } END {print sum} ' $(cat filenames.txt)

This assumes that the amount field is the whole line on line #3.

It also assumes that there are no filenames with whitespace in filenames.txt.
# 4  
Old 09-09-2008
Quote:
Originally Posted by cfajohnson

It also assumes that there are no filenames with whitespace in filenames.txt.
one thing is number of files are not fixed and the 3rd line contains text as
"Total amount = 8792187481274"

i need to read the number part and sum it for all the files based on the pattern present in a the given directory.
# 5  
Old 09-09-2008
Quote:
one thing is number of files are not fixed and the 3rd line contains text as
"Total amount = 8792187481274"

i need to read the number part and sum it for all the files based on the pattern present in a the given directory.
Code:
awk 'FNR==3 {sum+=$4} END {print sum} ' /path/to/dir/PATTERN

# 6  
Old 09-10-2008
Quote:
Originally Posted by cfajohnson
Code:
awk 'FNR==3 {sum+=$4} END {print sum} ' /path/to/dir/PATTERN

how do we specify the delimiter here...from the code above i can make out that Space has been assumed to be the delimiter..

but if i have something other than space than how do i modify it??
# 7  
Old 09-10-2008

Give the delimiter as an argument to the -F option:
Code:
awk -F delim '....'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies

2. Shell Programming and Scripting

Shell script count lines and sum numbers from multiple files

I want to count the number of lines, I need this result be a number, and sum the last numeric column, I had done to make this one at time, but I need to make this for a crontab, so, it has to be an script, here is my lines: It counts the number of lines: egrep -i String file_name_201611* |... (5 Replies)
Discussion started by: Elly
5 Replies

3. Shell Programming and Scripting

Read multiple text files and copy data to csv

hi i need to extract lines from multiple files to a csv file. for example, i have these 3 files file1.txt date:29dec1980 caller:91245824255 called:8127766 file2.txt date:11apr2014 caller:9155584558 called:8115478 file3.txt date:25jun2015 caller:445225552 called:8117485 (30 Replies)
Discussion started by: lp.descamps
30 Replies

4. Shell Programming and Scripting

Sum of a column in multiple files

I am performing the following operation on a file that looks like this 1000 0 10 479.0 1115478.07497 0.0 0.0 0.0872665 1000 10 20 1500.0 3470012.29304 0.0 0.0 0.261799 1000 20 30 2442.0 5676346.87758 0.0 0.0 0.436332 1000 30 40 3378.0 7737905.30957 0.0 0.0 0.610865 1000 40 50 4131.0... (2 Replies)
Discussion started by: kayak
2 Replies

5. Shell Programming and Scripting

Read multiple files, parse data and append to a file

Hi..Can anyone suggest a simple way of achieving this. I have several files which ends with extension .vcf . I will give example with two files In the below files, we are interested in File 1: 38 107 C 3 T 6 C/T 38 241 C 4 T 5 C/T 38 247 T 4 C 5 T/C 38 259 T 3 C 6 T/C... (8 Replies)
Discussion started by: empyrean
8 Replies

6. Shell Programming and Scripting

sum numbers of multiple files

Hi, I want to count the number of occurrences of numbers from a file of 6,000,000 lines. Because its too large, I decided to split the counts up in multiple files. So I have files of the counts of 5,000 lines. Now I want to add up the counts of all those files. The "counts file" looks like... (9 Replies)
Discussion started by: linseyr
9 Replies

7. UNIX for Dummies Questions & Answers

need [HELP] sum array multiple files

Hi.. I'm very newbie here.. I wonder if somebody can help me.. I have multiple directories with same out file name for each directories.. ./dirA/out.dat ./dirB/out.dat ./dirC/out.dat ..and so on.. for ./dirA/out.dat here is the structure content : for ./dirB/out.dat the same... (4 Replies)
Discussion started by: agiantz
4 Replies

8. Shell Programming and Scripting

need [HELP] sum array multiple files

Hi.. I'm very newbie here.. I wonder if somebody can help me.. I have multiple directories with same out file name for each directories.. ./dirA/out.dat ./dirB/out.dat ./dirC/out.dat ..and so on.. for ./dirA/out.dat here is the structure content : for ./dirB/out.dat the same... (6 Replies)
Discussion started by: agiantz
6 Replies

9. Shell Programming and Scripting

sum numbers in multiple files

I have 11 directories with around 200 files in each. In each directory the files are labeled out.0 through out.201 . Each file has around 118 numbers in a single column. I need to sum the files in each directory so each directory will have a resultant vector that is 118 numbers long. I then... (5 Replies)
Discussion started by: pattywac
5 Replies

10. Shell Programming and Scripting

Trying to read data multiple times

I am developing a script to automate Global Mirroring on IBM DS8100's. Part of the process is to establish a global copy and wait until the paired LUN's Out of Sync tracks goes to zero. I can issue a command to display the ouput and am trying to use AWK to read the appropriate field. I am... (0 Replies)
Discussion started by: coachr
0 Replies
Login or Register to Ask a Question