calculating column summation in a directory of flat files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting calculating column summation in a directory of flat files
# 1  
Old 10-11-2011
calculating column summation in a directory of flat files

Hello Guru s


I need your kind help to solve my below issue

I have a directory of flat files and have to calculate sum of some columns from the flat file .

Say for flat file 302 I need the column summation of 2 and 3 rd column
For flat file 303 I need the column summation of 5 and 6th column

Do you guys have any thought which logic will be the best to solve it
Can I store the flat file name and column in separate flat file and will calculate the sum via reading that file

The flat file separator is | (pipe)
Its kinda urgent .

Thanks for investing your thought and time
# 2  
Old 10-11-2011
Code:
$ nawk -F\| '{sum+=$2} {add+=$3} END {print sum,add}' file_302

# 3  
Old 10-11-2011
You could continue the original thread.

Create a file with the entries like..

filename,col1,col2

Code:
$ cat stat 
f1,2,3
f2,5,6
$ 
$
$ while IFS="," read fname cname1 cname2
> do
> awk -F\| -v c1=$cname1 -v c2=$cname2 '{sum1+=$c1;sum2+=$c2} END {print sum1 FS sum2}' $fname
> done < stat
8|12
20|24
$


Last edited by clx; 10-11-2011 at 06:03 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk/sed summation of one column based on some entry in first column

Hi All , I am having an input file as stated below Input file 6 ddk/djhdj/djhdj/Q 10 0.5 dhd/jdjd.djd.nd/QB 01 0.5 hdhd/jd/jd/jdj/Q 10 0.5 512 hd/hdh/gdh/Q 01 0.5 jdjd/jd/ud/j/QB 10 0.5 HD/jsj/djd/Q 01 0.5 71 hdh/jjd/dj/jd/Q 10 0.5 ... (5 Replies)
Discussion started by: kshitij
5 Replies

2. Shell Programming and Scripting

Summation of a particular column which are in different folders

I have three folders named f1, f2 and f3. The file names are same in all these folders. I would like to get the summation of the second column from all files. How can I do this ? file1 in f1 folder 143 143.69 60.2 81.30 40.4 62.39 166.3 83.38 107.2 60.30 37.5 144 192.62 107.9... (3 Replies)
Discussion started by: edisonjoe
3 Replies

3. Shell Programming and Scripting

Fastest way calculating directory

Hi expert, Is there any fastest way to calculate recursive directory, and I have total 600 directories have 100000 files and 10 directory approximately 9000000 - 10000000 each files per directory. currently using this command "du -k --max-depth=0" to get the size but very slow it take 24 hours... (9 Replies)
Discussion started by: rufino
9 Replies

4. Shell Programming and Scripting

How to Calculating space used in GB for any particular directory in UNIX?

How to Calculating space used in GB for any particular directory in unix Currently I am using : df -h which gives me space for each mout point ldndyn1:/vol/v01/dyn/sbcexp/dyn 1.1T 999G 29G 98% /sbcimp/dyn but I need for some internal particular directory... (3 Replies)
Discussion started by: RahulJoshi
3 Replies

5. HP-UX

add new column by calculating percentage used

hi.. DS_ID | PRIMARYDISKID | SPUID | GB_USED | GB_SIZE -------+---------------+-------+---------------+------------ 1 | 1027 | 1107 | 82.5849609375 | 356.203125 2 | 1048 | 1107 | 77.1767578125 | 356.203125 this is file having GB_USED , GB_SIZE two... (4 Replies)
Discussion started by: netdbaind
4 Replies

6. Shell Programming and Scripting

Summation of column value in flat file

Hello Guys Please find my below requirement I have a flat file with column headers in first line and data The structure like below col1 col2 col3 A 1 2 B 3 4 C 5 6 Say I have to take the summation of col2 (that will depend on the... (2 Replies)
Discussion started by: Pratik4891
2 Replies

7. Shell Programming and Scripting

Summation of Columns accross multiple files

I have around 1000 files in a directory (file1.txt, file2.txt, ... file1000.txt), each file is 2 columns, the first colums are same for all of them, second one is different. file1.txt 1 10 2 20 3 30 4 40 5 50 file2.txt 1 100 2 400 3 500 4 600 5 900 what I need is , I want... (3 Replies)
Discussion started by: scriptie
3 Replies

8. UNIX for Advanced & Expert Users

Summation of the column value in an interval.

Hey, I have a file with following data.. and I want to get the summation of 10th column. And this happens in every 10 seconds for a minute max. 242001 A mqsitst 794864 1249516 0 60 20 293050400 77016 * 02:05:33 - 0:04 DataFlowEngine EAITIBR2_BROKER... (6 Replies)
Discussion started by: varungupta
6 Replies

9. UNIX for Dummies Questions & Answers

Calculating the Standard Deviation for a column

Hi all, I want to calculate the standard deviation for a column (happens to be column 3). Does any know of simple awk script to do this? Thanks (1 Reply)
Discussion started by: kylle345
1 Replies

10. Shell Programming and Scripting

Column names in flat files

Hi all, I want to create column names in a flat file and then load the data through some other application. For example, I have a file with emp.txt and I need column names as eno,ename,sal in the first line. The delimiter here is comma and record delimiter is end of line or unix new line. Could... (1 Reply)
Discussion started by: srivsn
1 Replies
Login or Register to Ask a Question