Sum column values matching other field


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sum column values matching other field
# 1  
Old 04-14-2015
Sum column values matching other field

this is part of a KT i am going thru.
i am writing a script in bash shell, linux where i have 2 columns where 1st signifies the nth hour like 00, 01, 02...23 and 2nd the file size.

sample data attached.
Desired output is 3 columns which will give the nth hour, number of entries in nth hour and sum of the file sizes on nth hour.

i am trying below code but it's not working.

HTML Code:
for (( i=00; i<=23; i++ ))
  do
      filecount=`awk '{if ($1 == "$i") print $2;}' interim.txt |wc -l`
      filesize=`awk '{if ($1 == "$i") print $2;}' interim.txt | awk '{sum += $1} END {print sum}'|awk '{filesize=$1/1024/1024; print filesize"MB"}'`
      echo $i $filecount $filesize >> report.txt
   done
any help would be greatly appreciated.

ty.

Last edited by alpha_1; 04-14-2015 at 10:02 PM..
# 2  
Old 04-14-2015
try

Code:
awk '{arr[$1]+=$2;arr2[$1]++} END {for (i in arr) {print i,arr[i],arr2[i]}}' file

# 3  
Old 04-14-2015
Thanks a lot for this.
it is generating output but not is sorted order and file sizes are not in MB.
Also I need the hours where there is no information.

for example:

HTML Code:
00 21 1.2MB
01 0   0
02 41 5.41MB
03 0   0
.
.
.
23.......
# 4  
Old 04-15-2015
As you didn't mention the units of your file sizes, is reasonably difficult to guess what they are, and even more to output in MB.
Anyhow, try
Code:
awk     '
NR>1 &&
!($1 in CNT)    {print last, CNT[last], SIZ[last]/1048576}
                {SIZ[$1]+=$2
                 CNT[$1]++
                 last=$1
                }
END             {print last, CNT[last], SIZ[last]/1048576}
        ' /tmp/sample.txt
01 7 0.0996408
02 3 0.0981874
.
.
.
23 1 0.0107212

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sum the values in the column using date column

I have a file which need to be summed up using date column. I/P: 2017/01/01 a 10 2017/01/01 b 20 2017/01/01 c 40 2017/01/01 a 60 2017/01/01 b 50 2017/01/01 c 40 2017/01/01 a 20 2017/01/01 b 30 2017/01/01 c 40 2017/02/01 a 10 2017/02/01 b 20 2017/02/01 c 30 2017/02/01 a 10... (6 Replies)
Discussion started by: Booo
6 Replies

2. UNIX for Dummies Questions & Answers

Match sum of values in each column with the corresponding column value present in trailer record

Hi All, I have a requirement where I need to find sum of values from column D through O present in a CSV file and check whether the sum of each Individual column matches with the value present for that corresponding column present in the trailer record. For example, let's assume for column D... (9 Replies)
Discussion started by: tpk
9 Replies

3. Shell Programming and Scripting

Sum column values based in common identifier in 1st column.

Hi, I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column) The input is for example, after sorted: K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies

4. Shell Programming and Scripting

How to find sum of any 'n' number of values from file matching target value?

I have a simple text file having payment amount value on each line. At the end of day 'n' number of payments created difference in amount that I need to match from this file. I have information about how many payments created difference and difference amount. Please help me to build shell... (3 Replies)
Discussion started by: swats007
3 Replies

5. UNIX for Dummies Questions & Answers

awk to sum column field from duplicate row/lines

Hello, I am new to Linux environment , I working on Linux script which should send auto email based on the specific condition from log file. Below is the sample log file Name m/c usage abc xxx 10 abc xxx 20 abc xxx 5 xyz ... (6 Replies)
Discussion started by: asjaiswal
6 Replies

6. Shell Programming and Scripting

Sum up the column values group by using some field

12-11-2012,PNL,158406 12-11-2012,RISK,4564 12-11-2012,VAR_1D,310101 12-11-2012,VAR_10D,310101 12-11-2012,CB,866 12-11-2012,STR_VAR_1D,298494 12-11-2012,STR_VAR_10D,309623 09-11-2012,PNL,1024106 09-11-2012,RISK,4565 09-11-2012,VAR_1D,317211 09-11-2012,VAR_10D,317211 09-11-2012,CB,985... (7 Replies)
Discussion started by: manas_ranjan
7 Replies

7. Shell Programming and Scripting

Sum of column matching pattern/string

Hi All, I have the following output file created as a result of one script called pattern_daily.log $ cat pattern_daily.log Approved|2|Wed, Oct 24, 2012 11:21:09 AM Declined|1|Wed, Oct 24, 2012 11:21:15 AM Approved|2|Wed, Oct 24, 2012 11:24:08 AM Declined|1|Wed, Oct 24, 2012 11:24:18 AM... (4 Replies)
Discussion started by: Gem_In_I
4 Replies

8. Shell Programming and Scripting

Getting a sum of column values

I have a file in the following layout: 201008005946873001846130058030701006131840000000000000000000 201008006784994001154259058033001009527844000000000000000000 201008007323067002418095058034801002418095000000000000000000 201008007697126001722141058029101002214158000000000000000000... (2 Replies)
Discussion started by: jclanc8
2 Replies

9. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

10. Shell Programming and Scripting

How to sum column 1 values

I have a file file like this. I want to sum all column 1 values. input A 2 A 3 A 4 B 4 B 2 Out put A 9 B 6 (3 Replies)
Discussion started by: suresh3566
3 Replies
Login or Register to Ask a Question