Visit Our UNIX and Linux User Community


To perform sum aggregation on numeric fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To perform sum aggregation on numeric fields
# 1  
Old 04-23-2012
To perform sum aggregation on numeric fields

Hi all,
I have following scenario to perform sum aggregation on certain columns
Code:
Node           Allocated_Space   Pool_Name   CS_Group          Utilized Space
--------      ----------------   ----------   ---------           --------------
bdw1a_lun01   300                    bdw_p0      bdwprd_pool01      200              
bdw1a_lun02   400                    bdw_p1      bdwprd_pool01      200
bdw1a_lun03   300                    bdw_p2      bdwprd_pool01      130
bdw2a_lun04   500                    bdw_p3      bdwprd_pool01      230
bdw2a_lun05   500                    bdw_p4      bdwprd_pool01      300
bdw2a_lun06   300                    bdw_p5      bdwprd_pool01      240
dss1_lun01    100                                                    30 
dss2_lun02    200                                                   120
dss3_lun03    230                                                   200

expected output
Code:
Node           Allocated_Space   Pool_Name   CS_Group          Utilized Space
--------      ----------------   ----------   ---------           --------------
bdw1a_lun01   300                    bdw_p0      bdwprd_pool01      200              
bdw1a_lun02   400                    bdw_p1      bdwprd_pool01      200
bdw1a_lun03   300                    bdw_p2      bdwprd_pool01      130
bdw2a_lun04   500                    bdw_p3      bdwprd_pool01      230
bdw2a_lun05   500                    bdw_p4      bdwprd_pool01      300
bdw2a_lun06   300                    bdw_p5      bdwprd_pool01      240
            ------                                                 -----
             2300                                                  1300
            --------                                               ------
dss1_lun01    100                                                    30 
dss2_lun02    200                                                   120
dss3_lun03    230                                                   200
            -------                                                -------
             530                                                    350
            -------                                                --------

As you can see not all columns has value (it could be empty string).
I would like to sum only correct column value with the certain group of number of characters before "_lun".

Thanks in advance.

Last edited by Franklin52; 04-23-2012 at 06:40 AM.. Reason: Please use code tags for data en code samples
# 2  
Old 04-23-2012
Hi ckwan,

One way with perl. Format output to your needs:
Code:
$ cat infile
Node           Allocated_Space   Pool_Name   CS_Group          Utilized Space
--------      ----------------   ----------   ---------           --------------
bdw1a_lun01   300                    bdw_p0      bdwprd_pool01      200              
bdw1a_lun02   400                    bdw_p1      bdwprd_pool01      200
bdw1a_lun03   300                    bdw_p2      bdwprd_pool01      130
bdw2a_lun04   500                    bdw_p3      bdwprd_pool01      230
bdw2a_lun05   500                    bdw_p4      bdwprd_pool01      300
bdw2a_lun06   300                    bdw_p5      bdwprd_pool01      240
dss1_lun01    100                                                    30 
dss2_lun02    200                                                   120
dss3_lun03    230                                                   200
$ cat script.pl
use warnings;
use strict;

my ($last_node, $tot_alloc, $tot_used) = (undef, 0, 0);

while ( <> ) {

        if ( $. < 3 ) {
                print;
                next;
        }

        chomp;

        my ($node, $alloc, $used) = (split)[0,1,-1];

        if ( ! defined $last_node ) {
                $last_node = substr $node, 0, 3;
        }

        if ( eof ) {
                printf qq[%s\n], $_;
                $tot_alloc += $alloc;
                $tot_used  += $used;
                printf qq[\t\t-------\t\t\t\t-------\n];
                printf qq[\t\t%d\t\t\t\t%d\n], $tot_alloc, $tot_used;
                printf qq[\t\t-------\t\t\t\t-------\n];
                next;
        }


        if ( $last_node ne substr $node, 0, 3 ) {
                printf qq[\t\t-------\t\t\t\t-------\n];
                printf qq[\t\t%d\t\t\t\t%d\n], $tot_alloc, $tot_used;
                printf qq[\t\t-------\t\t\t\t-------\n];
                $tot_alloc = $tot_used = 0;
                $last_node = substr $node, 0, 3;
        }

        $tot_alloc += $alloc;
        $tot_used  += $used;

        printf qq[%s\n], $_;
}
$ perl script.pl infile
Node           Allocated_Space   Pool_Name   CS_Group          Utilized Space
--------      ----------------   ----------   ---------           --------------
bdw1a_lun01   300                    bdw_p0      bdwprd_pool01      200              
bdw1a_lun02   400                    bdw_p1      bdwprd_pool01      200
bdw1a_lun03   300                    bdw_p2      bdwprd_pool01      130
bdw2a_lun04   500                    bdw_p3      bdwprd_pool01      230
bdw2a_lun05   500                    bdw_p4      bdwprd_pool01      300
bdw2a_lun06   300                    bdw_p5      bdwprd_pool01      240
                -------                         -------
                2300                            1300
                -------                         -------
dss1_lun01    100                                                    30 
dss2_lun02    200                                                   120
dss3_lun03    230                                                   200
                -------                         -------
                530                             350
                -------                         -------

# 3  
Old 04-24-2012
Thanks for sharing birei.

Previous Thread | Next Thread
Test Your Knowledge in Computers #81
Difficulty: Easy
AT&T UNIX, SCO UnixWare, Sun Microsystems Solaris, HP-UX, IBM AIX, SGI IRIX and many big corporations who wanted license and profit from their own versions of Unix are not as common today as they were in the 1980s.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to update file with sum of matching fields in another file

In the awk below I am trying to add a penalty to a score to each matching $1 in file2 based on the sum of $3+$4 (variable TL) from file1. Then the $4 value in file1 is divided by TL and multiplied by 100 (this valvue is variable S). Finally, $2 in file2 - S gives the updated $2 result in file2.... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Sum numeric columns contained in a plain text file

Hi everyone, Here are the contents of a plain text file created by a SQL query: SUM(T.TRNQTY) COUNT(D.TRNSEQ) ---------------- ---------------- 1380 46 1393 59 2680 134 740 37 ... (5 Replies)
Discussion started by: gacanepa
5 Replies

3. Shell Programming and Scripting

Sum fields of different files using awk

I'm trying to sum each field of the second column over many different files. For example: file1: file2: 1 5 1 5 2 6 2 4 3 5 3 3 To get: file3 1 10 2 10 3 8 I found answer when there are only 2 files as... (10 Replies)
Discussion started by: rogeriog.em
10 Replies

4. Shell Programming and Scripting

Need help in finding sum for values in 2 different fields

Hi there, I have 2 files in following format cat file_1 Storage Group Name: aaaa HBA UID SP Name SPPort ------- ------- ------ 0 21 Storage Group Name: bbbb HBA UID... (2 Replies)
Discussion started by: jpkumar10
2 Replies

5. UNIX for Dummies Questions & Answers

Find and Replace random numeric value with non-numeric value

Can someone tell me how to change the first column in a very large 17k line file from a random 10 digit numeric value to a non numeric value. The format of lines in the file is: 1702938475,SNU022,201004 the first 10 numbers always begin with 170 (6 Replies)
Discussion started by: Bahf1s
6 Replies

6. Shell Programming and Scripting

Perform Hash sum

Hi Friends, I have a pipe delimited file with 20 million records. I want to perform the sum of the 13 field, for that I am using the code below, but the result is in exponential form. I want the number to be in floating number, so that I can use the number to comparision. Any help is highly... (3 Replies)
Discussion started by: anandapani
3 Replies

7. Shell Programming and Scripting

Check numeric fields greater than zero, and delete lines if appropriate

This be the latest in my problems sorting through router logs... I'm half way there on a problem, but I've hit the limitation of my knowledge Got some router interface log files of type router01:GigabitEthernet9/24 is up, line protocol is up (connected) router01: 0 input errors, 0 CRC, 0... (7 Replies)
Discussion started by: Yorkie99
7 Replies

8. Shell Programming and Scripting

calculating sum of fields in a file

Hey, I have a file and it has only one field. I need to calculate the sum of each filed as total. For e.g my file is 1 2 3 4 5 I need to calculate the total sum as 15. Please let me know how i can do it? (4 Replies)
Discussion started by: dsravan
4 Replies

9. Shell Programming and Scripting

Perl code to differentiate numeric and non-numeric input

Hi All, Is there any code in Perl which can differentiate between numeric and non-numeric input? (11 Replies)
Discussion started by: Raynon
11 Replies

10. Shell Programming and Scripting

sort sum fields

HI ALL, i have a problem when i do a sort sum with many fields. Is there a limit for fields? Do you know a solution? thanks in advance. the shell is: # SORT1 SORT1_rcode=777 if ; then echo "USE $DARSEQ/OTPU.FTPEPREC RECORD F,1000 " > $DARPARSRT/TPEKL508.SORT1_$$.srt ... (6 Replies)
Discussion started by: tullo
6 Replies

Featured Tech Videos