Breaking the sum of a column when specific pattern is found


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Breaking the sum of a column when specific pattern is found
# 1  
Old 05-20-2013
Breaking the sum of a column when specific pattern is found

Hi

Am trying to sum up a file
Code:
# cat /Out
maths                   2            0.0          
english                2091            0.2          
history               -1            0.0           
physics                18            0.0
--------------------------------------
maths                   2            0.0          
english                2091            0.2          
history               -1            0.0           
physics                18            0.0
-------------------------------------
maths                   2            0.0          
english                2091            0.2          
history               -1            0.0           
physics                18            0.0

Am trying to sum up only the second column of the file
I need to sum up like, When "-------------" is met the values that had sum up so far should be stored in another variable and then a new sum from zero has to be proceeded until again "-------------------" is met and store the value in another new variable

I reached till
Code:
we=`awk '{print $2}'  /Out |grep '[0-9]'|awk '{sum+=$1} END {print sum}'`

But this gives the sum of entire column..
Please help..
# 2  
Old 05-20-2013
Can you try this,

Code:
awk '{print $2}' /Out | grep '[0-9]' | awk '{RS="----"} {sum+=$1} END {print sum}'

Thanks
Latika

Last edited by Scrutinizer; 05-20-2013 at 08:26 AM..
# 3  
Old 05-20-2013
So, do you need the result in some kind of shell script? Then you would probably be better off with a shell loop and adding an resetting the totals inside the loop..
Code:
while read subject col2 col3
do
  case $subject in 
    --*) printf "%d\n" "$total"
         # Here some further processing with $total
         # ...
         # reset counter
         total=0 ;;
      *) total=$(( total + col2 ))
  esac
done < file
printf "%d\n" "$total"

# 4  
Old 05-20-2013
I tried the following code
Code:
# awk '{print $2}' /out | grep '[0-9]' | awk '{RS="----"} {sum+=$1} END {print sum}'
2094

This prints the last sum value between "---------" and "-------------" i need to print all sum values between "---------" and "-------------"

For eg.,
first SUM 10
second SUM 20
like wise between "---------" and "-------------" i need the total value
# 5  
Old 05-20-2013
Why store in variables?
This one prints
Code:
awk '{s+=$2} /----/ {print s; s=0}'

# 6  
Old 05-21-2013
Is this the area you like to sum (in red)?

Code:
maths                   2            0.0          
english                2091            0.2          
history               -1            0.0           
physics                18            0.0
--------------------------------------
maths                   2            0.0          
english                2091            0.2          
history               -1            0.0           
physics                18            0.0
-------------------------------------
maths                   2            0.0          
english                2091            0.2          
history               -1            0.0           
physics                18            0.0

Code:
awk '/---/{if (f==0) f=1; else f=0} f{s+=$2} END {print s}'
2110

This User Gave Thanks to Jotne For This Post:
# 7  
Old 05-21-2013
Hi Jotne,

Yes, You are right..
I need to sum up the values marked in RED..
But what I need is like
Code:
Code:
  maths                   2            0.0           english                2091            0.2           history               -1            0.0            physics                18            0.0 -------------------------------------- maths                   2            0.0           english                2091            0.2           history               -1            0.0            physics                18            0.0 ------------------------------------- maths                   2            0.0           english                2091            0.2           history               -1            0.0            physics                18            0.0
-----------------------------------------
maths                   2            0.0           english                2091            0.2           history               -1            0.0            physics                18            0.0
------------------------------------------

Here I need to find 4 sums between "---------" and "-------------"

But the code You suggested works only for one set of values that are marked in RED


I have tried a set of code like
Code:
cat New | awk '{print $2}' > trick
lineCnt=`cat trick | wc -l`
echo "The total number of lines is $lineCnt"
line=1
while [[ $line -le $lineCnt ]] && read LINE; do
    if [[ $LINE = "-----" ]]; then
        line=$((line+1))
        echo "The previous total is $total"
        total=0
    else
    #total=$LINE
    total=$((total+LINE))
    fi
done < trick

But I get the following error
Code:
./DataCompare.sh
The total number of lines is       19
The previous total is
./DataCompare.sh[14]: ------: 0403-053 Expression is not complete; more tokens expected.

Please correct if am wrong anywhere..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sum values of specific column in multiple files, considering ranges defined in another file

I have a file (let say file B) like this: File B: A1 3 5 A1 7 9 A2 2 5 A3 1 3 The first column defines a filename and the other two define a range in that specific file. In the same directory, I have also three more files (File A1, A2 and A3). Here is 10 sample lines... (3 Replies)
Discussion started by: Bastami
3 Replies

2. Shell Programming and Scripting

If first pattern is found, look for second pattern. If second pattern not found, delete line

I had a spot of trouble coming up with a title, hopefully you'll understand once you read my problem... :) I have the output of an ldapsearch that looks like this: dn: cn=sam,ou=company,o=com uidNumber: 7174 gidNumber: 49563 homeDirectory: /home/sam loginshell: /bin/bash uid: sam... (2 Replies)
Discussion started by: samgoober
2 Replies

3. Shell Programming and Scripting

To add a new line with specific text after the pattern is found using sed

hi guys, im trying to add the following line in my xml file <dbrollbacksegs <oa_var="s_db_rollback_segs">NOROLLBACK</dbrollbacksegs> when ever i find the following line <dbsharedpool oa_var="s_dbsharedpool_size">300000000</dbsharedpool> I have succedded till adding a new line... (1 Reply)
Discussion started by: smarlaku
1 Replies

4. Shell Programming and Scripting

Sum of column matching pattern/string

Hi All, I have the following output file created as a result of one script called pattern_daily.log $ cat pattern_daily.log Approved|2|Wed, Oct 24, 2012 11:21:09 AM Declined|1|Wed, Oct 24, 2012 11:21:15 AM Approved|2|Wed, Oct 24, 2012 11:24:08 AM Declined|1|Wed, Oct 24, 2012 11:24:18 AM... (4 Replies)
Discussion started by: Gem_In_I
4 Replies

5. Shell Programming and Scripting

Replace column that matches specific pattern, with column data from another file

Can anyone please help with this? I have 2 files as given below. If 2nd column of file1 has pattern foo1@a, find the matching 1st column in file2 & replace 2nd column of file1 with file2's value. file1 abc_1 foo1@a .... abc_1 soo2@a ... def_2 soo2@a .... def_2 foo1@a ........ (7 Replies)
Discussion started by: prashali
7 Replies

6. Shell Programming and Scripting

awk to sum specific field when pattern matches

Trying to sum field #6 when field #2 matches string as follows: Input data: 2010-09-18-20.24.44.206117 UOWEXEC db2bp DB2XYZ hostname 1 2010-09-18-20.24.44.206117 UOWWAIT db2bp DB2XYZ hostname ... (3 Replies)
Discussion started by: ux4me
3 Replies

7. Shell Programming and Scripting

extract specific line if the search pattern is found

Hi, I need to extract <APPNUMBER> tag alone, if the <college> haas IIT Chennai value. college tag value will have spaces embedded. Those spaces should not be suppresses. My Source file <Record><sno>1</sno><empid>E0001</empid><name>Rejsh suderam</name><college>IIT ... (3 Replies)
Discussion started by: Sekar1
3 Replies

8. Shell Programming and Scripting

Print out specific pattern column data

Input file: adc_0123 haa_1000 bcc_520 adc_0150 bcc_290 adc_0112 haa_8000 adc_0139 haa_7000 Output file: adc_0123 adc_0123 haa_1000 bcc_520 adc_0150 adc_0150 bcc_290 (3 Replies)
Discussion started by: patrick87
3 Replies

9. Shell Programming and Scripting

Print rows, having pattern in specific column...

Hello all, :) I have a pattern file some what like this, cd003 cd005 cd007 cd008 and input file like this, abc cd001 cd002 zca bca cd002 cd003 cza cba cd003 cd004 zca bac cd004 cd005 zac cba cd005 cd006 acz acb cd006 cd007 caz cab cd007 ... (25 Replies)
Discussion started by: admax
25 Replies

10. UNIX for Dummies Questions & Answers

Breaking output for specific pattern

Respected Sirs, I have a text file which is unfortunately unformatted. I have something like 191728/02/06226278 191828/02/06226279 191928/02/06226280 192028/02/06226281 192128/02/06226282 Can some one suggest me the way so that I can store 1917 28/02/06 226278 in different... (2 Replies)
Discussion started by: shoeb_syed
2 Replies
Login or Register to Ask a Question