Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Add values of similar patterns with awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Add values of similar patterns with awk
# 1  
Old 06-05-2016
Add values of similar patterns with awk
so my output is this:

Code:
session_closed=157
session_opened=151
session_closed=18
session_opened=17

there are two patterns here, but with different values. the two patterns are "session_opened" and "session_closed". i expect there will be many more other patterns.

what i want to do is whenever there's a duplicate of patterns, i want to add up all the numbers of each duplicate, so that the refined output looks like this:

Code:
session_opened=168
session_closed=175

i dont want to do a "uniq" here in case there are duplicates with the same exact number. i want to make sure I add up ALL the values of all the patterns.

i would like to do this in awk. i cant seem to come up with any ideas to get this done.
# 2  
Old 06-05-2016
Do you mean something like:
Code:
awk -F'=' '
{	c[$1] += $2
}
END {	for(i in c) printf("%s%s%s\n", i, FS, c[i])
}' file

As always, if you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
This User Gave Thanks to Don Cragun For This Post:
SkySmart
# 3  
Old 06-05-2016
Quote:
Originally Posted by Don Cragun
Do you mean something like:
Code:
awk -F'=' '
{	c[$1] += $2
}
END {	for(i in c) printf("%s%s%s\n", i, FS, c[i])
}' file

As always, if you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
this worked beautifully. anyway you can explain what each line is doing for me, please?? i know the first line is specifying the delimiter. but how are the numbers for each pattern being added. i do a lot of for loops like this in bash and i'd love to be able to do them all in awk.

thank you!
# 4  
Old 06-05-2016
Quote:
Originally Posted by SkySmart
this worked beautifully. anyway you can explain what each line is doing for me, please?? i know the first line is specifying the delimiter. but how are the numbers for each pattern being added. i do a lot of for loops like this in bash and i'd love to be able to do them all in awk.
thank you!
Hello SkySmart,

Following may help you in same, please do let me knnow in case you have any queries on same.
Code:
awk -F'=' '                                         ##### Making = as a field seprator  here.
{	c[$1] += $2                                 ##### creating an array named C whose index is $1 abd value is $2, so += means add the same indexes values to it's previous values, so that we could get a total sum of same index($1, first field's) as per your requirement.
}
END {	for(i in c) printf("%s%s%s\n", i, FS, c[i]) ##### Starting END block here, where starting a for loop in array c, so i is a vriable here and it will traverse through all items of array c, then printing the value of i(which is index pf array, you could say first field's value then printing FS(which is field seprator =) then printing the array c's current index(i)'s value by c[i].
}' file                                             ##### mentioning Input_file here.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
SkySmart
# 5  
Old 06-05-2016
You count the $2 values per $1 value.
That means you need a variable per each $1, ideally this is a $1-addressed array. I.e. $1 is the array key.
And the array stores the sum of the $2 values, i.e. each $2 value is added to it.
Because it is unknown how many values are to be added, you need an END section to print the array keys and their values.
This User Gave Thanks to MadeInGermany For This Post:
SkySmart
# 6  
Old 06-05-2016
There there is the old fashioned way when we only had 1k of memory.
Code:
sort <input >sorted
sub_total=0
grand_total=0
prev_desc=
first_pass=Y
while IFS="=" read desc amount
do
if [ $first_pass = "Y" ]
then
   first_pass="N"
   prev_desc=$desc
fi
if [ "$desc" -ne "$prev_desc" ]
then
  echo $prev_desc $sub_total
  final_total=`expr $grand_total + $sub_total`
  sub_total=0
  prev_desc=$desc
fi
sub_total=`expr $sub_total + $amount
done<sorted
echo $prev_desc $sub_total
grand_total=`expr $grand_total + $sub_total`
echo "Grand Total " $grand_total

Last edited by jgt; 06-05-2016 at 05:29 PM..
This User Gave Thanks to jgt For This Post:
SkySmart
# 7  
Old 06-06-2016
Quote:
Originally Posted by jgt
There there is the old fashioned way when we only had 1k of memory.
Code:
sort <input >sorted
sub_total=0
grand_total=0
prev_desc=
first_pass=Y
while IFS="=" read desc amount
do
if [ $first_pass = "Y" ]
then
   first_pass="N"
   prev_desc=$desc
fi
if [ "$desc" -ne "$prev_desc" ]
then
  echo $prev_desc $sub_total
  final_total=`expr $grand_total + $sub_total`
  sub_total=0
  prev_desc=$desc
fi
sub_total=`expr $sub_total + $amount
done<sorted
echo $prev_desc $sub_total
grand_total=`expr $grand_total + $sub_total`
echo "Grand Total " $grand_total

Even with an original Bourne shell and back in the days before test was a shell built-in, the -ne comparison operator was for comparing numeric values; not strings. For strings, the not equal comparison operator is and was !=. And, I assume the final_total above in red was intended to be grand_total.

With POSIX conforming shells (and small input files), sort and shell might be faster than awk. You might want to try the following:
Code:
#!/bin/ksh
sub_total=0
grand_total=0
prev_desc=
sort file | while IFS="=" read desc amount
do	if [ "$desc" != "$prev_desc" ]
	then	printf '%s=%d\n' "$prev_desc" "$sub_total"
		grand_total=$((grand_total + sub_total))
		sub_total=0
		prev_desc=$desc
	fi
	sub_total=$((sub_total + amount))
done
printf '%s=%d\n' "$prev_desc" "$sub_total"
grand_total=$((grand_total + sub_total))
printf 'Grand Total=%d\n' "$grand_total"

Although written and tested using a Korn shell, the above script should work with any POSIX-conforming shell. And, with a 1993 or later version of the Korn shell, if you change all three occurrences of %d in the above script to %.2f, this script can also handle amounts presented with or without a decimal point and up to 2 digits after the decimal point (instead of just processing whole numbers).
This User Gave Thanks to Don Cragun For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Add a number all the the values and log transform them using awk

Hi, is it possible to add a number to all the values in a file and log transform them using awk. thanx in advance input name c1 c2 c3 c4 r1 0 0.2 0.3 0.4 r2 0 0 0 1 r3 1 0 0 0 r4 0 0 1 ... (1 Reply)
Discussion started by: quincyjones
1 Replies

2. Shell Programming and Scripting

Delete values between 2 patterns

Hi, How can i delete values between 2 patterns like below:- input.txt 192.1.1.2.22 blablabala 23.1.A.1.2 blablabalbl 5.4.1.1.12 blablaba i need to delete all values between starting from "." no 3 and second column. the output should be: 192.1.1 blablabala... (15 Replies)
Discussion started by: redse171
15 Replies

3. Shell Programming and Scripting

Code to exclude lines with similar values

Hi!!! I have a problem with txt file. For example: File: CATEGORY OF XXX AAA 1 XXX BBB CCC AAA 1 XXX DDD EEE AAA 1 XXX FFF GGG AAA 1 XXX KKK LLL AAA 1 XXX MMM ... (4 Replies)
Discussion started by: Tzole
4 Replies

4. Shell Programming and Scripting

How to find similar values in different files

Hello, I have 4 files like this: file1: cg24163616 15 297 cg09335911 123 297 cg13515808 565 776 cg12242345 499 705 cg22905282 225 427 cg16674860 286 779 cg14251734 303 724 cg19316579 211 717 cg00612625 422 643 file2:... (2 Replies)
Discussion started by: linseyr
2 Replies

5. Shell Programming and Scripting

SED/AWK to edit/add field values in a record

Hi Experts, I am new to shell scripting. Need some help in doing one task given by the customer. The sample record in a file is as follows: 3538,,,,,,ID,ID1,,,,,,,,,,, It needs to be the following: 3538,,353800,353800,,,ID,ID1,,,,,COLX,,,,,COLY, And i want to modify this record in... (3 Replies)
Discussion started by: sugarcane
3 Replies

6. Shell Programming and Scripting

removing lines with similar values from file

Hello, got a file with this structure: 33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26 33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27 33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26 33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies

7. Shell Programming and Scripting

awk to search similar strings and add their values

Hi, I have a text file with the following content: monday,20 tuesday,10 wednesday,29 monday,10 friday,12 wednesday,14 monday,15 thursday,34 i want the following output: monday,45 tuesday,10 wednesday,43 friday,12 (3 Replies)
Discussion started by: prashu_g
3 Replies

8. Shell Programming and Scripting

Joining multiple files based on one column with different and similar values (shell or perl)

Hi, I have nine files looking similar to file1 & file2 below. File1: 1 ABCA1 1 ABCC8 1 ABR:N 1 ACACB 1 ACAP2 1 ACOT1 1 ACSBG 1 ACTR1 1 ACTRT 1 ADAMT 1 AEN:N 1 AKAP1File2: 1 A4GAL 1 ACTBL 1 ACTL7 (4 Replies)
Discussion started by: seqbiologist
4 Replies

9. Shell Programming and Scripting

Capture values using multiple regex patterns

I have to read the file, in each line of file i need to get 2 values using more than one search pattern. ex: <0112 02:12:20 def > /some string/some string||some string||124 i donot have same delimiter in the line, I have to read '0112 02:12:20' which is timestamp, and last field '124' which is... (4 Replies)
Discussion started by: adars1
4 Replies

10. Shell Programming and Scripting

AWK help to add up sequential values

Hello All! As a beginner user i want to add up sequential values in a text file and want to print total sum as output.The Text file will have values like the following: class1{root}>less SUM.txt 1140.00 1155.00 1183.00 ... # it continues # i tried to write a... (1 Reply)
Discussion started by: EAGL€
1 Replies
Login or Register to Ask a Question