Shell script - group by


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script - group by
# 1  
Old 05-02-2014
Lightbulb Shell script - group by

Hi,
I have text file as shown below.

Code:
root 25 oracle 25  batch 30  griduser 32 admin 35
root 25 oracle 25  batch 30  griduser 32
oracle 25  batch 30  griduser 32 xuser 45 admin 35

I want to group by based on user name, and the output need to be as below. Not necessary the username to be in order, but group by has to be done based on the username value.

Code:
root 50 oracle 75  batch 90  griduser 64 user 45 admin 70

Please help.
Thanks

---------- Post updated at 03:54 PM ---------- Previous update was at 03:52 PM ----------

typo error, pasting the correct output

Code:
root 50 oracle 75 batch 90 griduser 96 user 45 admin 70


Last edited by Scrutinizer; 05-03-2014 at 03:24 AM.. Reason: CODE tags
# 2  
Old 05-02-2014
Show us what you have tried so far to resolve this problem.
# 3  
Old 05-02-2014
Code:
RC=`awk 'END { print NR }' $FILENAME`
RC=`expr $RC`
OBJNO=4
if [ "$RC" -ge 4 ]; then
             SETVAL=1
             OBJDETAIL=""
             FIELD1=1
             FIELD2=2
       until [ $OBJNO -eq $SETVAL ]; do
             DATAVAL=`cat $FILENAME | awk '{a[$'$FIELD1']+=$'$FIELD2'}END{for(i in a)print i,",",int((int(a[i])/NR))}' `
             SETVAL=`expr $SETVAL + 1`
             FIELD1=`expr $FIELD1 + 2`
             FIELD2=`expr $FIELD2 + 2`
             OBJDETAIL="$OBJDETAIL"",""$DATAVAL"
        done
     echo $OBJDETAIL
fi

This code works when file has same username in the enitre field, now requirement changed.

Last edited by Corona688; 05-02-2014 at 05:31 PM..
# 4  
Old 05-02-2014
Please use code tags for posting code fragments or data samples.

Here is an awk approach:
Code:
awk '
        {
                for ( i = 1; i <= NF; i += 2 )
                        A[$i] += $( i + 1 )
        }
        END {
                for ( k in A )
                        printf "%s %s ", k, A[k]
                printf "\n"
        }
' file

This User Gave Thanks to Yoda For This Post:
# 5  
Old 05-05-2014
Lightbulb

Thanks, this is a great approach. It worked for me.
# 6  
Old 05-05-2014
A python:
Code:
# cat test.py
#!/usr/bin/env python
import re

text = '''root 25 oracle 25  batch 30  griduser 32 admin 35
root 25 oracle 25  batch 30  griduser 32
oracle 25  batch 30  griduser 32 xuser 45 admin 35'''

users = {}

for k,v in [i for i in re.findall('(?:([a-z]+)\s+(\d+))',text)]:
    if k in users:
        users[k] += int(v)
    else:
        users[k] = int(v)
print users

Code:
# ./test.py  
{'admin': 70, 'griduser': 96, 'xuser': 45, 'batch': 90, 'oracle': 75, 'root': 50}

# 7  
Old 05-05-2014
Below Awk sum the value correctly based on username, but not dividing the sum of values based on number of rows of user name. I need something like
root 50 oracle 75 .......can anyone shed some light please?
Code:
awk '        {                for ( i = 1; i <= NF; i += 2 )                        A[$i] += $( i + 1 )        }        END {                for ( k in A )                        printf "%s %s ", k, A[k]                printf "\n"        }' file


Last edited by vbe; 05-05-2014 at 01:29 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Emulate group-by in shell script

Hello All, I saw this problem on one of the forum and solved it using group-by in oracle sql, though I am a bit curious to implement it using shell script : There is a file having number of operations : Opeation,Time-Taken operation1,83621 operation2,72321 operation3,13288... (11 Replies)
Discussion started by: mukulverma2408
11 Replies

2. Shell Programming and Scripting

Shell Script to Group by Based on Multiple Fields in a file

Hi, I want to know if there is any simple approach to SUM a field based on group by of different fields for e.g. file1.txt contains below data 20160622|XXX1||50.00||50.00|MONEY|Plan1| 20160622|XXX1||100.00||100.00|MONEY|Plan1| 20160623|XXX1||25.00||25.00|MONEY|Plan1|... (3 Replies)
Discussion started by: cnu_theprince
3 Replies

3. Shell Programming and Scripting

Help me to perform count & group by operation in shell scripting?

Hi All, I want to display the distinct values in the file and for each distinct value how may occurance or there. Test data: test1.dat 20121105 20121105 20121105 20121105 20121106 20121106 20121106 20121105 I need to display the output like Output (2 Replies)
Discussion started by: bbc17484
2 Replies

4. Shell Programming and Scripting

Help Linux Shell Group exists

I am having some problems when writing shell as follows: shell runs but returns no results echo "enter group name: " dir="/home" read group if id -g $group > /dev/null 2>&1 then echo "group exits" else echo... (6 Replies)
Discussion started by: kingkner
6 Replies

5. Shell Programming and Scripting

Shell Script to ignore # and take corresponding user and group

Hi, I have a following file: role.IMPACT_USER.user=admin role.IMPACT_USER.user=dd12345 role.IMPACT_USER.user=ss76767 #role.IMPACT_USER.user=root #role.IMPACT_USER.group=System role.IMPACT_USER.group=ImpactUser #Description: Allow users to login in to Impact, start and stop service... (5 Replies)
Discussion started by: dbashyam
5 Replies

6. Solaris

su: No shell/No directory! if sys is added to a users secondary group

Hi, When I include a user to the secondary group "sys" GID=3 in Solaris 9 OS I'm not able to login. I get these error. The user home directory and the shell exists. Is this because of any security hardening. # su - agent No directory! # su agent su: No shell # grep taddm /etc/passwd... (14 Replies)
Discussion started by: agent001
14 Replies

7. Shell Programming and Scripting

"group by" using shell script?

not sure if it's called "group by" , but what i'm going to do is like this: i have a file below: 192.168.1.10 192.168.1.10 192.168.1.10 192.168.1.11 192.168.1.15 192.168.1.15 192.168.1.20 192.168.1.22 then i hope to get the result like this: 192.168.1.10 : 3 192.168.1.11 : 1... (6 Replies)
Discussion started by: tiger2000
6 Replies

8. Shell Programming and Scripting

Shell script to rename a group of files

Hello, I am having 1800 files in a directory with a specified format, like amms_850o_prod.000003uNy amms_850o_prod.000003u8x amms_850o_prod.000003taP amms_850o_prod.000003tKy amms_850o_prod.000003si4 amms_850o_prod.000003sTP amms_850o_prod.000003sBg amms_850o_prod.000003rvx... (12 Replies)
Discussion started by: atlantis
12 Replies

9. Shell Programming and Scripting

HELP! Group by in shell script (awk/sed?)

Hello, Could some expert soul please help me with this? I have following file format - task time abc 5 xyz 4 abc 5 xyz 3 ddd 10 ddd 2 I need to generate output as - task ... (5 Replies)
Discussion started by: sncoupons
5 Replies
Login or Register to Ask a Question