Visit Our UNIX and Linux User Community


"group by" using shell script?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting "group by" using shell script?
# 1  
Old 05-17-2010
"group by" using shell script?

not sure if it's called "group by" , but what i'm going to do is like this:

i have a file below:

Code:
192.168.1.10
192.168.1.10
192.168.1.10
192.168.1.11
192.168.1.15
192.168.1.15
192.168.1.20
192.168.1.22

then i hope to get the result like this:

Code:
192.168.1.10 : 3
192.168.1.11 : 1
192.168.1.15 : 2
192.168.1.20 : 1
192.168.1.22 : 1

the number in second column is how many times it appears in the file.

1. use sort , and uniq to get how many unique record in this file
2. use grep with wc -l command to get how many times it appears

is there any better to do so ?? any advice?

Thanks.

Last edited by radoulov; 05-17-2010 at 08:12 AM.. Reason: Please use code tags!
# 2  
Old 05-17-2010
Something like this ?
Code:
$ awk '{count[$1]++}END{for(j in count) print j":"count[j]}' file.txt

# 3  
Old 05-17-2010
Jaduks awk solution does not sort the output as requested by the OP. This can be remedied by piping the output to sort.

You can do it completely within a shell. For example using ksh93 the following script
Code:
#!/bin/ksh93

typeset -A count

# create an associative array
while read ip
do
    (( count[$ip]++ ))
done < infile


# sort print the associative array
while (( 1 ))
do
   (( !${#count[@]} )) && break;
   k=(${!count[@]})
   for j in ${!count[@]}
   do
      (( ${count[$j]} > ${count[$k]} )) && k=$j
   done
   echo "$k : ${count[$k]}"
   unset count[$k]
done

and produces the following sorted list
Code:
192.168.1.10 : 3
192.168.1.15 : 2
192.168.1.20 : 1
192.168.1.11 : 1
192.168.1.22 : 1

Associate arrays are also supported in bash V4 but use a slightly different syntax.

Last edited by fpmurphy; 05-17-2010 at 03:24 PM..
# 4  
Old 05-17-2010
Here is another simple solution:
Code:
for i in $(cat file.txt)
do
  echo "$i: $(grep -c $i file.txt)"
done | sort -u

# 5  
Old 05-17-2010
sort file|uniq -c

although the output is different, but you could do, to get the desired format:

sort file|uniq -c|awk '{print $2 " : " $1}'
# 6  
Old 05-17-2010
sort in awk:

Code:
 awk '{count[$1]++}END{for(j in count) print j":"count[j] |"sort -t: -k2r"}' urfile

# 7  
Old 05-17-2010
I like shahhe's offering for quickest and most likely to become a one-liner, except that it needs one (or two) last tweak to get the sort requested by the OP:

Code:
for i in $(<file.txt)
do
  echo "$i: $(grep -c $i file.txt)"
done | sort -u -t. -k4


Previous Thread | Next Thread
Test Your Knowledge in Computers #283
Difficulty: Easy
Early computers were built to perform a series of single tasks, like a calculator.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

3. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

4. Shell Programming and Scripting

Command Character size limit in the "sh" and "bourne" shell

Hi!!.. I would like to know what is maximum character size for a command in the "sh" or "bourne" shell? Thanks in advance.. Roshan. (1 Reply)
Discussion started by: Roshan1286
1 Replies

5. UNIX for Advanced & Expert Users

Command Character size limit in the "sh" and "bourne" shell

Hi!!.. I would like to know what is maximum character size for a command in the "sh" or "bourne" shell? Thanks in advance.. Roshan. (1 Reply)
Discussion started by: Roshan1286
1 Replies

6. UNIX for Dummies Questions & Answers

Command Character size limit in the "sh" and "bourne" shell

Hi!!.. I would like to know what is maximum character size for a command in the "sh" or "bourne" shell? Thanks in advance.. Roshan. (1 Reply)
Discussion started by: Roshan1286
1 Replies

7. AIX

"too big" and "not enough memory" errors in shell script

Hi, This is odd, however here goes. There are several shell scripts that run in our production environment AIX 595 LPAR m/c, which has sufficient memory 14GB (physical memory) and horsepower 5CPUs. However from time to time we get the following errors in these shell scripts. The time when these... (11 Replies)
Discussion started by: jerardfjay
11 Replies

8. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies

9. UNIX for Dummies Questions & Answers

No utpmx entry: you must exec "login" from lowest level "shell"

Hi I have installed solaris 10 on an intel machine. Logged in as root. In CDE, i open terminal session, type login alex (normal user account) and password and i get this message No utpmx entry: you must exec "login" from lowest level "shell" :confused: What i want is: open various... (0 Replies)
Discussion started by: peterpan
0 Replies

Featured Tech Videos