Awk -- why doesn't my min work?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk -- why doesn't my min work?
# 1  
Old 02-18-2010
Awk -- why doesn't my min work?

So, I have a files with entries in this format:

Code:
servername,username,sub_username,useless_datapoint,mail_size

So, a few sample lines:

Code:
server_a,bob,jane,useless,112351
server_a,bob,jim,useless,421193
server_a,bob,bob,useless,0029385
server_a,karen,will,useless,112351
server_a,karen,matt,useless,56849237
server_b,lois,ren,useless,98763
server_b,lois,stimpy,useless,3842798


Note that the second field is guaranteed to appear only on one server. If bob is a user on server_a, there will not be a user bob on any other server. Now, what I want to do is gather certain statistics from this data, such as summed mail_size under each user, the max and min mail_size for each sub_user under a particular user, etc.

Restated more clearly, for each unique user in $2, I want:

Sum of the mail_size values for all subusers under that user
The largest mail_size of all subusers under that user
The smallest mail_size of all subusers under that user

For some reason, I just can't get the last one to work. Here's what I have:

Code:
awk -F "," 'length($2) > 0  { idx=$2 ; dsizesum[idx]+=$NF ; usercount[idx]+=1 ; \
serverid[idx]=$1 } ; $5 >= max[idx] {max[idx] = $5} ; $5 < min[idx] {min[idx] = $5} ; \
END { for(i in dsizesum) { print serverid[i] "," i "," dsizesum[i] "," usercount[i] "," max[i] "," min[i]} }'  datafile

For very weird reasons outside my control, this has to be done as a command line rather than a script; hence the messiness. All of the numbers that it bothers to spit out I've verified as correct. The only problem is that it gives nothing for min[i]. I believe (please correct me if I'm wrong) that the only reason is that I just can't quite figure out how to initialize min[""]. Any suggestions?

Last edited by treesloth; 02-18-2010 at 04:16 PM..
# 2  
Old 02-18-2010
Code:
awk -F "," 'length($2) > 0  { idx=$2 ; dsizesum[idx]+=$NF ; usercount[idx]+=1 ; \
serverid[idx]=$1; if (min[idx] !~ "[0-9]" ){ min[idx]=$5; } } ; $5 >= max[idx] {max[idx] = $5} ; $5 < min[idx] {min[idx] = $5} ; \
END { for(i in dsizesum) { print serverid[i] "," i "," dsizesum[i] "," usercount[i] "," max[i] "," min[i]} }' file

# 3  
Old 02-18-2010
Even if you do need to do this on the command line it pays to split out the nawk across multiple lines in order to see what is wrong. Once you have it working then you can put it back to one line: -

Code:
awk -F "," '
$2 {
  dsizesum[$2]+=$NF
  usercount[$2]+=1
  serverid[$2]=$1
}
$5 >= max[$2] {max[$2] = $5}
! min[$2] || $5 <= min[$2] {min[$2] = $5} ## You need to check if min[$2] is null first
END {
  for(i in dsizesum) {
    print serverid[i] "," i "," dsizesum[i] "," usercount[i] "," max[i] "," min[i]
  }
}'  datafile

Output: -

Code:
server_b,lois,3941560,2,3842797,98763
server_a,bob,562929,3,421193,0029385
server_a,karen,56961588,2,56849237,112351

You may want to think about what happens if $2 is a null
# 4  
Old 02-19-2010
Many thanks for the replies. Both were very helpful. I'm sorry to say, steadyonabix, that I'm in the very bad habit of "developing" my awk queries on the command line. This probably goes back to my first UNIX/Linux webserver support days when a senior technician very patiently explained to me that Awk is "a command for extracting one column of data from many". I suppose that might considered true, in the same sense that C could be defined as "a method for printing 'Hello, world'". You're right, of course... awk is the only language I (ab)use that way... time to make new habits.

Last edited by treesloth; 02-19-2010 at 01:16 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

-ne 0 doesn't work -le does

Hi, I am using korn shell. until ] do echo "\$# = " $# echo "$1" shift done To the above script, I passed 2 parameters and the program control doesn't enter inside "until" loop. If I change it to until ] then it does work. Why numeric comparison is not working with -ne and works... (3 Replies)
Discussion started by: ab_2010
3 Replies

2. Shell Programming and Scripting

Why my awk doesn't work?

root@SDP_Wildcat_Pass-3-C1:~# cat /proc/driver/rtc rtc_time : 05:29:40 rtc_date : 2014-12-19 alrm_time : 01:51:53 alrm_date : 2014-12-20 alarm_IRQ : no alrm_pending : no update IRQ enabled : no periodic IRQ enabled : no periodic IRQ... (4 Replies)
Discussion started by: yanglei_fage
4 Replies

3. UNIX for Dummies Questions & Answers

Why doesn't this work?

find . -name "05_scripts" -type d -exec mv -f {}/'*.aep\ Logs' {}/.LogFiles \; Returns this failure: mv: rename ./019_0120_WS_WH_gate_insideTEST/05_scripts/*.aep\ Logs to ./019_0120_WS_WH_gate_insideTEST/05_scripts/.LogFiles/*.aep\ Logs: No such file or directory I don't know why it's trying... (4 Replies)
Discussion started by: scribling
4 Replies

4. Shell Programming and Scripting

HELP: If Doesn't Work in AWK

Hi! I have a somehow big file (almost 3000 lines long and thirteen columns). Some lines have no value at all or, at least, are incomplete. The columns' values that have no data are marked with a "-" and the corresponding line (the line that owns that value) should be discarded and not used. ... (5 Replies)
Discussion started by: Marcelo de Brit
5 Replies

5. Shell Programming and Scripting

Awk split doesn't work for empty delimiter

Does anyone know how will I make awk's split work with empty or null separator/delimiter? echo ABCD | awk '{ split($0,arr,""); print arr; }' I need output like: A B C D I am under HP-UX (5 Replies)
Discussion started by: Orbix
5 Replies

6. Shell Programming and Scripting

Awk: Can anyone tell me why this doesn't work?

If there exists a field in stdin, print it, otherwise, print hello..... These print nothing: cat /dev/null | awk '{if ( length > 0 ) print $1; else print "hello"}' cat /dev/null | awk '{if ( $1 ) print $1; else print "hello"}'But the scripts work if I run them directly in a terminal: ... (8 Replies)
Discussion started by: ksheller
8 Replies

7. Shell Programming and Scripting

awk -v -- Why doesn't my example work?

Hi. I've been playing around a bit. This isn't for any practical purpose-- it's really just a theoretical exercise. I wrote this little thing: foreach num ( 6 5 4 ) awk -v "number=$num" 'BEGIN{for(x=0;x<$number;x++) printf "-"; printf "\n"}' end I would expect the following output: ... (3 Replies)
Discussion started by: treesloth
3 Replies

8. UNIX for Dummies Questions & Answers

Script doesn't work, but commands inside work

Howdie everyone... I have a shell script RemoveFiles.sh Inside this file, it only has two commands as below: rm -f ../../reportToday/temp/* rm -f ../../report/* My problem is that when i execute this script, nothing happened. Files remained unremoved. I don't see any error message as it... (2 Replies)
Discussion started by: cheongww
2 Replies

9. UNIX for Advanced & Expert Users

chown doesn't work

Hi unix gurus, I am facing problem with chown command. I am not able to change the owner of any file in my home. It gives error "not owner", though I own the file(as ls shows). What could be the reason? Is it possible that usage chown can be disabled for users. $ $ ls -l total 2... (10 Replies)
Discussion started by: shriashishpatil
10 Replies

10. Shell Programming and Scripting

Why doesn't this work?

cat .servers | while read LINE; do ssh jason@$LINE $1 done exit 1 ./command.ksh "ls -l ~jason" Why does this ONLY iterate on the first server in the list? It's not doing the command on all the servers in the list, what am I missing? Thanks! JP (2 Replies)
Discussion started by: jpeery
2 Replies
Login or Register to Ask a Question