Calculating totals in AWK


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Calculating totals in AWK
# 1  
Old 08-28-2008
Calculating totals in AWK

Hello,

With the following small script I list the size of documents belonging to a certain user by each time selecting the bytes-field of that file ($7). Now it fills the array with every file it finds so in the end the output of some users contains up to 200.000 numbers. So how can I calculate the total used diskspace of each user by using awk? By this the output would only contain 1 number, instead of thousands and also I wouldn't have to use other things to calculate the totals which would save me a lot of time.

Here's the script:
find /tmp -type f -ls | awk '{
print $7 >> "UsedDiskSpace_" $5 ".txt"
}'



Thanks in advance!
# 2  
Old 08-28-2008
to calculate total space using awk
awk '{space+=$7}END{print space}' >> "UsedDiskSpace_" $5 ".txt"
# 3  
Old 08-28-2008
Thanks for your quick response, but how would you implement it in the script itself? I've tried it but it didn't went well. I'm rather new to the 'awk' command you see Smilie.

Thanks
# 4  
Old 08-28-2008
The system I'm working on at the moment doesn't support the -ls option with find. Can you post some lines of the output of your find command?

Regards
# 5  
Old 08-28-2008
Well, for example, when you perform a normal ls -l the output is:

-rwxr-xr-x 1 root exploit 325 Aug 9 2004 File1
-rwxr-xr-x 1 exp sys 384 Apr 12 2000 File2
-rwxrwxrwx 1 exp exploit 100 Mar 19 2007 File3
-rw-r--r-- 1 exp sys 597 Jun 24 1999 File4
-rwxr-xr-x 1 oracle system 242 Feb 5 2001 File5
-rwxr-xr-x 1 oracle system 184 Jul 5 2002 File 6
...

So from all those files I only need to know how big they are and then sort it per user. In the script I put the output from the awk command in a file, so it can be sorted by each user. If I open the DiskUsage_exp.txt file for example (from user 'exp'), this is the output:

384
100
597

But I have to perform the find command on the whole server, so you can imagine how many lines I would eventually get, the lines could ran up to thousands of numbers from a couple of users. So what I need is just the sum of those lines for each users. So for user 'exp' this would be '1081' and for user 'oracle' '426'.

Thanks
# 6  
Old 08-28-2008
This absorbs a lot of memory but you can give it a try:

Code:
find /tmp -type f -ls | awk '{a[$3]+=$5}END{for(i in a){print i, a[i]}}' > UsedDiskSpace

Regards
# 7  
Old 08-28-2008
Great! Thank you!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Calculating correlations across columns in awk

Hello, I came across a previous thread "awk-calculating-simple-correlation-rows" which calculated correlations across rows in awk. Code: awk '{ a = 0; for (i = 2; i <= NF; ++i) a += $i; a /= NF-1 b = 0; for (i = 2; i <= NF; ++i) b += ($i - a) ^ 2; b = sqrt(b) if... (7 Replies)
Discussion started by: Ross
7 Replies

2. Shell Programming and Scripting

Grand totals in awk

I have a one-liner script like this that gives a total of everything in various directories: for i in *; do (cd $i && cd statelist && echo $i && ls -la |awk 'NR>3 {SUM += $5}\ END { print "Total number of elements " SUM }');done It works just great but at the end I want to print a grand... (3 Replies)
Discussion started by: newbie2010
3 Replies

3. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

4. Shell Programming and Scripting

Calculating Running Variance Using Awk

Hi all, I am attempting to calculate a running variance for a file containing a column of numbers. I am using the formula variance=sum((x-mean(x))^2)/(n-1), where x is the value on the current row, and mean(x) is the average of all of the values up until that row. n represents the total number... (1 Reply)
Discussion started by: Jahn
1 Replies

5. Shell Programming and Scripting

Calculating average with awk

I need to find the average from a file like: data => BW:123 M:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1... (4 Replies)
Discussion started by: Slagle
4 Replies

6. Shell Programming and Scripting

Calculating the epoch time from standard time using awk and calculating the duration

Hi All, I have the following time stamp data in 2 columns Date TimeStamp(also with milliseconds) 05/23/2012 08:30:11.250 05/23/2012 08:30:15.500 05/23/2012 08:31.15.500 . . etc From this data I need the following output. 0.00( row1-row1 in seconds) 04.25( row2-row1 in... (5 Replies)
Discussion started by: ks_reddy
5 Replies

7. Shell Programming and Scripting

AWK way of calculating growth

Hi All, IS there any 'awk' way to manipulate following data? Fruit Date Count Apple 20/08/2011 5 Apple 27/08/2011 7 Apple 05/09/2011 11 Apple 12/09/2011 3 Apple 19/09/2011 25 . . . . Orange 20/08/2011 9 Orange 27/08/2011 20 Orange 27/08/2011 7 Orange 05/09/2011 15 Orange... (3 Replies)
Discussion started by: aniketdixit
3 Replies

8. Shell Programming and Scripting

Calculating an integer with awk

I would like to extract a number from $0 and calculate if it can be devided by 25. Though the number can also be less then 25 or bigger than 100. How do i extract the number and how can the integer be calculated? String: "all_results">39</span>I am looking for the number between "all_results"> ... (5 Replies)
Discussion started by: sdf
5 Replies

9. Shell Programming and Scripting

[Solved] awk calculating between lines

Hey guys, maybe you can help me with this... I want to read input.dat line by line, while doing a simple calculation between the second column value of the current line and the second column value of the next line (like a difference). input is something like this: 0 3.945757 1 ... (1 Reply)
Discussion started by: origamisven
1 Replies

10. Shell Programming and Scripting

summarising totals in awk

awk ' FILENAME == "all" { balance += substr($0,17,13) dt = substr($0,6,8) } END { for ( name in balance ) printf("%013s %3s of %8s\n", balance/100,name,dt) | "sort " } ' all > summation using this code i wanted to take summary totals of... (3 Replies)
Discussion started by: paresh n doshi
3 Replies
Login or Register to Ask a Question