subtotal columns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting subtotal columns
# 1  
Old 02-06-2008
subtotal columns

Hello

I have a file that has two (or more) different types of records I want to total. How would I do this using awk?

The file may contain several dozen records. The records are sorted on the database column - what I want to do is get the amount of space that each table has in that database and, add that amount to a variable. When the database changes, I want to print out that subtotal, reset the variable to zero and process the rest of the file.

I've seen examples where awk will total the ENTIRE column but, I need to create subtotals for each database when it changes (2,3,4,maybe more). I'm thinking awk would be the most effective way to go with this; is there something else that I could use? Any help would be greatly appreciated.

Thanks
# 2  
Old 02-06-2008
Hi, post your source file.
# 3  
Old 02-06-2008
Perl is very good at pulling data out of files and doing just about anything you want with it. But it all depends on your data and how it is formatted. Clarify your question and post the source file or some sample lines of the data. Anyone that wants to help will need that information regardless of what script/shell is used.
# 4  
Old 02-06-2008
subtotal columns - example file

Sorry I didn't provide this earlier. Here is a sample of the file

Product Table1 300
Product Table2 226
Product Table3 410
Employee Table1 250
Employee Table2 300

So, I want to set up a do loop to test the first column (database). While the database is the same, I want to accumulate the values in column 3 (space). When the value of column 1 changes, I want to print the total amount of space in that database - so when the Product database changes, I want to print the value 936. After I print tat out, I want to zero the total variable out and continue with the Employee database

Here is the code I've written so far - it doesn't do what it's supposed to do but, it's a start

total=0
temp_db=" "
cat $file | while read line
do
db=`echo $line | cut -f1 -d" "`
difference=`echo $line | cut -f3 -d" "`
let total=$difference + $total
if [ "$db" != "$temp_db" ] && [ "$db" != "" ]; then
echo "space on $temp_db = $total
temp_db=$db
total=0
fi
done
# 5  
Old 02-06-2008
Code:
nawk '
   {db[$1] += $3} 
END {
   for iter in db
      print iter, db[iter]
}' myFile

# 6  
Old 02-06-2008
Thanks

Thanks vgersh99 - your code works perfectly
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to add subtotal and total according 3rd field mentioned below table?

111AKKK|SHA|20|25 111|AKKK|SHA|15|35 112|ABC|FL|25|45 112|ABC|FL|15|15 115|AKKK|ASH|10|15 115|AKKK|ASH|20|20 112|ABC|FL|25|20 115|AKKK|ASH|30|35 111|AKKK|SHA|10|45 112|ABC|KL|15|15 112|ABC|KL|20|25 115|AKKK|ASH|30|35 please write a shell script output should be below mentioned... (26 Replies)
Discussion started by: udhal
26 Replies

2. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ... (4 Replies)
Discussion started by: Khushbu
4 Replies

3. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

4. Shell Programming and Scripting

Subtotal in UNIX

Please help me on below req Data in file ARIZONA HCPAZ 47 ARIZONA HCPAZCONT 3056 ARIZONA AZA 20 CALIFORNIA HC06 878 CALIFORNIA LC04 51 CALIFORNIA LC06 4039 CALIFORNIA HCPCACONT 4960 THE CAMDEN GROUP CAM 83... (7 Replies)
Discussion started by: skchevva
7 Replies

5. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

6. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

7. Shell Programming and Scripting

Combine columns from many files but keep them aligned in columns-shorter left column issue

Hello everyone, I searched the forum looking for answers to this but I could not pinpoint exactly what I need as I keep having trouble. I have many files each having two columns and hundreds of rows. first column is a string (can have many words) and the second column is a number.The files are... (5 Replies)
Discussion started by: isildur1234
5 Replies

8. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

9. UNIX for Dummies Questions & Answers

How do you subtotal lines in a file? Awk?

I have a file with 8 fields. I need the subtotals for fields 7 & 8 when field 5 changes. cat wk1 01/02/2011/18AB/17/18/000000071/000000033 01/02/2011/18AB/17/18/000000164/000000021 01/02/2011/18AB/17/18/000000109/000000023 01/02/2011/28FB/04/04/000000000/000000000... (2 Replies)
Discussion started by: MS75001
2 Replies

10. Shell Programming and Scripting

Single command for add 2 columns and remove 2 columns in unix/performance tuning

Hi all, I have created a script which adding two columns and removing two columns for all files. Filename: Cust_information_1200_201010.txt Source Data: "1","Cust information","123","106001","street","1-203 high street" "1","Cust information","124","105001","street","1-203 high street" ... (0 Replies)
Discussion started by: onesuri
0 Replies
Login or Register to Ask a Question