awk scripting - matching records and summing up time


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk scripting - matching records and summing up time
# 1  
Old 05-06-2009
awk scripting - matching records and summing up time

Hello. I just found out about awk, and it appears that this could handle the problem I'm having right now.

I first stumbled on the thread How to extract first and last line of different record from a file, and that problem is almost similar to mine.

In my case, an ASCII file will contain the following fields, among many others:
record date time code
----------------------------
dd 1-3-2006 1:00:20 AM 2
aa 1-2-2006 8:34:21 AM 1
bb 1-2-2006 7:34:21 AM 1
aa 1-2-2006 10:30:22 AM 2
cc 1-2-2006 10:34:21 AM 1
aa 1-2-2006 15:30:22 AM 3
dd 1-2-2006 8:04:11 PM 1
aa 1-2-2006 6:24:44 PM 1
bb 1-2-2006 10:30:22 AM 2
aa 1-2-2006 11:03:19 AM 2
aa 1-2-2006 16:03:19 AM 3
bb 1-2-2006 12:03:19 PM 2
cc 1-2-2006 4:04:11 PM 1
dd 1-3-2006 12:20:00 AM 2
bb 1-2-2006 7:24:44 PM 1
dd 1-3-2006 2:20:02 AM 1
....
.
.


At the end of any given interval of days, I am trying to output
aa <total time>
bb <total time>
cc <total time>
dd <total time>

Where total total time elapsed from matching code 1 are deducted by total time elapsed from matching codes 2 and 3, and so forth.

(example from above sample, sorted to display matching records)
aa 1-2-2006 8:34:21 AM 1
aa 1-2-2006 6:24:44 PM 1 (total time 1: X)
aa 1-2-2006 10:30:22 AM 2
aa 1-2-2006 11:03:19 AM 2 (total time 2: Y)
aa 1-2-2006 15:30:22 AM 3
aa 1-2-2006 16:03:19 AM 3 (total time 3: Z)

total time = X-(Y+Z)

I'm hoping to do it with Python but I've seen some awesome awk scripts and I'm wondering if it could be done with awk instead.

I'm also confounded with computing time past midnight. (ie record dd). If code records don't match, total time is not computed but is marked in the output as
xx "data error"
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk for matching fields between files with repeated records

Hello all, I am having trouble with what should be an easy task, but seem to be missing something fundamental. I have two files, with File 1 consisting of a single field of many thousands of records. I also have File 2 with two fields and many thousands of records. My goal is that when $1 of... (2 Replies)
Discussion started by: jvoot
2 Replies

2. Shell Programming and Scripting

To get older than last 7days records using awk scripting to generate report and send email

Hello All, I have need as below: 1--> I need to get all users(who submit jobs) and their details by using below command: qstat -u \* output of the above command looks line below: job-ID prior name user-id state "submit/start at" queue jclass slots ja-task-ID... (5 Replies)
Discussion started by: VasuKukkapalli
5 Replies

3. UNIX for Dummies Questions & Answers

1st time awk user strings not matching right....

So I was given a file,and I want to count how many occurrences happen with a specific string. I have two, that could have up to 3 different outcomes. Now my trouble I believe starts with this string, "news.cais.net" but why? as of now my output is this... accepted rejected ... (3 Replies)
Discussion started by: squidGreen
3 Replies

4. Shell Programming and Scripting

awk print matching records and occurences of each record

Hi all , I have two files : dblp.xml with dblp records and itu1.txt with faculty members records. I need to find out how many dblp records are related to the faculty members. More specific: I need to find out which names from itu1.txt are a match in dblp. xml file , print them and show how many... (4 Replies)
Discussion started by: iori
4 Replies

5. Shell Programming and Scripting

awk split columns after matching on rows and summing the last column

input: chr1 1 2 3 chr1 1 2 4 chr1 2 4 5 chr2 3 6 9 chr2 3 6 10 Code: awk '{a+=$4}END{for (i in a) print i,a}' input Output: chr112 7 chr236 19 chr124 5 Desired output: chr1 1 2 7 chr2 3 6 19 chr1 2 4 5 (1 Reply)
Discussion started by: jacobs.smith
1 Replies

6. Shell Programming and Scripting

awk pattern matching name in records

Hi, I'm very new to these forums. I was wondering if someone could help an AWK beginner with a pattern matching an actor to his appearance in movies, which would be stored as records. Let's say we have a database of 4 movies (each movie a record with name, studio + year, and actor fields with... (2 Replies)
Discussion started by: Jill Ceke
2 Replies

7. Shell Programming and Scripting

Help summing a file using awk

I'm trying to sum a text file using AWK. Here is an example of the file: 600|3H68| 46 600|3H69| 46 600|3H6F| 290 600|3H6G| 24 600|3HDY| 1 600|3HDY| 3 600|3HE0| 1 600|3HE0| 3 I would like to sum the third field if the first... (7 Replies)
Discussion started by: Drenhead
7 Replies

8. Shell Programming and Scripting

Using awk to summing from a given line

My file is something like this : 03.097 03.094 03.093 03.095 03.091 04.089 06.093 07.225 08.196 06.097 06.094 05.096 04.086 I'd like to sum it from a given line to another one , e.g.: from line 10 until line 20 What s the awk way solving this ? (1 Reply)
Discussion started by: firelink
1 Replies

9. Shell Programming and Scripting

Summing up a matrix using awk

Hi there, If anyone can help me sorting out this small task would be great. Given a matrix like the following: 100 3 3 3 3 3 ... 200 5 5 5 5 5 ... 400 1 1 1 1 1 ... 500 8 8 8 8 8 ... 900 0 0 0 0... (5 Replies)
Discussion started by: JRodrigoF
5 Replies

10. Shell Programming and Scripting

Removing non matching records

Hi all I have a file with records starting with "Page" has a first column. some of the records have some other junk characters has first column. so pls help me to remove rows which is not having "Page" has a first column. Thanks, Baski (2 Replies)
Discussion started by: baskivs
2 Replies
Login or Register to Ask a Question