AWK aggregate records


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users AWK aggregate records
# 1  
Old 08-26-2008
AWK aggregate records

Hy all,

I have a problem...can some one help me...


I have a file of records sort:
30|239|ORD|447702936929 |blackberry.net |20080728|141304|00000900|2|0000000000000536|28181|0000000006|0000000001|10|1

30|239|ORD|447702936929 |blackberry.net |20080728|141304|00000340|2|0000000000007076|29211|0000000024|0000000007|10|1

30|239|ORD|447702936929 |blackberry.net |20080728|141305|00000900|3|0000000000001568|28181|0000000002|0000000002|10|1

What i want to do is aggregate this records by the following conditions

In tha case that the record have the same key ($1, $4, $5) and the field $9 it's =2:

VAR= $7 + $8 + 2

Next i go to the next record and:
if field $7 <= VAR, then i aggregate this two records

else...next record...

the last record need to have the $9 = 3, because itīs indicate that my leg is closed.

i just can aggregate when i have a multiple records with $9=2 and the last record with $9=3.

the result of the example it's something like this:

30|239|ORD|447702936929 |blackberry.net |20080728|141304|00002140|1|0000000000000536|28181|0000000006|0000000001|10|1

When i aggregate the record, i sum the field $8 and set the $9 with 1.
# 2  
Old 08-27-2008
I need a little more info. After aggregating, what do you want to print out? When there's no aggregation, do you want to print out anything?

Here's a start:
Code:
awk -F\| '
  VAR && $7 <= VAR { AGGREGATE HERE; VAR=0; next; }
  $9 == 2 && $1 == $4 && $1==$5 { VAR=$7+$8+2; next; }
  { PRINT OTHER LINES HERE }
'

# 3  
Old 08-27-2008
hello....

this is what i deed....

I donīt know if itīs the best idea...but works....

your sugestion do something like this?

BEGIN {
FS="|";
c1 = -1;
c4 = -1;
c5 = -1;
c8 = 0;
time = 0;
}
{
if (($1 == c1) && ($4 == c4) && ($5 == c5))
{
# accumulate value $8
if ($7 <= time)
{
c8 = c8 + $8
# updates variable time
time = $7 + $8 + 2
}
#write to output
if ($9 == 4)
{
print c1"|"c2"|"c3"|"c4"|"c5"|"c6 "|"c7 "|"c8"|1|"c10"|"c11"|"c12"|"c13"|"c14"|"c15 >> file_complete
close(file_complete)
#restart variables
time = 0
c8 = 0
next
}
}
else
{ # the record don't have the same key has the last record
if (c1 == -1)#when reads the 1st line of the file only keeps the relevant fields of the first record
{
if ($9 == 2) #save the fields from the beginning of the call, to put in output
{
c1 = $1
c2 = $2
c3 = $3
c4 = $4
c5 = $5
c6 = $6
c7 = $7
c8 = $8
c10 = $10
c11 = $11
c12 = $12
c13 = $13
c14 = $14
c15 = $15
time = $7 + $8 + 2
next
}
}
if (c1 != -1)#print output of the previous
{
print c1"|"c2"|"c3"|"c4"|"c5"|"c6"|"c7"|"c8"|1|"c10"|"c11"|"c12"|"c13"|"c14"|"c15 >> file_complete
close(file_complete)
# save the current record if is the beginning of the call
if ($9 == 2)
{ # save the fields from the beginning of the call, to put in output
c1 = $1
c2 = $2
c3 = $3
c4 = $4
c5 = $5
c6 = $6
c7 = $7
c8 = $8
c10 = $10
c11 = $11
c12 = $12
c13 = $13
c14 = $14
c15 = $15
time = $7 + $8 + 2
}
}
}
}
END {
#print the output for the case that not caught the last registration with closing ($ 9 = 4, but caught with $ 9 = 3)
if ($9 == 3)
{
print c1 "|" c2 "|" c3 "|" c4 "|" c5 "|" c6 "|" c7 "|" c8 "|1|" c10 "|" c11 "|" c12 "|" c13 "|" c14 "|" c15 >> file_complete
close(file_complete)
}
}
# 4  
Old 08-27-2008
It looks like you understand what you're doing.

You can save yourself a lot of time by setting OFS to "|" and just doing print $0 instead of every variable independently. Also, while you can do it in one big awk program, its easier -- syntactically -- to break it up into multiple awk programs. Now, I don't mean multiple instances of awk. Every awk invocation can contain multiple programs, like this:
Code:
awk 'BEGIN { FS="/" }  <condition1> { program1... } <condition2> { program2}'

If condition1 matches, program1 will run. If condition2 matches, program2 will run (regardless of whether program1 ran or not). If you want program1 to stop processing and go to the next line, you use "next;". If you want program2 to get the next line now, you can do "getline;". (That might be GNU specific.)

Also, when you post on this forum, it helps to embed your code in [code] tags. It keeps the spacing.
# 5  
Old 08-27-2008
sorry....iīm new in this things.....

About your sugestion, it seams very good ....i will try split my AWK in another AWKs....

thanks...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Aggregate data within the file

Guys, I need to roll up data within the file and build a new file with the output and the same format as the original file. The data should be rolled up for each unique combination of ord,line,date and hour.. The last column appr is always " " Below is the format Original File: ... (8 Replies)
Discussion started by: venky338
8 Replies

2. Solaris

IPMP over aggregate in Solaris 11

hi all, i start with solaris 11 and i am disapointed by the change on ip managing. i want to set a ipmp over tow aggregate but i dont find any doc and i am lost with the new commande switch1 net0 aggregate1 | net1 aggregate1 |-----| |... (1 Reply)
Discussion started by: sylvain
1 Replies

3. Shell Programming and Scripting

Common records using AWK

Hi, To be honest, I am really impressed and amazed at the pace I find solutions for un-solved coding mysteries in this forum. I have a file like this input1.txt x y z 1 2 3 a b c 4 -3 7 k l m n 0 p 1 2 a b c 4 input2 x y z 9 0 -1 a b c 0 6 9 k l m 8 o p 1 2 a f x 9 Output... (9 Replies)
Discussion started by: jacobs.smith
9 Replies

4. Shell Programming and Scripting

simple aggregate task

Hi experts, I need an help on the task below. INPUT: values separated by the tab,first row is the header 20110609 AS A 300.5000 20110609 AS R 200.5000 20110609 BR A 111.5000 20110609 BR R 222.5000 20110610 AS A 100.5500 20110610 AS ... (2 Replies)
Discussion started by: hernand
2 Replies

5. Shell Programming and Scripting

Skip first and last n records with awk

Hi, I have an awk code that reads an input file, checks the 4th column and tells if its fine. #!/bin/ksh { if ($4 == 0) print "fine" else print "some problem" }' FILENAME My problem is that, I dont want to check the first 3 and last 3 lines. This can be hard coded by using BEGIN and END... (9 Replies)
Discussion started by: gotam
9 Replies

6. Shell Programming and Scripting

Awk Multiple Files & Aggregate

file 1: 70|236|PPS|0501011818|mms|20090706|001452|00000024|2|0000000000000000|00000|0000000000|0000000000|40948000|1 70|236|PPS|0501020076|mms|20090705|204408|00000019|2|0000000000000000|00000|0000000000|0000000000|40947930|1... (3 Replies)
Discussion started by: magedfawzy
3 Replies

7. IP Networking

Aggregate two internet connections

Hi I have a question related to load balancing.I have two separate internet connections with 2Mbps speed and i would like to aggregate this two connections intro one connection with 4Mbps.Is it possible to do that, to put a Linux or Unix machine as a gateway?I read some stuff to split the... (3 Replies)
Discussion started by: tafil
3 Replies

8. UNIX Desktop Questions & Answers

Aggregate title to an archive.log

Hello how are you, i have a question i have a file ale.log and i want to agregate a title and later a space when the text is over and put another title (when the text is over) how can i do this? thank you Example Last ------>(Title) i want to agregate pupu pupu pupu pupu... (1 Reply)
Discussion started by: enkei17
1 Replies

9. Shell Programming and Scripting

awk - Number of records

Hi, Is it possible to find the total number of records processed by awk at begining. NR gives the value at the end. Is there any variable available to find the value at the begining? Thanks ---------- Suman (1 Reply)
Discussion started by: suman_jakkula
1 Replies

10. UNIX for Dummies Questions & Answers

aggregate ethernet ports under Solaris

I have been looking for info on how to aggregate 2 ore 3 NIC's into into one big pipe. Any advice would be appreciated. -Chuck (4 Replies)
Discussion started by: 98_1LE
4 Replies
Login or Register to Ask a Question