Awk - Compare fields and increment variables


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk - Compare fields and increment variables
# 1  
Old 05-22-2009
Awk - Compare fields and increment variables

Hi,

My first post to this group...

I have a need to to parse a source file which is a capture from a network analyser.

I have two fields that need to be checked:
- Field 7 represents the packet length (an integer), and
Field 4 represents a network address (e.g. 192.168.25.3)

- The first check is to find 2 consecutive lines that have the same integer in Field 7 i.e. the same length. Original file may not always have these lines consecutive though, but I am ok to ignore those lines if it is too difficult to include those.
- Then, once we have these two lines, check the text in Field 4 for these lines and inidicate the value within the text that is 'first' and increment a variable.

What I'm after is to understand how many times address A is first compared to address B.


My expected output from the sample below would be:

"239.25.30.25 is first once" and "239.25.30.26 is first twice.

Even an output like "239.25.30.25 - 1, 239.25.30.26 - 2" would be great.


Example source:


No. Time Source Destination Protocol Info Length
1 20:44:19.525910000 192.168.30.25 239.25.30.25 UDP Source port: dnp Destination port: 20000 94
2 20:44:19.525932000 192.168.30.26 239.25.30.26 UDP Source port: dnp Destination port: 20000 94
3 20:44:19.525989000 192.168.30.26 239.25.30.26 UDP Source port: dnp Destination port: 20000 114
4 20:44:19.526037000 192.168.30.25 239.25.30.25 UDP Source port: dnp Destination port: 20000 114
13 20:44:19.693262000 192.168.30.26 239.25.30.26 UDP Source port: dnp Destination port: 20000 193
14 20:44:19.693295000 192.168.30.25 239.25.30.25 UDP Source port: dnp Destination port: 20000 193



I believe Awk should be able to take of this, but my awk skills are not good enough to come up with something decent.


I hope someone may be able to point me in the right direction.

Thanks,
Mario
# 2  
Old 05-22-2009
Code:
awk '
    $1 ~ /^[0-9]/ {
        if (!last) { last=$12; ip=$4 }
        else
        {
            if (last==$12) ipc[ip]++
            last=0
        }
    }

    END { for (ip in ipc) print ip, ipc[ip] }
' inputfile

produces the following output from your example file:
239.25.30.25 1
239.25.30.26 2
# 3  
Old 05-22-2009
somethng like this you can try

Code:
awk 'BEGIN { prev=0 ; count=1 } { if ( prev==$NF) count++;else { count=1;;prev=$NF } print $4,"-", $12 "- " count}'  File.txt

hope tht the last column is sorted
# 4  
Old 05-22-2009
i couldn't understand your problem(req) fully but i tried this..
Code:
awk 'FNR%2{var=$NF;next}{if(var==$NF){if($4=="239.25.30.26"){v25 += 1}else{v26 += 1}}}END{print "239.25.30.25 - "v25"\n239.25.30.26 - "v26}' filename

# 5  
Old 05-22-2009
Thanks

That's great, you've given me what I needed!!

Thank you so much for replying.


Best Regards,
Mario
# 6  
Old 05-22-2009
vidyadhar85,

To answer your question, I have a text file containing data from a network capture.

The data is duplicated (on purpose) and is sent to two destinations (multicast addresses). Sometimes data for one destination is received first, other times data to the other destination is first.

I'm trying to work out which destination is usually first depending on the sample I capture.

I've just seen that depending on which awk string I run from the replies above, I get different output / results from the replies received, so will probably still need to verify which gives me the most correct answer for a particular sample.


I think cambridge's script works best for me so far.


Thanks again.

Mario
# 7  
Old 05-22-2009
Note that my script only works with consecutive lines. It gets more convoluted if you want to handle other cases, as you'll need to decide how many lines should be allowed between each for it to be a valid sample.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 files and find missing fields awk

Hello experts! I have 2 files. file1 is a list file containing uniquely names. e.g.: name1 number number name2 number number name5 number number name10 number number ... file2 is a data file arbitrary containing the names of file1 in paragraphs separated by "10" e.g. name4 ... (3 Replies)
Discussion started by: phaethon
3 Replies

2. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

I'm trying to compare 2 files for differences in a selct number of fields. When differnces are found it will write the whole record of the second file including appending '|C' out to a delta file. Each record will have 20 fields, but only want to do comparison of 1st 15 fields. The 1st field of... (7 Replies)
Discussion started by: sljnk
7 Replies

3. Shell Programming and Scripting

awk question ? set 2 variables by matching fields

Hello, I'm trying to get the TOP and BASE numbers printed out File looks like this: 2300 CAR # 2300 is the TOP 2310 CAR 2335 CAR 2455 CAR # 2455 is the BASE 1000 MOTOR # 2455 will become this TOP 2000 MOTOR 3000 MOTOR 4000 MOTOR # 4000 is the BASE 2345 BIKE # 4000... (8 Replies)
Discussion started by: charlieglen
8 Replies

4. UNIX for Dummies Questions & Answers

Compare values of fields from same column with awk

Hi all ! If there is only one single value in a column (e.g. column 1 below), then return this value in the same output column. If there are several values in the same column (e.g. column 2 below), then return the different values separated by "," in the output. pipe-separated input: ... (11 Replies)
Discussion started by: lucasvs
11 Replies

5. Shell Programming and Scripting

awk (or other) script that assigns fields from a line to multiple variables

Hey all, Unfortunately I have only basic knowledge of awk and/or scripting. If I have a file with lines that can look similar to this: Name=line1 Arg1=valueA Arg2=valueB Arg3=valueC Name=line2 Arg1=valueD Name=line3 Arg1=valueE Arg3=valueF Name=line4 Arg2=valueG ... (4 Replies)
Discussion started by: Rike255
4 Replies

6. Shell Programming and Scripting

using awk compare two variables in bash

ok this is probably going to turn out to be something really stupid but i've tried to use the following command in a script but the output is just a blank screen and i have to use Ctrl c to exit it. awk 'BEGIN {printf "%.2f\n", '${bashArray}'>='$Variable' {print '${bashArray}'}}' the command... (2 Replies)
Discussion started by: zagreus360
2 Replies

7. Shell Programming and Scripting

awk to compare diff output by fields

Diff output as follows: < AAA BBB CCC DDD EEE 123 > PPP QQQ RRR SSS TTT 111 > VVV WWW XXX YYY ZZZ 333 > AAA BBB CCC DDD EEE 124 How can i use awk to compare the last field to determine if the counter has increased, and need to ensure that the first 4 fields must have the same... (15 Replies)
Discussion started by: ux4me
15 Replies

8. Shell Programming and Scripting

AWK Compare files, different fields, output

Hi All, Looking for a quick AWK script to output some differences between two files. FILE1 device1 1.1.1.1 PINGS device1 2.2.2.2 PINGS FILE2 2862 SITE1 device1-prod 1.1.1.1 icmp - 0 ... (4 Replies)
Discussion started by: stacky69
4 Replies

9. Shell Programming and Scripting

Compare fields in 2 files using AWK

Hi unix gurus, I have a urgent requirement, I need to write a AWK script to compare each fields in 2 files using AWK. Basically my output should be like this. file1 row|num1|num2|num3 1|one|two|three 2|one|two|three file2 row|num1|num2|num3 1|one|two|three 2|one|two|four ... (5 Replies)
Discussion started by: rashmisb
5 Replies

10. Shell Programming and Scripting

Need awk script to compare 2 fields in fixed length file.

Need a script that manipulates a fixed length file that will compare 2 fields in that file and if they are equal write that line to a new file. i.e. If fields 87-93 = fields 119-125, then write the entire line to a new file. Do this for every line in the file. After we get only the fields... (1 Reply)
Discussion started by: Muga801
1 Replies
Login or Register to Ask a Question