awk script to find repeated IP adress from trace file (.tr)


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers awk script to find repeated IP adress from trace file (.tr)
# 1  
Old 03-01-2017
awk script to find repeated IP adress from trace file (.tr)

Code:
+ 8.00747 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 0 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 8.00747 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 0 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.4.5 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 8.01033 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 0 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 16.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 1 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.5 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 16.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 1 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 16.0037 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 1 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 24.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 2 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 24.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 2 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.6 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 24.0037 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 2 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 32.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 3 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 32.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 3 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.7 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 32.0037 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 3 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 40.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 4 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 40.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 4 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 40.0037 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 4 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 48.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 5 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 48.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 5 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 48.0037 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 5 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 56.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 6 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 56.0008 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 6 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 56.0037 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 6 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
+ 64.002 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Enqueue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 7 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
- 64.002 /NodeList/0/DeviceList/0/$ns3::PointToPointNetDevice/TxQueue/Dequeue ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 7 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)
r 64.0048 /NodeList/1/DeviceList/0/$ns3::PointToPointNetDevice/MacRx ns3::PppHeader (Point-to-Point Protocol: IP (0x0021)) ns3::Ipv4Header (tos 0x0 DSCP Default ECN Not-ECT ttl 63 id 7 protocol 17 offset (bytes) 0 flags [none] length: 540 10.1.3.3 > 10.1.2.4) ns3::UdpHeader (length: 520 49153 > 9) Payload (size=512)

# 2  
Old 03-01-2017
Showing us a sample file and giving us a thread title asking for an awk script to find repeated IP address without describing what constitutes the IP address (or addresses) that are to be examined when looking for repeats makes it hard to help you reach your goal. And once a repeat has been found, what do you want to do with it?

What operating system and shell are you using?

What have you tried to solve this problem?

Why do you need to find repeated IP addresses?

With the sample input you have provided, exactly what output are you hoping to produce?
This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 03-01-2017
1...Operating system is linux and more accurately it is ubuntu 14.0.4 in a virtual environment.

2..I have tried awk script for this. i m displaying code below for it.
Code:
BEGIN {
count = 0;
i=0;

start_time=8.0
}
	{
        	event = $1
		time = $2
		node_id = $19
		source_add = $29
		dest_add = $31


	if (dest_add == "10.1.2.4)" && time="56.0037") {
if(start_time>time){
start_time=time+10.0
}
 
      source[i] = $29
}
print "\n\tSource IP address at time =", source[i],"\n\n";
	i++;
}
}
END {
print "\n\tSource IP address at time =", i, "\n\n";
}

3..I'm working on a trace file to get repeated IP source adresss.

4.. i need output as follow:
No. of repeated source IP and destination pair with time slots.

note: i m using Bourne Again shell (bash)

Last edited by Nipa; 03-01-2017 at 04:41 AM.. Reason: need to answer
# 4  
Old 03-01-2017
Showing us code that is not working helps..., but it would help a lot more if you showed us the output that you are trying to produce from the sample input you provided (as requested in my last post).

Please also explain why the time needs to be set to 56.0037 if the 29th field in your input is 10.1.2.4).

Please also explain why your code sometimes prints the 29th field on a line with the label Source IP address at time = and at the end prints the number of lines found in the input file + 1 with the same label.

I don't see anything in your code that makes any attempt to look for duplicated values of any kind.
# 5  
Old 03-01-2017
hey Don Cragun !!

I'm new to awk script and .tr files. The code posted in last post was my trial code. So i request for the appropriate suggestions..

As from .tr file you can see that <10.1.3.7,10.1.2.4> these are the most repeated pair at different time.
So i want it to be displayed like below format.


I want output like this:
Code:
-----------------------------------------------------------
Number of duplicate                      Time(in secs)
 <source,destination> 
-----------------------------------------------------------
<10.1.3.7,10.1.2.4>                           8.00747


Thanks,Smilie

Last edited by rbatte1; 03-01-2017 at 07:06 AM.. Reason: Added CODE tags
# 6  
Old 03-01-2017
In fact, I CAN'T see that 10.1.3.7 is the most repeated pair, nor how and why it should be associated to 8.00747. The most repeated (to me) seems to be 10.1.3.3 > 10.1.2.4, occurring 20 times between 8.00747 and 64.0048. Mayhap you can build on this:

Code:
awk '
        {IX = $29 " > " $31
         sub (/\)/, _, IX)
         CNT[IX]++
         MIN[IX] = $2<MIN[IX] || !MIN[IX]?$2:MIN[IX]
         MAX[IX] = $2>MAX[IX]            ?$2:MAX[IX]
        }
END     {for (c in CNT) printf "%20s %3d %9.5f %9.5f\n", c, CNT[c], MIN[c], MAX[c]
        }
' file
 10.1.3.3 > 10.1.2.4  20   8.00747  64.00480
 10.1.3.7 > 10.1.2.4   1  32.00080  32.00080
 10.1.3.6 > 10.1.2.4   1  24.00080  24.00080
 10.1.3.5 > 10.1.2.4   1  16.00080  16.00080
 10.1.4.5 > 10.1.2.4   1   8.00747   8.00747

# 7  
Old 03-01-2017
hello..

thanks for the code..

Can you please explain this file..

i got error
Code:
awk: 1: unexpected character '''

I used awk -f filename.awk filename.tr for running the code.

cheers!!



Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 03-01-2017 at 08:27 AM.. Reason: Added CODE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to find data in three file and perform replace operation

Have three files. Any other approach with regards to file concatenation or splitting, etc is appreciated If column55(billngtype) of file1 contains YMNC or YPBC then pick the value of column13(documentnumber). Now find this documentnumber in column1(Billdoc) of file2 and grep the corresponding... (4 Replies)
Discussion started by: as7951
4 Replies

2. Shell Programming and Scripting

Find repeated word and take sum of the second field to it ,for all the repeated words in awk

Hi below is the input file, i need to find repeated words and sum up the values of it which is second field from the repeated work.Im trying but getting no where close to it.Kindly give me a hint on how to go about it Input fruits,apple,20,fruits,mango,20,veg,carrot,12,veg,raddish,30... (11 Replies)
Discussion started by: 100bees
11 Replies

3. Shell Programming and Scripting

awk based script to find the average of all the columns in a data file

Hi All, I need the modification for the below mentioned code (found in one more post https://www.unix.com/shell-programming-scripting/27161-script-generate-average-values.html) to find the average values for all the columns(but for a specific rows) and print the averages side by side. I have... (4 Replies)
Discussion started by: ks_reddy
4 Replies

4. Shell Programming and Scripting

Awk based script to find the median of all individual columns in a data file

Hi All, I have some data like below. Step1,Param1,Param2,Param3 1,2,3,4 2,3,4,5 2,4,5,6 3,0,1,2 3,0,0,0 3,2,1,3 ........ so on Where I need to find the median(arithmetic) of each column from Param1...to..Param3 for each set of Step1 values. (Sort each specific column, if the... (5 Replies)
Discussion started by: ks_reddy
5 Replies

5. Shell Programming and Scripting

sort file with email adress

Hi All, I have a file which is "|" pipe delimited. The file has 3 fields. the last field contains email ids which has different host names. I want to sort the file based on host name, which is in 3rd filed and needs to create a file for each host. For example, if out of 1000 records, 10... (6 Replies)
Discussion started by: ace_friends22
6 Replies

6. Shell Programming and Scripting

How to find repeated string in a text file

I have a text file where I need to find the string = ST*850* This string is repetaed several times in the file, so I need to know how many times it appears in the file, this is the text files: ISA*00* *00* *08*925485USNR *ZZ*IMSALADDERSP... (13 Replies)
Discussion started by: cucosss
13 Replies

7. Shell Programming and Scripting

Trying to get an IP adress from a file

This is probably a real n00b question but i`m not able to figure it out. I have a folder of configuration files that contain IP-adresses. The line i`m interested in looks like this: IP_ADDRESS="123.123.123.1123" Some have muliple ip adresses, so the line will look like : ... (5 Replies)
Discussion started by: DaneV
5 Replies

8. Shell Programming and Scripting

To find the repeated name in subject

Hi, I am using a mail application, When I use to check with the command, I will get the outputs as q -s 6128175 (9, 9/6128175) Return-path: minka.bell@abcd.org From: "Minka Bell" <minka.bell@abcd.org> To: <dcrouch@xyz.org>, Subject: Open items with new PICT system Date: Wed,... (12 Replies)
Discussion started by: gsiva
12 Replies

9. Shell Programming and Scripting

To find the IP adress in the log file

Hi, I need to find out the repeated IP address from the apache log file from my box. I did try to come out with the script, but I could not grep out the repeated Ip Address from the error_logs and need to redirect to a file. Can you guys please help me out of this problem. Thanks in... (9 Replies)
Discussion started by: gsiva
9 Replies

10. UNIX for Dummies Questions & Answers

how to find a word repeated in a file

Hi everyone, I have a file in which a word is repeated more than one time and I want to know how many times it is repeated. ex: if i repeated word 'guru' in 10 lines I can get the o/p as: cat filename | grep -c 'guru'. How ever if the word is repeated more than one time, then how can I... (4 Replies)
Discussion started by: gurukottur
4 Replies
Login or Register to Ask a Question