Sponsored Content
Full Discussion: awk NR issue
Top Forums Shell Programming and Scripting awk NR issue Post 302570440 by jamie_123 on Thursday 3rd of November 2011 09:51:30 AM
Old 11-03-2011
awk NR issue

Hi guys,

i am trying to analyze a text file using awk and am not able to solve this issue.

This is the piece of code that I have written

Code:
BEGIN {
        ## Time to count MACs -> 5 seconds.
        TIME_LIMIT = 5;
	   k = 50000;
}

## For every line.
{
       time_in_seconds = $1

       mac = $2;
     
       if ( mac_pair[mac] ) {
               diff = time_in_seconds - mac_pair[mac];
		##print diff
                if ( diff > TIME_LIMIT ) {
                        ++counter;
                        printf "%s\n", "Incrementing counter: " $0;
                } 
        } else {
                ## Increment counter for first time of this MACs.
                ++counter;
                printf "%s\n", "Incrementing counter: " $0;
        }

      mac_pair[mac] = time_in_seconds;
## NEW ROUTINE
	if ( NR = k){
	remove_time = time_in_seconds-5;
	##print remove_time
	for (x in mac_pair)
	  { if (mac_pair[x] < remove_time ) 
	    { delete mac_pair[x]
	      print x}
	  }
	k = k + 50000;
	}
}

END {
        printf "%s = %d\n", "Counter is:", counter;
}

The code goes through lines, matches and increments a counter if there is no match for the previous 5 seconds. The issue I m facing is i have to run this over huge files and my ram gets exhausted. That is why I have included a new routine which will discard packets after a set number of lines have been processed.

If i try to print NR before and after it runs this routine, i see it gets altered. Also the routine within if ( NR = k) runs for every line. I am not sure why.

here are some sample input

Code:
1319170474.165379 164:186:219:250:181:93:255:255:255:255:255:255:130.192.9.212:130.192.9.255:55549:1947
1319170474.308849 0:13:41:11:173:255:1:0:94:0:0:13:192.168.100.129:224.0.0.13:8192:33198
1319170474.543025 0:224:129:88:169:186:255:255:255:255:255:255:192.168.10.126:192.168.10.255:631:631
1319170474.546619 0:25:209:27:17:125:255:255:255:255:255:255:130.192.9.131:255.255.255.255:631:631
1319170474.165379 164:186:219:250:181:93:255:255:255:255:255:255:130.192.9.212:130.192.9.255:55549:1947
1319170474.308849 0:13:41:11:173:255:1:0:94:0:0:13:192.168.100.129:224.0.0.13:8192:33198
1319170474.543025 0:224:129:88:169:186:255:255:255:255:255:255:192.168.10.126:192.168.10.255:631:631
1319170474.546619 0:25:209:27:17:125:255:255:255:255:255:255:130.192.9.131:255.255.255.255:631:631

help me out Smilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk issue

Hi, # grep "^Listen" httpd.conf | awk '{print $2}' FrontEnd_1_IP:8081 FrontEnd_2_IP:8081 8081 8082 8083 # I need to get the values one at a time but I just can't manage to do that. Thanks, Bianca (20 Replies)
Discussion started by: potro
20 Replies

2. Shell Programming and Scripting

Awk issue

Can someone please explain below code. $LIST|awk ' /^$/ { next } substr($0,1,4)=="Exiting" { mk = 1; next } mk==1 { print $3,$7,$10,$14; exit } Cheers, gehlnar (5 Replies)
Discussion started by: gehlnar
5 Replies

3. Shell Programming and Scripting

Issue in awk

In the following code, Im trying to imbed many statements in a single awk statement. But it gives an error on that, for i in `less usage_types_dwh.txt` do cd /u01/app/evident/analysis_lab/usg_type grep $i svc_type.txt | head -1 | awk 'BEGIN {FS=","} {print $1 "==" $2 ":" $3 ":" $4;... (2 Replies)
Discussion started by: alishehzadpaul
2 Replies

4. Shell Programming and Scripting

Issue with AWK

I have this input file 0FB7,1083,Synchronized,FriNov121655,2816_7RAID5,05F:1_10F:1,10000000NoneNone,DC_db00p01 0FB7,1150,Split,MonApr180658,2816_7R5GC,N/A,N/A,N/A 06C4,0710,Synchronized,WedMar91105,2816_7RAID5,04E:1_11E:1,10000000NoneNone,DL_nb00p25... (1 Reply)
Discussion started by: greycells
1 Replies

5. Shell Programming and Scripting

AWK Issue

Hey, this is my code, cat $fulltrpath | while read line do inputfile=$(sed 1q $fulltrpath | awk '{ FS = "\t"; print $2$1}') outputpath=$(sed 1q $fulltrpath | awk '{ FS = "\t"; print $3 }') echo $inputfile echo $outputpath cp $inputfile $outputpath let path++ done if i... (1 Reply)
Discussion started by: inshafccna
1 Replies

6. Shell Programming and Scripting

issue trying to use awk

Hi Gurus, I am facing a similar issue usiung an awk command. Below is my requirement: ---DATA--- A;F;G A;D;E A;D;E B;Z;P C;Z;Q Expected: A F<TAB>G D<TAB>E D<TAB>E B D<TAB>E (1 Reply)
Discussion started by: rajangupta2387
1 Replies

7. Shell Programming and Scripting

awk issue

Hi all, i am trying to use below command to see the output of hardware inventory, but i only see 2 first line no output of the command. awk '/Hardware/ {print $0}' XXX_result.txt Hardware inventory: Hardware inventory: any idea how to see whatever is under hardware inventory. i... (11 Replies)
Discussion started by: Jared
11 Replies

8. Shell Programming and Scripting

Variable value substitution issue with awk command issue

Hi All, I am using the below script which has awk command, but it is not returing the expected result. can some pls help me to correct the command. The below script sample.ksh should give the result if the value of last 4 digits in the variable NM matches with the variable value DAT. The... (7 Replies)
Discussion started by: G.K.K
7 Replies

9. Shell Programming and Scripting

awk - issue to get the right IP

On AIX 5.3 and AIX 6.1, I have this script for checking printers being pingable or not. for i in `lsallq` do echo "Queue Name: " $i echo "----------------------------------------" for j in `lsallqdev -q $i` do echo " Device Name:" $j hname=`echo... (3 Replies)
Discussion started by: Daniel Gate
3 Replies

10. UNIX for Dummies Questions & Answers

Help with awk issue

OK, so I am trying to use awk as a method of accessing a table stored in a file to then provide the capability of a look up table. The table is stored in a file named "/Users/jhaney/Desktop/assetTypeMapping.tsv" and looks like this: aCategory aLetter aNumber AssetCat1 A 123 ... (10 Replies)
Discussion started by: jhaneyzz
10 Replies
netmasks(4)                                                        File Formats                                                        netmasks(4)

NAME
netmasks - network mask database SYNOPSIS
/etc/inet/netmasks /etc/netmasks DESCRIPTION
The netmasks file contains network masks used to implement IP subnetting. It supports both standard subnetting as specified in RFC-950 and variable length subnetting as specified in RFC-1519. When using standard subnetting there should be a single line for each network that is subnetted in this file with the network number, any number of SPACE or TAB characters, and the network mask to use on that network. Network numbers and masks may be specified in the conventional IP `.' (dot) notation (like IP host addresses, but with zeroes for the host part). For example, 128.32.0.0 255.255.255.0 can be used to specify that the Class B network 128.32.0.0 should have eight bits of subnet field and eight bits of host field, in addition to the standard sixteen bits in the network field. When using variable length subnetting, the format is identical. However, there should be a line for each subnet with the first field being the subnet and the second field being the netmask that applies to that subnet. The users of the database, such as ifconfig(1M), perform a lookup to find the longest possible matching mask. It is possible to combine the RFC-950 and RFC-1519 form of subnet masks in the net- masks file. For example, 128.32.0.0 255.255.255.0 128.32.27.0 255.255.255.240 128.32.27.16 255.255.255.240 128.32.27.32 255.255.255.240 128.32.27.48 255.255.255.240 128.32.27.64 255.255.255.240 128.32.27.80 255.255.255.240 128.32.27.96 255.255.255.240 128.32.27.112 255.255.255.240 128.32.27.128 255.255.255.240 128.32.27.144 255.255.255.240 128.32.27.160 255.255.255.240 128.32.27.176 255.255.255.240 128.32.27.192 255.255.255.240 128.32.27.208 255.255.255.240 128.32.27.224 255.255.255.240 128.32.27.240 255.255.255.240 128.32.64.0 255.255.255.192 can be used to specify different netmasks in different parts of the 128.32.0.0 Class B network number. Addresses 128.32.27.0 through 128.32.27.255 have a subnet mask with 28 bits in the combined network and subnet fields (often referred to as the subnet field) and 4 bits in the host field. Furthermore, addresses 128.32.64.0 through 128.32.64.63 have a 26 bits in the subnet field. Finally, all other addresses in the range 128.32.0.0 through 128.32.255.255 have a 24 bit subnet field. Invalid entries are ignored. SEE ALSO
ifconfig(1M), inet(7P) Postel, Jon, and Mogul, Jeff, Internet Standard Subnetting Procedure, RFC 950, Network Information Center, SRI International, Menlo Park, Calif., August 1985. V. Fuller, T. Li, J. Yu, K. Varadhan, Classless Inter-Domain Routing (CIDR): an Address Assignment and Aggregation Strategy, RFC 1519, Network Information Center, SRI International, Menlo Park, Calif., September 1993. T. Pummill, B. Manning, Variable Length Subnet Table For IPv4, RFC 1878, Network Information Center, SRI International, Menlo Park, Calif., December 1995. NOTES
/etc/inet/netmasks is the official SVr4 name of the netmasks file. The symbolic link /etc/netmasks exists for BSD compatibility. SunOS 5.10 7 Jan 1997 netmasks(4)
All times are GMT -4. The time now is 02:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy