Are you using awk's native lshift/rshift/or functions OR did you provide your own implementations (as previously discussed)?
Yes I was using awk native functions. Here is the script:
Code:
function range2cidr(ipStart, ipEnd, result, bits, mask, newip) {
bits = 1
mask = 1
while (bits < 32) {
newip = or(ipStart, mask)
if ((newip>ipEnd) || ((lshift(rshift(ipStart,bits),bits)) != ipStart)) {
bits--
mask = rshift(mask,1)
break
}
bits++
mask = lshift(mask,1)+1
}
newip = or(ipStart, mask)
bits = 32 - bits
result = (result)?result ORS dec2ip(ipStart) "/" bits : dec2ip(ipStart) "/" bits
if (newip < ipEnd) result = range2cidr(newip + 1, ipEnd,result)
return result
}
# convert dotted quads to long decimal ip
# int ip2dec("192.168.0.15")
#
function ip2dec(ip, slice) {
split(ip, slice, /[.]/)
return (slice[1] * 2^24) + (slice[2] * 2^16) + (slice[3] * 2^8) + slice[4]
}
# convert decimal long ip to dotted quads
# str dec2ip(1171259392)
#
function dec2ip(dec, ip, quad) {
for (i=3; i>=1; i--) {
quad = 256^i
ip = ip int(dec/quad) "."
dec = dec%quad
}
return ip dec
}
function sanitize(ip) {
split(ip, slice, /[.]/)
return slice[1]/1 "." slice[2]/1 "." slice[3]/1 "." slice[4]/1
}
BEGIN{
FS=" - |-|:"
}
# sanitize ip's
!/^#/ && NF {
f1= sanitize($(NF-1))
f2= sanitize($NF)
print range2cidr(ip2dec(f1), ip2dec(f2))
}
END {print ""}
After you mentioned native functions lshift/rshift/or I decided to try the other previously "discussed implementations" again (see post #7) bit_lshift/bit_rshift,bit_or. That particular script produced wrong output since it didn't have the changes you made to correct the code. I replaced lshift/rshift/or with bit_lshift/bit_rshift,bit_or in the script and awk does work. No segmentation fault. See the new script below which produces accurate results:
Code:
function range2cidr(ipStart, ipEnd, result, bits, mask, newip) {
bits = 1
mask = 1
while (bits < 32) {
newip = bit_or(ipStart, mask)
if ((newip > ipEnd) || ((bit_lshift(bit_rshift(ipStart,bits),bits)) != ipStart)) {
bits--
mask = bit_rshift(mask,1)
break
}
bits++
mask = bit_lshift(mask,1)+1
}
newip = bit_or(ipStart, mask)
bits = 32 - bits
result = (result)?result ORS dec2ip(ipStart) "/" bits : dec2ip(ipStart) "/" bits
if (newip < ipEnd) result = range2cidr(newip + 1, ipEnd,result)
return result
}
# convert dotted quads to long decimal ip
# int ip2dec("192.168.0.15")
#
function ip2dec(ip, slice) {
split(ip, slice, /[.]/)
return (slice[1] * 2^24) + (slice[2] * 2^16) + (slice[3] * 2^8) + slice[4]
}
# convert decimal long ip to dotted quads
# str dec2ip(1171259392)
#
function dec2ip(dec, ip, quad) {
for (i=3; i>=1; i--) {
quad = 256^i
ip = ip int(dec/quad) "."
dec = dec%quad
}
return ip dec
}
# Bitwise OR of var1 and var2
function bit_or(a, b, r, i, c) {
for (r=i=0;i<32;i++) {
c = 2 ^ i
if ((int(a/c) % 2) || (int(b/c) % 2)) r += c
}
return r
}
# Rotate bytevalue left x times
function bit_lshift(var, x) {
while(x--) var*=2;
return var;
}
# Rotate bytevalue right x times
function bit_rshift(var, x) {
while(x--) var=int(var/2);
return var;
}
function sanitize(ip) {
split(ip, slice, /[.]/)
return slice[1]/1 "." slice[2]/1 "." slice[3]/1 "." slice[4]/1
}
BEGIN{
FS=" - |-|:"
}
# sanitize ip's
!/^#/ && NF {
f1= sanitize($(NF-1))
f2= sanitize($NF)
print range2cidr(ip2dec(f1), ip2dec(f2))
}
END {print ""}
Benchmarks processing a file containing approximately 236K IP address ranges with the new script. Same benchmarks as before (see post #7).
ipcacl 15 min
mawk 59 sec
gawk 1 min 45 sec
awk 2 min 46 sec
Appears the issue may have been the implementation of the native functions lshift/rshift/or in the version of awk that OpenBSD uses. I would recommend that anyone that uses gawk use the first script with the native functions since the script is over twice as fast.
vgersh99 when I get the chance I'll try the debugging you suggested.
Dear Srs :-)
I'm looking for a shell script, that given a network in CIDR format it lists all IPs, for example:
Preferredly a shell script, but a Perl, Python, C, etc.. is also welcome :-)
I have been looking in sipcalc, ipcalc, etc.. options but this feature is not implemented :-(
... (10 Replies)
Hi,
Please anyone help to achive this using perl or unix scripting .
This is date in my table 20090224,based on the date need to check the files,If file exist for that date then increment by 1 for that date and check till max date 'i.e.20090301 and push those files .
files1_20090224... (2 Replies)
Hello everybody,
I'm coding a network program and i need it to "understand" ip ranges, but i don't know how to make to parse an IP CIDR range, let's say "172.16.10.0/24" to work with the specified IP range.
I've found a program which does it, but i don't understand the code. Here is the... (3 Replies)
I have a list of about 200,000 lines in a text file that look like this:
1 1 120
1 80 200
1 150 270
5 50 170
5 100 220
5 300 420
The first column is an identifier, the next 2 columns are a range (always 120 value range)
I'm trying fill in the values of those ranges, and remove... (4 Replies)
Hi,
I have two files
file1 chr1_22450_22500
chr2_12300_12350
chr1_34500_34550
file2 11000_13000
15000_19000
33000_44000
If the file 1 ranges fall between file2 ranges then assign the value of file2 in column 2 to file1
output:
chr2_12300_12350 11000_13000
chr1_34500_34550 ... (7 Replies)
Looking for a simple way to convert ranges to a numerical sequence that would assign the original value of the range to the individual numbers that are on the range.
Thank you
given data
13196-13199 0
13200 4
13201 10
13202-13207 3
13208-13210 7
desired... (3 Replies)
Hi all, I would appreciate if someone could share how to convert CIDR notation to netmask and vice versa.
The value below is just an example. it could be different numbers/ip addresses.
Initial Output, let say file1.txt
Final Output, let say file2.txt (3 Replies)
Hi,
Recently I had to convert a 280K lines of ip ranges to the CIDR notation and generate a file to be used by ipset (netfilter) for ip filtering.
Input file:
000.000.000.000 - 000.255.255.255 , 000 , invalid ip
001.000.064.000 - 001.000.127.255 , 000 , XXXXX
001.000.245.123 -... (10 Replies)
Weary of seeing our load average go up to 50+, I just did a major block on these networks (stats over a less than 20 min interval):
https://www.unix.com/members/1-albums215-picture866.png (3 Replies)